4:4:4 10bit single CMOS HD project

Les Dit · July 17th, 2004, 12:26 PM

Ben,
In case you haven't seen this paper, take a look !
He covers many of the methods, and has code to try them out.
The code is for MATLAB but it's easy to see what it does.

I like the variable gradient method, but it's the slowest.
But what do we care, it's quality we want, and it's still faster than taking film to the lab !

He covers the computational cost on the methods as well, to help see what they take.

www-ise.stanford.edu/~tingchen/main.htm

-Les

<<<-- Originally posted by Ben Syverson : Jason and David,

Yeah, the current implementation of linBayer is, as the name suggests, purely linear. I do this for two reasons: it's fast, and it's lossless. By lossless I mean that the original values are maintained, so if you wanted to, you could take the RGB image and recover 100% of the original RAW data.

I'm also developing a higher-quality algorithm, probably based on splines.

I'll keep you posted!

- ben -->>>

David Newman · July 17th, 2004, 12:36 PM

<<<-- Originally posted by Jason Rodriguez : Why would the Mac version be unlikely unless you release code (do you mean open source it?)?? -->>>

We don't have any Macs or a development enviroment for a Mac. At least for today all CineForm tools will be PC based.

Jason Rodriguez · July 17th, 2004, 12:37 PM

Question for Obin, Steve:

Based on some new bayer interpolation stuff that's recently become available, how much would it cost to buy the equipment for shooting a short film with this stuff?

I'm assuming you'd need the SI-1300

A PC to attach to with card and capture software,

Hard-drives (of course :-)

Software for editing, color-correction (I already have that).

A lens.

Can XCAP be used on a short film? Do you need Stream-pix as well to get a RAW bayer image (no interpolation, just a black-and-white checker patterned image)? Do you also have to mount the camera inside another camera like you did Obin?

And lastly,

Have the rolling shutter artifacts been "solved"??

Do we even have sound-sync?

I'm really starting to entertain the notion of shooting a short film with one of these cameras. I've got the script, and I'm ready to go if the problems have been sorted out with the logistics of the camera (rolling shutter, and that weird vertical smearing I've seen in some of Obin's stuff). Altough depending on pricing, I may want to see what's coming around the corner (even Kinetta stuff-but that would be through rental).

Hey Obin, could you check an make sure it's not the lens making those smearing patterns?

I guess I'll think about it now, and see what happens when the Altasens stuff hits the market.

BTW, Mini-ITX isn't going to work for this stuff, is it?

Ben Syverson · July 17th, 2004, 12:39 PM

Thanks Les!

I've read the paper, but I'm going a different route. I'm developing a logic-based (as opposed to mathematic) de-zippering routine. I'll posts some results in the next hour or so.

- ben

Giroud Francois · July 17th, 2004, 01:39 PM

actually it seems this post is gone on another topic (compression and software) but i just take a look on the imperx web site and they get a nice chip and a pcmcia camera link card that could help to build a very compact/low power design.

Ben Syverson · July 17th, 2004, 01:51 PM

Giroud,

I looked with great interest at the Imperx products, but once I got the quote sheet, my interest waned. Their cameras start at $4600 for anything over 1004x1004 in color... In contrast, I'm putting my entire rig together for about $2500, including camera, laptop, high speed HD and a brand new high resolution lens from Edmunds Optical. Of course, it remains to be seen if it will actually all work. :)

- ben

Jason Rodriguez · July 17th, 2004, 02:17 PM

Found a new Pentium M-based embedded product coming to market soon.

http://www.kontron.com/products/pdpr...kps=1082&kp=75

This one will have 64-bit PCI-X slots through a special riser card. Also has dual SATA interface and an IDE interface for a boot drive.

Steve Nordhauser · July 17th, 2004, 02:51 PM

Jason on smearing:
That smearing with oversaturated pixels is a Micron problem (all Micron 1.3Mpix cameras will have it). Definitely not the lenses. I bugged them about it and they said run slower, which negates what we want to do about rolling shutter. I was told (but haven't seen) that the 3.2Mpix Micron doesn't do that as much - different pixel architecture. The 3.2Mpix should be able to do both 720p and 1920x1080@24fps, 10 bit. If Obin and Rob want to upgrade in about a month I'll work it out with them. The SI-3300 will have less sensitivity though.

Jason on recording:
To do what you want to do as a minimalist, you can use our standard camera, c mount lens, direct-to-disk recording software (streampix will beat xcap for this) and then your own processing chain. Rob or Obin might have the recording thing done, but maybe not in time for you.

Giroud and Ben on Imperx:
They have the only Cardbus cameralink frame grabber that I know of. If there is a request, I can get them to qualify our cameras (maybe the SI-3300 first) on it but you are at the mercy of their software support. This is the joy of an industry standard. They can use our cameras.

Jason Rodriguez · July 17th, 2004, 03:10 PM

Does the Altasens have verticle pixel smearing?

Obin Olson · July 17th, 2004, 03:22 PM

jason I am working on design for a standalone camera very "Kinetta" like but simple and easy to use ...do you have any idea what that new microboard chipset is? i need hardware that will run with Linux for the UI of this stand-alone Rig ;)

I jsut looked at that board page looks like, 855GME, 6300ESB(Hance Rapids) chipsets

anyone know if that is a standard chipset? or?

I think that may be the Perfect Board for a camera!!

My brother is a CAD/FX guy.. I will start working out the system design soon...fun fun fun!

Ben Syverson · July 17th, 2004, 03:27 PM

Hey,

The new version of linBayer is done for now. I'll come back to it a bit later. Here's a comparison.

Old linBayer image

New linBayer algorithm

New linBayer algorithm (aggressive de-zippering)

The aggressive de-zippering option is great for random fine detail like text, but it can introduce stray pixel artifacts on strong, fine horizontal and vertical lines. For those images, leaving aggressive de-zippering off will be the best compromise.

Best of all, there is no computationally intensive math. It's just about exactly as fast as linear interpolation. Also, the process is still lossless. All of the original pixel values are untouched, which means you'll be able to re-do the de-bayer process later if a better algorithm comes along.

I've posted the new build of the mac version to the linBayer website. The PC version will follow a bit later. Right now I have to go to the lake and play scrabble.

- ben

Jason Rodriguez · July 17th, 2004, 03:30 PM

According to the info page:

Up to 2.4GHz Pentium M processors using Intel 855GME, 6300ESB(Hance Rapids) chipsets

BTW Obin, are you still going to be useing the Microns? Right now it seems as though that veritcal smear problem is a real show-stopper since it really limits your dynamic range. Steve, I know you said that the newer chips is going to have "less" of this problem, but if it's still a problem nonetheless, hmm, then I guess the Altasens will be worth waiting around for. I mean I think it would be silly to pay $3000 for a camera and then have a major artifact to deal with.

One more thing though, is this problem ONLY when you oversaturate the chip, or just when you have a bright area?

BTW Obin,

remember that there's a riser card needed for the 64-bit PCI-X slot, so that's extra space that you don't see on that board. I'm assuming that the card will fit over the top (so you'll have to find a place to mount the hard-drives somewhere else).

Another thing Obin, check out Ben's plug-in when he gets the PC version up. The new version is very good, you'll be straining your eyes to see any artifacting. And the nice thing is it works now, not five months or a year from now :-)

P.S. that wasn't mean to be a jab at anyone, just saying this a great solution for getting up and going right away.

Les Dit · July 17th, 2004, 03:53 PM

Obin, do you have an JVC HD-10 you can use for tests ?
-Les

Obin Olson · July 17th, 2004, 04:42 PM

the AWESOME thing about that board is it's got sata AND pci-x !! we can get 4 sata drives and a pci-x grabber and the datarates would be HUGE! I bet 60fps from the 1080p chip at 10bit ;0 ?? can raid the 4 sata disks OR have the software write frame 1 -> disk 1 frame 2 disk2 etc

as far as I can tell Jason the chip only does that smear when you hit 255 white or above..I could be wrong..for me I feel we can shoot around that issue and the resolution and bit depth far outweigh the problem. It is an issue nonetheless.

I do not have the jvc hdcamera

Jason Rodriguez · July 17th, 2004, 05:07 PM

If it's only when you hit 255, then I think you're fine, but if it's simply brighter areas, then that's a problem, because the hot stripes ruin your dynamic range-you can't lift up the blacks or else you get these nice light stripes running through the image. Not good.

This might be something you want to check into further.

July 17th, 2004, 12:36 PM	#812
David Newman CTO, CineForm Inc. Join Date: Jul 2003 Location: Cardiff-by-the-Sea, California Posts: 8,095	<<<-- Originally posted by Jason Rodriguez : Why would the Mac version be unlikely unless you release code (do you mean open source it?)?? -->>> We don't have any Macs or a development enviroment for a Mac. At least for today all CineForm tools will be PC based. __________________ David Newman -- web: www.gopro.com blog: cineform.blogspot.com -- twitter: twitter.com/David_Newman

July 17th, 2004, 02:51 PM	#818
Steve Nordhauser Silicon Imaging, Inc. Join Date: May 2004 Location: Troy, NY USA Posts: 325	Jason on smearing: That smearing with oversaturated pixels is a Micron problem (all Micron 1.3Mpix cameras will have it). Definitely not the lenses. I bugged them about it and they said run slower, which negates what we want to do about rolling shutter. I was told (but haven't seen) that the 3.2Mpix Micron doesn't do that as much - different pixel architecture. The 3.2Mpix should be able to do both 720p and 1920x1080@24fps, 10 bit. If Obin and Rob want to upgrade in about a month I'll work it out with them. The SI-3300 will have less sensitivity though. Jason on recording: To do what you want to do as a minimalist, you can use our standard camera, c mount lens, direct-to-disk recording software (streampix will beat xcap for this) and then your own processing chain. Rob or Obin might have the recording thing done, but maybe not in time for you. Giroud and Ben on Imperx: They have the only Cardbus cameralink frame grabber that I know of. If there is a request, I can get them to qualify our cameras (maybe the SI-3300 first) on it but you are at the mercy of their software support. This is the joy of an industry standard. They can use our cameras. __________________ Silicon Imaging, Inc. We see the Light! http://www.siliconimaging.com

July 17th, 2004, 12:26 PM	#811
Les Dit Major Player Join Date: Oct 2003 Location: Southern Cal-ee-for-Ni-ya Posts: 608	Ben, In case you haven't seen this paper, take a look ! He covers many of the methods, and has code to try them out. The code is for MATLAB but it's easy to see what it does. I like the variable gradient method, but it's the slowest. But what do we care, it's quality we want, and it's still faster than taking film to the lab ! He covers the computational cost on the methods as well, to help see what they take. www-ise.stanford.edu/~tingchen/main.htm -Les <<<-- Originally posted by Ben Syverson : Jason and David, Yeah, the current implementation of linBayer is, as the name suggests, purely linear. I do this for two reasons: it's fast, and it's lossless. By lossless I mean that the original values are maintained, so if you wanted to, you could take the RGB image and recover 100% of the original RAW data. I'm also developing a higher-quality algorithm, probably based on splines. I'll keep you posted! - ben -->>>

July 17th, 2004, 12:37 PM	#813
Jason Rodriguez Trustee Join Date: Mar 2003 Location: Virginia Beach, VA Posts: 1,095	Question for Obin, Steve: Based on some new bayer interpolation stuff that's recently become available, how much would it cost to buy the equipment for shooting a short film with this stuff? I'm assuming you'd need the SI-1300 A PC to attach to with card and capture software, Hard-drives (of course :-) Software for editing, color-correction (I already have that). A lens. Can XCAP be used on a short film? Do you need Stream-pix as well to get a RAW bayer image (no interpolation, just a black-and-white checker patterned image)? Do you also have to mount the camera inside another camera like you did Obin? And lastly, Have the rolling shutter artifacts been "solved"?? Do we even have sound-sync? I'm really starting to entertain the notion of shooting a short film with one of these cameras. I've got the script, and I'm ready to go if the problems have been sorted out with the logistics of the camera (rolling shutter, and that weird vertical smearing I've seen in some of Obin's stuff). Altough depending on pricing, I may want to see what's coming around the corner (even Kinetta stuff-but that would be through rental). Hey Obin, could you check an make sure it's not the lens making those smearing patterns? I guess I'll think about it now, and see what happens when the Altasens stuff hits the market. BTW, Mini-ITX isn't going to work for this stuff, is it?

July 17th, 2004, 12:39 PM	#814
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Thanks Les! I've read the paper, but I'm going a different route. I'm developing a logic-based (as opposed to mathematic) de-zippering routine. I'll posts some results in the next hour or so. - ben

July 17th, 2004, 01:39 PM	#815
Giroud Francois Inner Circle Join Date: Feb 2004 Location: switzerland Posts: 2,133	actually it seems this post is gone on another topic (compression and software) but i just take a look on the imperx web site and they get a nice chip and a pcmcia camera link card that could help to build a very compact/low power design.

July 17th, 2004, 01:51 PM	#816
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Giroud, I looked with great interest at the Imperx products, but once I got the quote sheet, my interest waned. Their cameras start at $4600 for anything over 1004x1004 in color... In contrast, I'm putting my entire rig together for about $2500, including camera, laptop, high speed HD and a brand new high resolution lens from Edmunds Optical. Of course, it remains to be seen if it will actually all work. :) - ben

July 17th, 2004, 02:17 PM	#817
Jason Rodriguez Trustee Join Date: Mar 2003 Location: Virginia Beach, VA Posts: 1,095	Found a new Pentium M-based embedded product coming to market soon. http://www.kontron.com/products/pdpr...kps=1082&kp=75 This one will have 64-bit PCI-X slots through a special riser card. Also has dual SATA interface and an IDE interface for a boot drive.

July 17th, 2004, 03:10 PM	#819
Jason Rodriguez Trustee Join Date: Mar 2003 Location: Virginia Beach, VA Posts: 1,095	Does the Altasens have verticle pixel smearing?

July 17th, 2004, 03:22 PM	#820
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	jason I am working on design for a standalone camera very "Kinetta" like but simple and easy to use ...do you have any idea what that new microboard chipset is? i need hardware that will run with Linux for the UI of this stand-alone Rig ;) I jsut looked at that board page looks like, 855GME, 6300ESB(Hance Rapids) chipsets anyone know if that is a standard chipset? or? I think that may be the Perfect Board for a camera!! My brother is a CAD/FX guy.. I will start working out the system design soon...fun fun fun!

July 17th, 2004, 03:27 PM	#821
Ben Syverson Major Player Join Date: Mar 2004 Location: chicago Posts: 434	Hey, The new version of linBayer is done for now. I'll come back to it a bit later. Here's a comparison. Old linBayer image New linBayer algorithm New linBayer algorithm (aggressive de-zippering) The aggressive de-zippering option is great for random fine detail like text, but it can introduce stray pixel artifacts on strong, fine horizontal and vertical lines. For those images, leaving aggressive de-zippering off will be the best compromise. Best of all, there is no computationally intensive math. It's just about exactly as fast as linear interpolation. Also, the process is still lossless. All of the original pixel values are untouched, which means you'll be able to re-do the de-bayer process later if a better algorithm comes along. I've posted the new build of the mac version to the linBayer website. The PC version will follow a bit later. Right now I have to go to the lake and play scrabble. - ben

July 17th, 2004, 03:30 PM	#822
Jason Rodriguez Trustee Join Date: Mar 2003 Location: Virginia Beach, VA Posts: 1,095	According to the info page: Up to 2.4GHz Pentium M processors using Intel 855GME, 6300ESB(Hance Rapids) chipsets BTW Obin, are you still going to be useing the Microns? Right now it seems as though that veritcal smear problem is a real show-stopper since it really limits your dynamic range. Steve, I know you said that the newer chips is going to have "less" of this problem, but if it's still a problem nonetheless, hmm, then I guess the Altasens will be worth waiting around for. I mean I think it would be silly to pay $3000 for a camera and then have a major artifact to deal with. One more thing though, is this problem ONLY when you oversaturate the chip, or just when you have a bright area? BTW Obin, remember that there's a riser card needed for the 64-bit PCI-X slot, so that's extra space that you don't see on that board. I'm assuming that the card will fit over the top (so you'll have to find a place to mount the hard-drives somewhere else). Another thing Obin, check out Ben's plug-in when he gets the PC version up. The new version is very good, you'll be straining your eyes to see any artifacting. And the nice thing is it works now, not five months or a year from now :-) P.S. that wasn't mean to be a jab at anyone, just saying this a great solution for getting up and going right away.

July 17th, 2004, 03:53 PM	#823
Les Dit Major Player Join Date: Oct 2003 Location: Southern Cal-ee-for-Ni-ya Posts: 608	Obin, do you have an JVC HD-10 you can use for tests ? -Les

July 17th, 2004, 04:42 PM	#824
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	the AWESOME thing about that board is it's got sata AND pci-x !! we can get 4 sata drives and a pci-x grabber and the datarates would be HUGE! I bet 60fps from the 1080p chip at 10bit ;0 ?? can raid the 4 sata disks OR have the software write frame 1 -> disk 1 frame 2 disk2 etc as far as I can tell Jason the chip only does that smear when you hit 255 white or above..I could be wrong..for me I feel we can shoot around that issue and the resolution and bit depth far outweigh the problem. It is an issue nonetheless. I do not have the jvc hdcamera

July 17th, 2004, 05:07 PM	#825
Jason Rodriguez Trustee Join Date: Mar 2003 Location: Virginia Beach, VA Posts: 1,095	If it's only when you hit 255, then I think you're fine, but if it's simply brighter areas, then that's a problem, because the hot stripes ruin your dynamic range-you can't lift up the blacks or else you get these nice light stripes running through the image. Not good. This might be something you want to check into further.