4:4:4 12-bit Uncompressed DVX100

[Jeff Donald]

Up-rez file first, then color correct, enhance etc., sharpening should be last in workflow. Sharpening will vary depending on the intended purpose of the files. sharpening for output to film will be different than sharpening for video projection, broadcast etc.

[Juan P. Pertierra]

That makes sense....i guess i was assuming that sharpening was part of the up-rezzing since Photozoom does it under the preset-settings...

If anyone knows the details of what algorithm photozoom/spline uses that would be great...if it's not patented or something to that effect I can probably implemented in my software.

[Randall Larsen]

Juan and listmembers,

There is an interesting discussion on one chip bayer filter cameras such as Arri, Dalsa, and Kinetta on one of the CML lists.
The verdict was that 3 chip cameras are best unless its a 4K resolution chip that implementing the bayer filter. Arri limits their output to 1920x1080 or so to limit the motion artifacts introduced by bayer filtering.

http://www.siliconimaging.com/Specif...ual%20R1_7.pdf

Has the spec for the camera link interface 1.2gig used by
most industrial and scientific camera mfgs for their HD sensors.

Silicon imaging has built a camera link to gigabit ethernet interface. This allows the camera to be 100 meters from the computer with copper cat 6 UTP cable no fiber!

Now camera link may be overkill for the signal coming out of the
DVX100 but it would be nice to have a design that could be transported to a new HD chip camera when they become cheap enough. Maybe gigabit ethernet is better than 800mbs firewire?

[Jason Rodriguez]

<<<-- Originally posted by Randall Larsen : The verdict was that 3 chip cameras are best unless its a 4K resolution chip that implementing the bayer filter.-->>>

I'm on the CML and I do not believe that was the verdict, except by certain individuals who make products for 3-chip cameras.

According to papers written by Dalsa, a bayer sensor will have about 75% of the resolution that a 3-chip camera will have at any given resolution setting. So a bayer sensor will give you around 1440 horizontal pixels compared to the 1920 that a true HD camera can give you. Now you ARE getting 1920 horizontal pixels out of the bayer imager, it's just that like HDCAM, the actual amount of real resolution is around 1440 horizontal pixels. Which isn't bad, we've been living this for a while with HDCAM. In fact all the HD formats right now except for HDCAM SR and D-5 pre-filter the image before compression.

I saw output from the Arri D-20 at NAB and it was great. I saw the output from the Dalsa Origin at NAB and it was great. The arri BTW is only a 3K chip, and they're basically outputting 2K horizontally-a little more thant the 75% I was talking about earlier (more like 66%), which is probably to play it safe.

Anyways, I'm sure the Kinetta will look great. And all this rumoring that Bayer "won't be any good" just doesn't seem to pan out with the experiences I've had viewing footage off these systems that have good bayer-sampling algorithms.

[Randall Larsen]

Jeff Donald,

Thanks for the photo batch tip.

Jason Rodriguez, and listmembers

Good point about the LUT (Juan would need to do this in an FPGA).

I like the idea of uprezzing right away. I don't know if that is going to complicate the color correction by introducing artifacts.
As I understand it the photoZoom algorithm is interpolating the missing pixels with some sort of S-spline based interpolation. We really have to find some articles on this as well as L3 and wavelet compression.

For transfer to film I'd say leave putting "detail" or enhancement in until last. You don't know how this is going to play with the light and filter effects the DP uses. Perhaps you don't want any "detail" correction at all if you've got 720P.

In designing the camera interface Juan really does have to think about the whole production pipeline. What will the workflow be?

Monitoring:

Juan,

Could the "monitor" have a histogram (digital) or waveform monitor overlay feature (analog). Maybe a vectorscope implemented in software would be nice (if we had analog).

The advantage of a waveform monitor would be to assure that
we are only lowering the gain enough (or NDing enough) to keep the highlights from clipping. If we lower the gain or the light levels too much we lose detail at the low end.

We might lower the light and simultaneously raise the gain because:

Its very difficult to get good rack focus effects with 1/3" chips. You have to ND a lot or use long lenses to cause the depth of field to decrease enough to get cinema style DOF effects.

This is one of the advantages of the mini-35 approach to using the 1/3" sensors yet retaining cinema style DOF.

There is also an optimal setting for gain vs. the signal to noise ratio. As everybody knows if we raise the gain more than necessary we get more noise than necessary. If you have got 12 bits maybe you don't need to raise the gain so much.

Juan's 12 bit shot of the trees was amazing. Uncompensated inside the foliage it was totally dark.

However, because of the extra bits you could actually pump up picture in all but the darkest areas (with the sacrifice of the
clipping the highway shot).

With some dodging, softclipping, and some black stretch you could create a picture that showed lots of detail and fit within the dynamic range of your viewing monitor (8-bit) or film stock (10-bit).

The image would simulate more realistically what the human eye and brain see with foveal sampling.

FOCUS:
Maybe the slow auto focus built in to the DVX100 is good enough. There are also conversion tables to convert the percentages on the lens focal distances to ft or meters. A real camera crew "tapes" critical focal distances. Of course that procedure assumes you have a well calibrated lens.

[Randall Larsen]

Jason Rodriguez,

Perhaps I overstated the "consensus" for 3 chip cameras on CML.

Do you know of any 35mm image area sensors that are publically available? Dalsa and Arri have 35MM image area sensors but the biggest ones I've found that one can order run at 24fps in 1" optical format.

Dalsa has one camera (maybe they sell the chip) that can be pushed to 20fps. The chip probably could be pushed to 24fps by reading sections in parallel and assembling the image in a frame store.

I am glad to hear a report on the Arri-20. This camera I believe has the advantage of a 35mm Motion Picture imaging area. Dalsa also claims this advantage.

75% of the resolution of a 3 chip configuration may be the visual impact of Bayer to a viewer. One or two postings doubted that a Bayer camera of similar resolution as a 3 chip camera would be as good for green screen and matte work. Some said it might.

How did the bayer camera's look when recording Motion? Did you notice any artifacts?

I personally would rather have 1920x1080 than 1440 by 1080. Perhaps as you say it doesn't make a difference. But in the long run Display of HD will improve. 1440 may look soft in the future. Maybe I will want to perform a digital zoom on my footage to repurpose some of my shots. Maybe I will need to extract close ups from my wide shots. I need all the resolution I can get!

Did any single chip camera mfg. show 60fps images from any of these cameras?

I assume Kinetta did not show any footage? or live demo?
I like the altasens sensor but how will Kinetta get around its
2/3" format size in that small camera (no room for a field lens)?

1440 line HDCAM is not really good enough for tape to film transfer or for effects work. HDCAM SR should be OK.

However there is no need to buy expensive D6 or HDCAM SR or D5 recorders when you could output 12 bit uncompressed raw data to relatively cheap SATA and EIDE drives over firewire 800 or gigabit ethernet. There are lossless compression schemes that could be implemented (like Dalsa's L3) to save bandwidth and disk space without giving up the camera resolution you paid for.

I wuld like to avoid HDCAM or other compressed formats. These formats are part of the camera companies business models now.
Its a market segmentation scheme. Anybody should be able to record full bit depth and full resolution signals if they really want to. That is what Juan's mod is all about.

If you need to archive the shot because you don't trust Hard drives use blue ray or blue laser recorders.

The different compressed formats no longer reflect the technical limitations of recording digital video. They are an artifact of the market segmentation scheme camera vendors want to impose on us to extract consumer surplus (Broadcaster dollars) from the marketplace.

So Jason, I never said "Bayer won't be any good." I just said "Bayer won't be as good." Will Bayer be cost effective? Perhaps? There are yet roadblocks to be overcome.

There are roadblocks in building a good 3 chip camera as well.
Can Sony work out the problems it has with chromatic aberration in its prism system? Can film lenses be practically used with prism cameras using the ground glass technique perfected by the Germans?

Can CMOS fabricators build 14 megapixel chips that can be read out full resolution at 60 fps?

The whole beauty of Juan's mod is that for practical purposes a raw data file from a DVX100 may be "good enough" for HDTV and for low budget film transfers.

[Capra Mauro]

Hi Jaun,
i've downloaded the clip, in my opinion the speckless aren't casuals: they appear where the color is closer to white. Maybe that the caps you are using didn't handle this value and so "clips" the relative pixel. If you individuate the value that clip you can choice the right headroom of the caps (i think).
Furthermore the RGB seems to be not well aligned: the green channel is too much (someting like 2 pixels) down and right (look at the writes on the thank).
For me the standard output solution is better than the uprezzed one: it avoid the artifacts problems, it is lighter to be processed and stored and it's good enough for normal TV projects.
I hope that this can help you.
Bye

[John Cabrera]

Juan,

I guess I haven't actually posted this, but please put me on the list for the mod... if you do actually have a list going.

Thanks,
John Cabrera

[John Cabrera]

Juan,

I noticed something that maybe you're already aware of in regards to the alignment of the rgb cannels. I'd been trying to move them to the correct alignment (I'm sure as everyone has tried) to no avail. One pixel one way shifts green to magenta. I tried upconverting so that I could work in essentially half and quarter steps, but still no luck. Then I noticed something with your raw file DVComp2_RAW.tif. The green channel seems to have a slightly larger field of view (about a pixel worth). If you look in channel mode you can see it by switching between the red channel and the green channel while focusing on the far right balcony post that's sticking out of frame right. There's a slight shift in the amount of the post that's viewable. However if you look at the other side of the frame (pretty much any point, but I chose the upper left coner of the frame at the top of the sliding glass door), you'll notice that there isn't nearly that much of shift anywhere along that side of the frame. Of course there's a little shift with all three channels, but its clear to my eye that what's happening on the other side of the frame isn't a shift at all but rather a wider field of view in the green. The blue channel seems to be about right compared to the red. I tried correcting the green by resizing it. I enlarged it by a pixel on the horizontal axis and this seemsed to help out a bit. Just thought I'd mention it. If that's being caused by the optics of the camera, you may be able to fix it in the programming of the software by doing a slight resize of the green channel... of course, if it's being caused by the prosumer optics it may be slightly different in everyone's camera.

[Ben Syverson]

If you resize the green channel (which carries the majority of the luminance information we see as "detail"), you get into issues of resampling, which would be a generational loss. Also, there's no way the channel could be resampled in "real-time" unless you used a very low-quality interpolator. Even an average interpolator such as a 16 pixel bicubic or spline would take too long...

- ben

[Thomas Smet]

Hey Juan could you capture a still frame of an object in front of a solid blue or solid green background. It could just be a small piece of green or blue paper that is fairly saturated. Maybe you could just put your hand in front of the paper. It would be nice if we could test one of the most important reasons of having a 4:4:4 mod which would be keying. We may also have an easier time to figure out any alignment issues of color channels this way.

If you could just point one light at the object and color background. Shadows shouldn't matter because we will not be trying to get a perfect key. We just need enough to test the edging.

[John Cabrera]

Ben,

According to what Jaun proposed earlier about uprezing, he seems to believe that an interpolator such as bicubic or spline could be implemented in his software. Take a look at some of Juan's last posts.

As for image quality loss, I did a resample in photoshop of the green channel using bicubic and my eye didn't notice any quality loss (I only resized by 1 pixel)... what my eye did notice was an improvement in the green channel's croma shift... and I don't know about you, but that lack of alignment is what I consider the poor quality. You gotta give a little to get a little, you know.

[Ben Syverson]

John,

Any quality interpolation will not happen in realtime on modern computers. It's just a mathematical reality -- at 24fps, you're looking at over 7 million pixels a second to pass through the interpolator, and each pixel needs to see a neighborhood of at least 16 pixels.

But maybe Juan has some awesome algorithms he should patent immediately. :)

I would hope that if he did implement interpolation, he'd use Spline rather than Bicubic. Juan, if you want any help with the interpolation code, let me know -- I've got some laying around for really nice 16 and 36 pixel spline interpolation...

- ben

[Juan P. Pertierra]

Randall(waaay up there):
I just wanted to make this clear: ALL the raw captures i have posted so far are 10-BIT. Nothing I have posted so far is 12-bit because my test gear doesn't handle the extra 6 bits. So everything i have posted is handicapped in color precision and a little latitude from the final prototype output.

Now, about the interpolation...I think it is best to allow the user to decide, and if Ben you can email some Spline interpolation algrithm that would greatly help. I'm sure it's not that difficult to implement, specially on the PC.

It would get really difficult to do it on an FPGA, specially since it will be running at around 100Mhz or so. I don't think real-time is an option.

About the alignment, my current code from which the clip came out of does not align the channels, right now just puts them on top of each other as they are...it's a matter of three variables but I haven't added that yet.

I did notice however what others have pointed out...that the green channel seems to be offset from one side of the frame to the other, but i think this is due to just the level of manufacturing and alignment of the camera...we might just have to live with this.

Only solution I can think of, is to uprez the R,G,B channels separately and then use the extra resolution to do a fractional alignment in the original image space.

Green/Blue Screen capture: Sure! I just don't have a blue/green screen so what do you guys suggest? Can i get something at lowe's or just get a green piece of construction paper/poster board?

That's a good idea...probably one of the most important tests. I imagine lighting is also very critical for a good key?

[John Cabrera]

For the last film I did, we used green fluorescent poster board from the local Walgreens drug store for one of our days of pickups when we didn't have the green screen stage anymore. It worked just as well as the stage we worked on with the exception of the seam lines between pieces of boards.

Also, I've looked at it several times, and it's not a fractional offset, but rather a slightly different image proportionally... everything on the left side of the frame is much better aligned than on the right... which tells me that the images themselves have are too different in FOV for them to ever align right... I already tried uprezzing the footage by 400% to do fractional alignment and it did not work. I agree that it's probably just an issue with the manufacturing, but when I did a resizing of the green channel in Photoshop to compensate (enlarged it by 1 pixel), the alignment got noticeably better. I agree that using the algorithm for uprezzing the footage to HD would merely be a perk, but using the algorithm to compensate for this manufacturing defect (not an offset defect but rather a FOV defect) may be a crucial part of getting a good green screen key with the camera. Having the green channel gradually shifting like that across the image will cause problems with green screen, I promise.