4:4:4 12-bit Uncompressed DVX100

Juan P. Pertierra · May 3rd, 2004, 06:19 PM

Another outdoor shot...this one makes a bit more use of the dynamic range. There are some speckles in the sky, which will be fixed with a larger cap, haven't had time to get it.

This is completely RAW, but what semed to work for me was to collapse the layers into one, and drop down the RGB level upper margin until it meets the edge of the histogram.

I'll post the DV frame soon...according to DV, the sky, road and houses are white.

http://expert.cc.purdue.edu/~pertierr/cap9_RAW.psd

Nicholi Brossia · May 3rd, 2004, 07:05 PM

Very excellent work Juan. I've been following this post for quite some time now and am constantly impressed by your findings.

Lately I've been thinking about getting a DVX100A, but feel that your modified (Juan-ified?) DVX100 would be a much better choice. Unfortunately, the DVX100 is pretty hard to find these days, but the DVC80 (the DVX100's interlace-only, non-filmlike counterpart) is still available. Will this modification work with the DVC80 as well? And if so, will it be as capable of recording 30fps progressive and the wider color/brightness range that you're achieving?

I have a feeling everything will translate to the DVC80 just fine since it is the exact same optical and ccd setup as the DVX100, from what I understand. Since you're pulling the data straight off the ccds, then the digital capabilities of the camera shouldn't come into play... at least that's my uneducated theory.

Juan P. Pertierra · May 3rd, 2004, 09:41 PM

Nicholi,

I would have to see the technical manual for the DVC80 to know for sure. However, i too think that it probably is layed out in a very similar way, and the critical aspect is actually not the CCD's but if they share the same internal circuitry.

My mod will work exactly the same on either the DVX100 or DVX100A. However, both cameras are identical up to the A/D converters so my mod will produce the same output on both, and thus there will be no advantage to having the Advanced model, unless you want to use the DV output for something.

Cheers!
Juan

Juan P. Pertierra · May 3rd, 2004, 09:51 PM

Here is my 'best guess' color correction for the last capture, and the DV counterpart. The DV frame might not be exactly the same frame but it is within 24 frames of the correct one :)

http://expert.cc.purdue.edu/~pertierr/cap9_RAW.tif
http://expert.cc.purdue.edu/~pertierr/cap9_DV.tif

Obin Olson · May 3rd, 2004, 10:03 PM

OMG Juan, I am at a hotel with the laptop...I just downloaded your TREES shot....it blows my mind what you can do with that image! I would have said it was trash that dark....this is amazing!

this stuff is looking so good I almost want to forget my HD camera project!

Nick Hiltgen · May 3rd, 2004, 10:37 PM

Just discovered this thread a couple of days ago, I really think it's cool what you're trying to do/doing. In regards to the noise issue, I'm not positive but I could've sworn I saw some noise in the DV footage as well, so I'm thinking that it's probably not a result of the tweaking or 4:4:4 process. I'm curious as to how the actual image capturing (sequences) is coming along, Also much like adam mentioned earlier I wonder if this can be done with any other camera (specifialy the xl1) But hey it looks awesome so far (especially the tree's raw file) And I can't wait to see how the whole thing reproduces a moving picture!

Stephen van Vuuren · May 3rd, 2004, 11:07 PM

Juan:

The tree shots are quite impressive. It just makes realize how I hate the DV Codec. I've always suspected the CCD's latitude was for more than "video" was claimed and your results really "highlight" that.

Stephen van Vuuren · May 3rd, 2004, 11:12 PM

Juan:

I did some analysis on the RAW Tiff on the noise.

In the TIFF, the noise is a single pixel that register either 0 or 1 in Green channel.

The adjoining pixels are perfect and have the same R & B pixel values as the noise pixel.

To my eye, it looks like an error in writing the green pixel value.

Juan P. Pertierra · May 3rd, 2004, 11:25 PM

stephen:

Thanks for the encouragement! I am pretty impressed myself with the results, and these frames are part of clips, but i have a few more lines of code to write before I can post the entire clips in color frames, and i'm also juggling finals :)

About the noise, i found out what was causing it, and it just takes a larger capacitor to fix, but i haven't had time to get it today. I added a smaller capacitor than what is needed which is why there is must less noise in these pictures...the turning point was finding that the capacitor was needed :) now it's just a matter of buying the right one...

It was just ground-loop noise in the lines.

Juan

John Cabrera · May 3rd, 2004, 11:46 PM

So I wonder why everything gets substantially brighter between what you're pulling off, Juan, and the DV encoding. Is this simply the camera's own color correction function doing this to the image. How could a person get a sense for the final look of the lighting on this mod... from this DV frame, it seems that what you're seeing in the viewfinder is very different from the frames you're pulling. And you said you even color corrected this version too... How much darker was it pulled straight off the chips?

I think the best thing about these two frames is the red in that chair... look at all those blockies in the DV clip. In your capture version it's prestine beautiful red.

John

Juan P. Pertierra · May 3rd, 2004, 11:52 PM

John,

It's not that everything gets brighter, it's just that the 8-bit DV footage clips at a much lower brightness. The reason for this is that given that the DV format standard is 8-bit but the images are captured in 12-bits, the engineers at Panasonic had to decide where to sacrifice the bits that would be missing in the DV footage. They decided to sacrifice hi-end brightness, but gaining more low-level detail. They could've set the white at the saturation of the CCD's and get the same latitude i'm getting, but then they would get much less dynamic range because the 8-bits would be spread over a larger area.

You're right about the viewfinder, it shows you what the DV footage is. Even though the viewfinder is not totally useless, there will be a way to preview the RAW footage out of my device.

>And you said you even color corrected this >version too... How much darker was it pulled >traight off the chips?

The RAW, uncorrected photoshop file is at:
http://expert.cc.purdue.edu/~pertierr/cap9_RAW.psd

John Cabrera · May 4th, 2004, 12:23 AM

That totally makes sense, and explains why the whites are clipped the way they are. And there's no question that you're getting amazing latitude, but mostly I was referring to the low-levels and mid tones. Compare for example the shady area inside the trees. That also appears substantially darker than your DV clip. If it was only due to the way the 8-bits in the DV codec are distributed, then wouldn't the whites be clipped, but the darks and midtones be the same, or at least an approximation. They're substantially darker to my eye.

I did a color correction of the PSD you posted, trying to match the aproximate darks and midtones, while keeping the sky completely intact... it's pretty amazing to see that kind of image, I'll tell you that. Congratulations!

John

Capra Mauro · May 4th, 2004, 06:48 AM

i'm observing the cap9 imgs:
1
in the DV frame there's the "same" (or worse) noise that is present in the raw frame
???
Is this caused by the presence of your gear in the circuitery?
If not there's something wrong on your DVX!

2
the DV frame seems to be interlaced: why?

PS
I'm sorry for the dryness of my english but... you're my man!!!

Juan P. Pertierra · May 4th, 2004, 11:48 AM

About the noise in the DV frame, i just noticed it and realized that I didn't unhook some test equipment while i was capturing and thus added yet another ground loop...

Now, i have no idea why this is interlaced. It was definitely captured in 24P mode, but apparently it got interlaced when i captured it in FCP or put it into Shake to grab the frame...that's strange.

Juan

Les Dit · May 4th, 2004, 12:16 PM

How are you getting YUV signals from the ccd's? Aren't they RGB ?

Also: look at the sustained transfer rated for hard drives, not the interface burst rates. Big difference.

May 3rd, 2004, 07:05 PM	#467
Nicholi Brossia Major Player Join Date: Jun 2003 Location: Oklahoma City, OK Posts: 389	Very excellent work Juan. I've been following this post for quite some time now and am constantly impressed by your findings. Lately I've been thinking about getting a DVX100A, but feel that your modified (Juan-ified?) DVX100 would be a much better choice. Unfortunately, the DVX100 is pretty hard to find these days, but the DVC80 (the DVX100's interlace-only, non-filmlike counterpart) is still available. Will this modification work with the DVC80 as well? And if so, will it be as capable of recording 30fps progressive and the wider color/brightness range that you're achieving? I have a feeling everything will translate to the DVC80 just fine since it is the exact same optical and ccd setup as the DVX100, from what I understand. Since you're pulling the data straight off the ccds, then the digital capabilities of the camera shouldn't come into play... at least that's my uneducated theory. __________________ Nicholi Brossia

May 3rd, 2004, 11:07 PM	#472
Stephen van Vuuren Space Hipster Join Date: Oct 2001 Location: Greensboro, NC Posts: 1,508	Juan: The tree shots are quite impressive. It just makes realize how I hate the DV Codec. I've always suspected the CCD's latitude was for more than "video" was claimed and your results really "highlight" that. __________________ stephen v2 www.insaturnsrings.com

May 3rd, 2004, 11:12 PM	#473
Stephen van Vuuren Space Hipster Join Date: Oct 2001 Location: Greensboro, NC Posts: 1,508	Juan: I did some analysis on the RAW Tiff on the noise. In the TIFF, the noise is a single pixel that register either 0 or 1 in Green channel. The adjoining pixels are perfect and have the same R & B pixel values as the noise pixel. To my eye, it looks like an error in writing the green pixel value. __________________ stephen v2 www.insaturnsrings.com

May 3rd, 2004, 11:46 PM	#475
John Cabrera Regular Crew Join Date: Apr 2004 Location: California Posts: 67	Clipped Whites So I wonder why everything gets substantially brighter between what you're pulling off, Juan, and the DV encoding. Is this simply the camera's own color correction function doing this to the image. How could a person get a sense for the final look of the lighting on this mod... from this DV frame, it seems that what you're seeing in the viewfinder is very different from the frames you're pulling. And you said you even color corrected this version too... How much darker was it pulled straight off the chips? I think the best thing about these two frames is the red in that chair... look at all those blockies in the DV clip. In your capture version it's prestine beautiful red. John

May 4th, 2004, 12:23 AM	#477
John Cabrera Regular Crew Join Date: Apr 2004 Location: California Posts: 67	Hmm That totally makes sense, and explains why the whites are clipped the way they are. And there's no question that you're getting amazing latitude, but mostly I was referring to the low-levels and mid tones. Compare for example the shady area inside the trees. That also appears substantially darker than your DV clip. If it was only due to the way the 8-bits in the DV codec are distributed, then wouldn't the whites be clipped, but the darks and midtones be the same, or at least an approximation. They're substantially darker to my eye. I did a color correction of the PSD you posted, trying to match the aproximate darks and midtones, while keeping the sky completely intact... it's pretty amazing to see that kind of image, I'll tell you that. Congratulations! John

May 3rd, 2004, 06:19 PM	#466
Juan P. Pertierra Major Player Join Date: Jul 2003 Posts: 479	Another outdoor shot...this one makes a bit more use of the dynamic range. There are some speckles in the sky, which will be fixed with a larger cap, haven't had time to get it. This is completely RAW, but what semed to work for me was to collapse the layers into one, and drop down the RGB level upper margin until it meets the edge of the histogram. I'll post the DV frame soon...according to DV, the sky, road and houses are white. http://expert.cc.purdue.edu/~pertierr/cap9_RAW.psd

May 3rd, 2004, 09:41 PM	#468
Juan P. Pertierra Major Player Join Date: Jul 2003 Posts: 479	Nicholi, I would have to see the technical manual for the DVC80 to know for sure. However, i too think that it probably is layed out in a very similar way, and the critical aspect is actually not the CCD's but if they share the same internal circuitry. My mod will work exactly the same on either the DVX100 or DVX100A. However, both cameras are identical up to the A/D converters so my mod will produce the same output on both, and thus there will be no advantage to having the Advanced model, unless you want to use the DV output for something. Cheers! Juan

May 3rd, 2004, 09:51 PM	#469
Juan P. Pertierra Major Player Join Date: Jul 2003 Posts: 479	Here is my 'best guess' color correction for the last capture, and the DV counterpart. The DV frame might not be exactly the same frame but it is within 24 frames of the correct one :) http://expert.cc.purdue.edu/~pertierr/cap9_RAW.tif http://expert.cc.purdue.edu/~pertierr/cap9_DV.tif

May 3rd, 2004, 10:03 PM	#470
Obin Olson Trustee Join Date: Jan 2003 Location: Wilmington NC Posts: 1,414	OMG Juan, I am at a hotel with the laptop...I just downloaded your TREES shot....it blows my mind what you can do with that image! I would have said it was trash that dark....this is amazing! this stuff is looking so good I almost want to forget my HD camera project!

May 3rd, 2004, 10:37 PM	#471
Nick Hiltgen Trustee Join Date: May 2003 Location: Atlanta GA Posts: 1,427	Just discovered this thread a couple of days ago, I really think it's cool what you're trying to do/doing. In regards to the noise issue, I'm not positive but I could've sworn I saw some noise in the DV footage as well, so I'm thinking that it's probably not a result of the tweaking or 4:4:4 process. I'm curious as to how the actual image capturing (sequences) is coming along, Also much like adam mentioned earlier I wonder if this can be done with any other camera (specifialy the xl1) But hey it looks awesome so far (especially the tree's raw file) And I can't wait to see how the whole thing reproduces a moving picture!

May 3rd, 2004, 11:25 PM	#474
Juan P. Pertierra Major Player Join Date: Jul 2003 Posts: 479	stephen: Thanks for the encouragement! I am pretty impressed myself with the results, and these frames are part of clips, but i have a few more lines of code to write before I can post the entire clips in color frames, and i'm also juggling finals :) About the noise, i found out what was causing it, and it just takes a larger capacitor to fix, but i haven't had time to get it today. I added a smaller capacitor than what is needed which is why there is must less noise in these pictures...the turning point was finding that the capacitor was needed :) now it's just a matter of buying the right one... It was just ground-loop noise in the lines. Juan

May 3rd, 2004, 11:52 PM	#476
Juan P. Pertierra Major Player Join Date: Jul 2003 Posts: 479	John, It's not that everything gets brighter, it's just that the 8-bit DV footage clips at a much lower brightness. The reason for this is that given that the DV format standard is 8-bit but the images are captured in 12-bits, the engineers at Panasonic had to decide where to sacrifice the bits that would be missing in the DV footage. They decided to sacrifice hi-end brightness, but gaining more low-level detail. They could've set the white at the saturation of the CCD's and get the same latitude i'm getting, but then they would get much less dynamic range because the 8-bits would be spread over a larger area. You're right about the viewfinder, it shows you what the DV footage is. Even though the viewfinder is not totally useless, there will be a way to preview the RAW footage out of my device. >And you said you even color corrected this >version too... How much darker was it pulled >traight off the chips? The RAW, uncorrected photoshop file is at: http://expert.cc.purdue.edu/~pertierr/cap9_RAW.psd

May 4th, 2004, 06:48 AM	#478
Capra Mauro Tourist Join Date: Jan 2004 Location: Asti - Italy Posts: 4	i'm observing the cap9 imgs: 1 in the DV frame there's the "same" (or worse) noise that is present in the raw frame ??? Is this caused by the presence of your gear in the circuitery? If not there's something wrong on your DVX! 2 the DV frame seems to be interlaced: why? PS I'm sorry for the dryness of my english but... you're my man!!!

May 4th, 2004, 11:48 AM	#479
Juan P. Pertierra Major Player Join Date: Jul 2003 Posts: 479	About the noise in the DV frame, i just noticed it and realized that I didn't unhook some test equipment while i was capturing and thus added yet another ground loop... Now, i have no idea why this is interlaced. It was definitely captured in 24P mode, but apparently it got interlaced when i captured it in FCP or put it into Shake to grab the frame...that's strange. Juan

May 4th, 2004, 12:16 PM	#480
Les Dit Major Player Join Date: Oct 2003 Location: Southern Cal-ee-for-Ni-ya Posts: 608	How are you getting YUV signals from the ccd's? Aren't they RGB ? Also: look at the sustained transfer rated for hard drives, not the interface burst rates. Big difference.