abrupt highlights clipping - Page 7 at DVinfo.net

Go Back   DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Sony XDCAM EX Pro Handhelds
Sony PMW-300, PXW-X200, PXW-X180 (back to EX3 & EX1) recording to SxS flash memory.


Reply
 
Thread Tools Search this Thread
Old February 6th, 2008, 09:55 PM   #91
Trustee
 
Join Date: Nov 2005
Location: Sydney Australia
Posts: 1,570
I just tried a simple test with the EX1, hasn't been out of it's box today :)

Lamp with a diffused shade again a dark wall. Camera in manual, lamp filling around half the frame. EX1 histogram shows a nice lump in the middle, nothing anywhere near clipping.
Zoom in so the light is now full frame, the lump moves to the right of the histogram, looks like I'm getting pretty close to clipping. Zoom all the way into the slightly hotter middle of the lamp and on comes the Too Bright warning. Note I have not changed exposure while zooming in.

This is very different to how the scopes in Vegas work. The waveform display is the sum of all lines. If I'd recorded what I shot above and checked it with the Vegas scopes I'd see a lump in the middle of the waveform that was clipped. As I zoomed in the lump would get broader.

I just simulated the same kind of shot using Vegas generated media. Using Vegas's Histogram metering I get much the same thing. It's almost impossible to detect clipping with a histogram. In fact from what I'm seeing with histograms the more clipping there is in the shot the less noticeable it is in the histogram. With a whole frame clipped you just see a tiny line on the RH side of the display.

This article here:
http://www.luminous-landscape.com/tu...stograms.shtml
gives a detailed explaination of what a histogram shows. About half way down the page there's an example of a scene very similar to the one causing all this grief.
Bob Grant is offline   Reply With Quote
Old February 7th, 2008, 01:00 AM   #92
Major Player
 
Join Date: Sep 2007
Location: Palm Desert, California
Posts: 311
I started a thread on this in the Vegas forum. An EX1 histogram that looks like it goes from 10 to 90 put into Vegas clips like it wants to go to 120. By this I mean extrapolate the shape of the histogram into the clip area that Vegas scope cut of. AND the Vegas scopes clip at the BOTTOM END TO! Like the blacks want to go to -20!

Also Piort I DID give my opinion as to what is causing this much earlier in this thread and no one commented on it so maybe everyone thinks I am wrong, but here is my take again. The EX1 gammas are not straight line or even a partial exponential of a log of a straight line, I think it is programmed to have the Gamma changes with luminosity. The area behind the trees has a lower luminosity (because the trees block the light) and the gamma drops in that area which brings the blown white back into the blue. I think this how the camera gets its enormous exposure latitude and why the Vegas scopes get stretched. JMO as they say.
Michael H. Stevens is offline   Reply With Quote
Old February 7th, 2008, 02:47 AM   #93
Inner Circle
 
Join Date: Aug 2006
Location: Poland
Posts: 4,086
Quote:
Originally Posted by Michael H. Stevens View Post
The area behind the trees has a lower luminosity (because the trees block the light) and the gamma drops in that area which brings the blown white back into the blue. I think this how the camera gets its enormous exposure latitude and why the Vegas scopes get stretched. JMO as they say.
This explanation is the only one that can be thought of, Michael - and probably this is the reason nobody commented on it when you first suggested it. Assuming now that this is a byproduct of the "enormous EX1 latitude", one should simply be aware of it and, in backlit situations, either use cine gammas, or modify the Knee point/curvature of a standard gamma. I'd like this to be true, as the solution is so easy.

However, even on the scopes I posted here, the V1 also went well over 100 in Vegas, yet they sky is consistently blown to white, including the parts behind tree twiggies. This looks much more natural.

On top of that, please take another look at the picture Christopher linked to earlier in this thread:

http://www.siliconcine.net/temp/Blew_Skies.jpg

Judging from the overall look, I'd say a cine gamma curve was used - and yet do you see what's happening to the trees at the right upper horizon part?

So, I will be completely reassured only after considerable time of trouble-free shooting with the above recipe in mind, i.e. when I'm 100% sure that with cine gammas (or standard curves with modified knee), nothing like this can happen to my picture.
__________________
Sony PXW-FS7 | DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive

Last edited by Piotr Wozniacki; February 7th, 2008 at 05:16 AM.
Piotr Wozniacki is offline   Reply With Quote
Old February 7th, 2008, 04:19 AM   #94
Regular Crew
 
Join Date: Jan 2007
Location: Australia
Posts: 107
Piotr, factory default settings used on the EX1/HVX comparision, however, I can not be certain.
http://www.siliconcine.net/temp/Blew_Skies.jpg

Looking at the standard curve for XDCAM cam, IMO, why would you bother?
http://www.siliconcine.net/temp/XDCAM_Gamna_Curves.jpg

I understand your quest to solve this. Myself, I do not foreshadow dialing out of the four cine gamma options.
Christopher Barry is offline   Reply With Quote
Old February 7th, 2008, 04:37 AM   #95
Inner Circle
 
Join Date: Aug 2006
Location: Poland
Posts: 4,086
Quote:
Originally Posted by Christopher Barry View Post
Piotr, factory default settings used on the EX1/HVX comparision, however, I can not be certain.
http://www.siliconcine.net/temp/Blew_Skies.jpg

Looking at the standard curve for XDCAM cam, IMO, why would you bother?
http://www.siliconcine.net/temp/XDCAM_Gamna_Curves.jpg

I understand your quest to solve this. Myself, I do not foreshadow dialing out of the four cine gamma options.
Christopher, if this was indeed shot with the same, standard curve, it'd mean it's not just my camera - and the solution being cine gammas, or standard ones but with modified knee. Thanks.

You're right in seeing this thread as a quest to solve/explain/understand an issue, and not another EX1 bashing. The fact is I only use standard curves for indoor, lowlight shooting when I usually crush blacks with their noise and try to catch as much mids and highs as possible. For outdoor shooting, the cine curves are so much better...
__________________
Sony PXW-FS7 | DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive

Last edited by Piotr Wozniacki; February 7th, 2008 at 05:18 AM.
Piotr Wozniacki is offline   Reply With Quote
Old February 7th, 2008, 04:59 AM   #96
Regular Crew
 
Join Date: Dec 2007
Location: Viersen, Germany
Posts: 120
After filming a few shots I think Cine2 looks best.
I tried to get the same dynamic range with the STD3 Gamma, but I used too much knee as you can see.

The STD3-Picture looks much softer und muddy (maybe because I used too much Knee? 70)

http://img163.imageshack.us/img163/3...99cine2kf6.jpg
http://img352.imageshack.us/img352/4...3knee70ex9.jpg

regards Dennis
Dennis Schmitz is offline   Reply With Quote
Old February 7th, 2008, 05:00 AM   #97
Major Player
 
Join Date: Oct 2001
Location: Washington D.C. Metro Area
Posts: 384
You have all over thought this one

The trees are acting as nets or scrims in the "problem" images. They are dropping the exposure of the sky... and hence your results.

As you zoom in, or even better walk closer, to the branches this should "magically" go away. This is because eventually the branches are too far apart to operate effectively as a scrim/net.

You will observe the same effect on diffuse lights, and especially lights with nets or scrims on them.

You will observe this with any camera. The exact circumstances will vary a bit based mostly on the sensor resolution, lens resolution and to some degree on the exposure. The main point is that regardless of what camera you are using you will find some combination of fine elements that will trigger this perception- that includes your eyes.

That is just the way light works.

All the talk of gamma levels and matrices is off the mark. No really - all of it. Go shoot that same scene with an HVX200, RED, a Viper or 35mm and you'll get similar results. (Again accounting for the sensor and lens resolution.) That isn't to say anyone's been wrong- just wrong that it has anything to do with the "problem."

The one note is that the issue is made more prominent by the XDCAM codec. Capture that as ProRes and while you'll still see the problem, but it will be less objectionable.

The case presented here can be "solved" in one of two ways:

1) drop exposure so that the entire sky has a valid exposure level. The differences in exposure in the sky will thus be less objectionable.

2) increase exposure levels so the sky well and truly blows out, then do a sky replacement in post.

3) use a filter or grip kit to bring levels down in the sky. I'd try a graduated ND filter first.

A circular polarizer might help a great deal. It would bring exposures down a bit and may have some effect on the fringing.

If it was a film set with a still camera I'd drop a net very close to the lens (and out of focus) to bring the level down in the sky. The net will also give that portion of the image a little softening. The problem with nets is that the edges are sometimes too thick.

For a lot of ENG/EFP video work I'd just live with it during acquisition and maybe throw a vignette and a soft blur on the image to drop the levels and sharpness there a bit. That would soften the effect a bit. There is no post solution that will really fix that sort of thing though once you shoot it.
__________________
Alexander Ibrahim
http://www.alexanderibrahim.net
Alexander Ibrahim is offline   Reply With Quote
Old February 7th, 2008, 06:32 AM   #98
Inner Circle
 
Join Date: Aug 2006
Location: Poland
Posts: 4,086
OK guys - you got me convinced that this is not actually a flaw, but a feature. I did some backlit shooting today with all the factory setting, but the Knee point of the STD3 curve dialed down from 90 to 75.

The result is that I can no expose the darker foreground much better before they sky even starts to overblow (which was expected). However, once it does, the trees act exactly the same i.e. before the exposure is too high (and whole sky is simply white), it goes though the same stage where the sky areas right behind the trees become blue-gray (ugly and unnatural, as the rest of the sky is already white).

Now, my question is this: apart from changing the knee point, should I also change the slope and/or sauration in order to minimize this effect? Please give me some theory background to this; let me stress that I'm no longer looking for "flaws", but investigating "features" :)
__________________
Sony PXW-FS7 | DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive
Piotr Wozniacki is offline   Reply With Quote
Old February 7th, 2008, 08:00 AM   #99
Trustee
 
Join Date: Nov 2005
Location: Sydney Australia
Posts: 1,570
The steeper the top slope of the curve the less risk of banding in gradient highlights e.g. sky. If you look at the curve you'll see that a quite large change in light (the X axis) produces a very small change in the output (the Y axis).Unfortunately the sky can as you've seen fall right along that slope. Part of it ends up clipped to white, the next darkest region has some color in it but two channels are clipped still so the color shifts.
Drop the exposure and the sky slides down the shallow slope and out of clipping. Increase the exposure and it all clips to white. Except you are maybe overloading the sensor. Look at the curve and see just how much light you're pushing into the camera to hit clipping. With that amount of light the sensor itself starts to do and see wierd things, like lens artifacts, light bouncing around between lens elements and inside the prism.
My suggestion would be not to use that curve in that shooting scenario. If you want to shoot that scene with natural looking blue sky and still preserve detail in the shadows you either need to add light to the shadows or reduce the brightness of the sky. I think someone previously had mentioned how to knock the sky down with a net, this is standard fare, either that or else adding fill lighting to bring the dark parts of the scene up.
Or even simpler, shoot with the sun in a different position or at a different time of day.

One thing that doesn't help is Sony haven't published any real gamma curves. The ones they have have no scales, they give a hint as to what's happening but clearly for example the std curve cannot go to 200% or if it does then the cine curves only go to 50%. Maybe what they mean is the top of the std curve is 100% for that curve and the top end of the cine curves ins 100% for that curve. Putting them all on the one chart like that might be graphically appealing but very confusing.
Bob Grant is offline   Reply With Quote
Old February 7th, 2008, 08:25 AM   #100
 
Join Date: Nov 2001
Location: santa fe, nm
Posts: 3,264
Images: 10
If you look at the black wedge WFM pictures I posted, you'll see that the top end of the gamma curves in CINE mode are a fraction of the top end point for the STD curves, reinforcing the XDCAM HD curves you showed, Christopher.
This surprised me when I first saw them, which is why I, initially, normalized them to 100 IRE.
Bill Ravens is offline   Reply With Quote
Old February 7th, 2008, 09:02 AM   #101
Regular Crew
 
Join Date: Jan 2007
Location: Australia
Posts: 107
Bob and Bill, the F900 Standard REC709 setting is up at around 150 IRE, likely the same in the XDCAM graph.
http://www.siliconcine.net/temp/XDCAM_Gamna_Curves.jpg

Alfonso Parra, DoP, has performed F900 tests on the Digital Praxis custom curves. If you look at this image/graph, at 400%, the F900 Standard curve runs up to 150 IRE. The other curves are the Digital Praxis customs, somewhat similar to the EX1's cine gamma, except closer to Cineon log and some ideal for intense post grading and film out. This provides some perspective of what the STD curve values may be, and IMO, can be applied to the EX1 STD curve, as approximations of the X and Y values. The EX1 is after all part of the CineAlta family.
http://www2.alfonsoparra.com/php/ima...nt/167/049.jpg
Christopher Barry is offline   Reply With Quote
Old February 7th, 2008, 09:10 AM   #102
 
Join Date: Nov 2001
Location: santa fe, nm
Posts: 3,264
Images: 10
This might be a stupid question, but, isn't the implication that the cam is more light sensitive in STD mode?
Furthermore, highlights will appear to clip in any NLE that can't handle IRE values over 100-110%?
Or conversely, crush the blacks if the highlights are exposed properly?
Bill Ravens is offline   Reply With Quote
Old February 7th, 2008, 09:27 AM   #103
Regular Crew
 
Join Date: Jan 2007
Location: Australia
Posts: 107
Bill, I looked at your WFM wedges, thanks. When looking at Alfonso's wedges, they appear exposed 18% grey wedge to 50 IRE on the WFM, and then observe how the different curves responded to shadow and highlight detail.
http://www2.alfonsoparra.com/php/ver...&id=134&ln=eng

The image recorded is not always appealing, however, in post there should be the most amount of detail preserved to extract, however, an F900 and the EX1 have different codecs and data rates, and one will be limited by how robust the EX1, in HQ mode, holds up, and how far your can exploit it in post, if one makes extreme presets, similar to Cineon log curve.

My EX1 arrives next week, so lots of testing to follow, in due course.
Christopher Barry is offline   Reply With Quote
Old February 7th, 2008, 09:32 AM   #104
 
Join Date: Nov 2001
Location: santa fe, nm
Posts: 3,264
Images: 10
Exactly!! ;o)
Bill Ravens is offline   Reply With Quote
Old February 7th, 2008, 09:38 AM   #105
Inner Circle
 
Join Date: Aug 2006
Location: Poland
Posts: 4,086
This is becoming funny... I know you guys don't regard me high as a camea operator, but I have just finished my next couple of tests, and have a couple more surprizes for you:

1. Even with cine curves, at the moment EX1 starts clipping (and you cannot totally avoid it), the resulting colour patches are awful to watch (see the left grab: as if the blue colour resosution was insufficient). In the right grab, I was those tiny fractions of a stop on the safe side, and the sky exposed better.

2. I'm not sure, but I guess I have found the vignetting problem source; do you recognize the upper right corner thingy in the right grab? - well, it is the ND filter not quite snapped in...What a silly construction, and why no knob to rotate it quick and precisely?

Anyway, even though I really appreciate your experience and knowledge, I can now state it with 100% certainty: the highs compression algorhitm on the EX1 is faulty. You're simply wrong saying that all cameras do that; my V1E doesn't do that - period.
Attached Thumbnails
abrupt highlights clipping-image54.jpg   abrupt highlights clipping-image53.jpg  

__________________
Sony PXW-FS7 | DaVinci Resolve Studio; Magix Vegas Pro; i7-5960X CPU; 64 GB RAM; 2x GTX 1080 8GB GPU; Decklink 4K Extreme 12G; 4x 3TB WD Black in RAID 0; 1TB M.2 NVMe cache drive
Piotr Wozniacki is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 01:53 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network