EX1 Noise In The Shadows?
 

Hi folks,

I saw another post some time ago about noise in the shadows under halogen lighting.
I can't seem to find it..... can anyone point the way?

Why am I getting so much noise in the shadows under halogen lighting? Is there anything I can do?

Yes: Maximize the use of the dynamic range of the sensors (which is very good compared to other sub 10k-camcorders), by using the cine1-gamma (it's got the most steep curve at highlights compared to all other curves, so it minimizes the chance, that the picture looks overexposed because of a flat curve, though there's still some unused exposure-headroom) and using above-100IRE-information.
Also think about decreasing the contrast in the shadows a bit, to get a more punchy look. This of course will decrease the shadow-noise (transforming grayish noisy shadows to more black shadows).
I wrote a FCP-plugin for altering the contrast in the way you want it to:
http://www.dvinfo.net/conf/non-linea...st-plugin.html

I forgot that part...
Because halogen light has a lower white-temperature, so especially the blue (low wavelength) channel is darker and has to get amplified (done by the white-balance-feature of the camera) to satisfy the specification that white means the signals of the red, green and blue channel are equal strong. And that amplification of course also amplifies noise.
If you want to know, why the blue channel gets dark at low white-temperatures and what white-temperature actually means, read that article:
http://en.wikipedia.org/wiki/Thermal_radiation
If your halogen-source is bright enough, it makes sense to use a blue-filter in front of your light-source or (if there are many light-sources) your lens, to fit the optimal white-temperature of the camera (optimal means if no channel gets amplified then something white results in red=green=blue).

Big thanks Dominik :)

I guess I never expected to see so much noise with the EX1.

I tried to plug your filter in but it does not show up in the FCP effects list.
Library/ Application Support/ Final Cut Pro System Support/ Plugins.

I know I can crush the blacks in post to bring down this noise but am wondering if there is another way. I've tried Bill's Cine 1, Cine 3, Cine 4 and another STD Gamma, HiSat3 and none of them help to reduce this noise in the shadows.

I remember reading a comment 2 - 3 months ago from another person on this site who was a little surprised with the noise as well.

No, it's ~/Library/Plug-Ins/FxPlug/
Cine4 and the stds are bad, because they've got a high shadow-contrast, so the noise in there gets even more accentuated.
The ex1 produces not so much noise. It's more a high shadow-contrast, which makes it more visible. If you would compare a picture of an ex1 (with a cine-gamma) to a picture of for example a fx1 of the same scene, the first thing you would notice would not be higher noise of the ex1 in the shadows, but a much more flat/high-dynamic-range look. You have to realize that those gammas are coming from very professional 100k-cameras, which probably have higher-quality electronics/better SNR.

Hi Dominik,

I'm running 10.5.4 on a Macbook Pro and do not see this ~/Library/Plug-Ins/FxPlug/. It's just not there.

I understand your comment on the better cameras with their SNR and the EX1's high/ flat dynamic range look.

Been seeing some great things from the new Canon 5D.

You have to create it! I guess I should make an installer for it...
Good. What I was trying to say is, that other sub 10k-cameras (except Canon 5D ;)) would look much noisier if you would apply the same gamma-curves to their pictures.
Did you see 1080p-footage from it? Do you have a link? I read about it and am very interested.

for the 5D there is the Canon site:

Canon Digital Learning Center - Sample EOS 5D Mark II Video: Reverie

and this blog:

ProLost: Reverie

I'll make a folder for your plug in but I'm not going to hold me breath for FCP to know the file path.

I found your post quite interesting, but can't help to think something is getting lost in translation here. You are in Germany so I'm thinking your first language is German, so maybe that's why I'm confused.
Actually, Cine 1 and Cine 4 have the same dynamic range. They clip on the same point too, so Cine 1 doesn't make more use of above-100IRE info.


Did you mean "increasing" contrast? Decreasing it won't get you a more punchy look.


This is hardly a good solution for noise. All you are doing is crushing your blacks in camera. You can do that in post too. Grayish blacks is a way to get more dynamic range out of the camera. Most HD shot for filmout or color correction go for "grayish" blacks or a flatter look to leave more options for post-production.

I think we should agree on a common and unambiguous definition of "contrast", and hence increasing or decreasing it.

In the common sense, high contrast = punchy, with less gradation.

In the specialist jargon: high contrast = a lot of mid tones (gradation), i.e. less "punchy".

When you adopt the latter concept, the somewhat vague description of Cine Gammas in the EX manual adopt a bit more meaning :)

Yes, but because of the heavy highlight-compression of cine4:
-the picture often looks overexposed/too bright, although nothing clipped, leading the cameraman to reduce the exposure and loosing signal/noise-ratio.
-highlight-information gets lost because of mpeg2-quantization
(The above-100IRE-hint is independent from the cine1-hint. You should check the parenthesis ;))
With "contrast" I mean the slope of the gamma-curve at a specific brightness-point.
Decreasing shadow-contrast automatically leads to increased midtone-contrast, if you think about a s-curve (like I did).
I didn't want to recommend crushing blacks in the camera, but a smooth gradual increase of contrast from black to midtones in post-production. That's why I mentioned my plugin, because with that you can do exactly that.

Again, Cine4 is the same as Cine1. Just the midtones are brighter. Cine4 is not bad at all. The shadows in Cine1 and Cine4 are the same. So I don't understand why you say Cine1 makes cleaner shadows.

Don't really understand what you mean by "shadow-contrast" here.


I have also found filming under HMIs and Daylight the EX1 produces a cleaner picture than under tungsten lighting. Even more interesting is that dialing the white balance down to about 2700-2900 also “seems” to produce cleaner images under the same circumstances. Boosting the gain up produces less noise with a white balance of 2700 under tungsten lighting than using a proper white balance temperature. It seems to be related to the red channel or maybe the IR contamination.

Using a filter over the lens sounds more practical than gelling every light. Problem with that is that first you would have one more filter over the lens since you must use a IR cut filter with the EX1. Second problem is speed. What filter are you using over your lens and what grade?

Well, that’s a mistake of the operator not the Cine4. That shows the need of a waveform monitor to proper check levels of to know how the gamma you are using behaves.

Do you mean dynamic range when you say contrast? Decreasing dynamic range would of course produce a punchier look.

By shadow-contrast do you mean just contrast? The difference between light and dark areas?

I’m sorry but I don’t think we need to agree on any common term to define what you are talking about. The term already exists and it’s called dynamic range.

More contrast means more punch in the common or professional sense. Now the ability to record more contrast which is what you are talking about is defined by dynamic range.

High dynamic range = a lot of mid tones (gradation), i.e. less "punchy".

No. The shadows of cine4 and the shadows of the stds are the same. That's the cause why cine4 is called "video bright". Cine1 has much darker shadows (and darker mids) than cine4.
With weaker shadow-contrast I mean that the slope/derivation/rise at the lower end of the gamma-curve is more flat, so the shadows look darker. A steep slope of the curve at the lower end would accentuate the shadows, because steep means, that the same change of input results in more change of the output. Flat means the same change of input results in less change of output - and then because we're coming from black (we're talking about the lower end) it looks nearer to black.
If you don't understand me, I can create a graphic for explanation.
I don't have problems with IR, although I don't use an IR-filter.
That's why I wrote "if your halogen-source is bright enough".
I've got halogen-lights with blue-filters attached and some kind of a gas-light which produces daylight-white.
Yes, it's the mistake of the operator to use cine4, if the scene doesn't have much dynamic-range. Flat scenes want to be shot with cine1.

Were you thinking of my thread? Should footage be this noisey? - The Digital Video Information Network

Btw, I would interpret the terms "contrast" and "dynamic range" in this way:

I really had the impression that what seems to get lifted when comparing Cine1 to Cine4 are the mid tones while high lighting handling and shadow detail would stay the same. When I switch between Cine1 and Cine4 my shadows don't seem to change. Just the mid tones. Do this test: Set the camera to auto-iris and point it to a chip chart and watch the mid gray chip. Now change between cine1 and cine4. You will see that the mid gray will vary but the auto-iris will not change the exposure.



So by shadow-contrast you mean shadow gradation?

If not, a graphic would be great ;-)




Really? Pretty much every EX1 has IR problems. Are you using a 80a or 80b filter or just bluing all your lights and avoiding tungsten all together? Maybe that could explain why IR wouldn’t affect your camera much as tungsten is most responsible for accentuating IR contamination?


So if I understood you right, you seem to recommend Cine4 for high contrast scenes like a day exterior with lots of shadow and Cine1 for low contrast scenes. But how about low light? Low light is a kind of low contrast and yet cine1 is so bad at it .


To sum it up, what gamma do you think:



Produces less noise?



Records more dynamic range?



Best for High contrast situations?



Best for low light situations?



Best for low contrast situations?

from what I've seen, all the cine settings make compromises of some sort. If you want a RAW spectrum captured, that has as much data as this cam/cmos can capture, use a STD setting.(not std2). Then post process to get the best image possible. All the cine settings are compromises to make the monitor look good.

OK, I'm sure I'll get an argument, but, trust me....LOL

Cine4 HAS lighter shadows than cine1 (black of course stays black). I will post a test-image soon.
The auto-iris won't change because the gamma-curves get applied after exposure-measurement.
I attached an illustration. I hope that makes things more clear.
(These curves aren't measured but customly set up by my estimation/experience.)
I don't know the filter-types. They were included with the lights.
I just checked it out again without filters and yes, there are some reddish colors even after white-balance. I guess I oversighted it becauce I was used to that tungsten has very warm colors.
Yes.
No, low light is not concerned with low contrast. Low light is just low light, like a low constant factor before everything. If you put a too strong nd-filter before your cam, even a daylight-situation becomes low-light, but that doesn't change the contrast (ratios between dark and light areas).
The gammas don't produce noise (except the stds, read my comment to Bills statement underneath), but are more or less accentuating dark-areas (where the SNR is bad).
Cine1-4 record basically the same dynamic range. But the stds without knee have less dynamic range, because they're throwing away sensor-range which was left as dynamic headroom for knee (see attached illustration).
Cine4. Or cine1 with slight underexposing and compensation for that in post (my preferred way).
I don't like cine3 a lot, because it's got some built-in negative black stretch.
Is not concerned with gamma-curves. If it gets dark, you need gain/different exposure-settings - not a different gamma.
Cine1.
No, stds without knee are wasting sensor-range (see attached illustration) and so are downgrading SNR. With properly set knee they are not neutral ("raw") any more.

Dominik, it has been an useful and interesting conversation. But I’m still confused with what you are trying to say, especially that you seem to be starting to contradict some of your earlier posts.

I’m looking forward to the images then. Because on my tests, shadows didn’t change and I’m not talking blacks.


Thanks for the graphics. I think I understand what you mean by shadow-contrast or high light contrast. It’s what I would refer to gradation or how much difference you see in the shadows or high lights as in, do they clip abruptly or gradually. Right?

This is what I thought. Maybe a 80 filter on the lens instead of the IR one would be a better move as it would potentially “cure” two problems with one strike? Although the 80 filters cut 2 stops. Not sure how much the 486 or the new IR filter cuts if any.

Well, low light means not enough light for an exposure yes, but most low light tend to be low contrast especially indoors.

That’s noise. If Cine4 is indeed brighter in the shadows it means it is lifting the blacks and if it is lifting the blacks it will produce more noise in the shadows. You said it yourself earlier that Cine1 is cleaner in the shadows right? You were talking about noise I thought here:

So for you, which of the gammas produce less noise?

I don’t understand how they can all record the same dynamic range in your opinion when you say Cine4 records more info in the blacks. That would mean it records more dynamic range than Cine1 for example. If you recommend Cine4 for high contrast it also means it records more dynamic range. Or?

By compensating in post you mean lifting the blacks? This will cause noise. Or did I miss understood you?

And so you recommend Cine4 for high contrast but use Cine1?

I thought Cine4 had built in black stretch, if it produces brighter shadows .

Actually, it is. This is the reason the F900R and F23 have hypergammas for high contrast and for low light. If it was not related to gamma why would there be a hypergamma for low light?

By the way, I found a post by you from another thread which I don’t understand either. :)

Here it is:
What do you mean by “and take noise”?

Bill, what std gamma do you use and what knee, black, black gamma etc settings to get the widest dynamic range or RAW as you put it?

Interesting discussion. It seems Dominik (whose observations / opinions I share almost 100%), gives the term "contrast" a meaning I proposed, which was rejected by Michael :)

STD gammas are producing some very strange white lines (halos?) in high contrast areas.
So I won't recommend using them.


Dennis

Comparison is attached, and cine4s shadows ARE lighter.
Yes. Hopefully they don't clip.
That heuristic is of course true.
It's not producing, but accentuating, because the signal-to-noise doesn't change, because the the picture-information in those shadows gets amplified, too.
Again, the cines are all identical in terms of signal-to-noise-ratio. But the stds without knee lower the clipping-point, so you have to recude the exposure to capture the same highlight-information, and so you are reducing the signal-to-noise-ratio in all picture-areas.
Theoretically, as long as an used gamma-curve does/can have an inverse, it can't destroy information. Practically, there's quantization-noise especially caused by the mpeg2-compression, which compression-technique is build on reducing quantization-precision on frequency-components which are rated as "less important" for our human visual system. But experience shows that all gamma-curves are accentuating the shadows/shadow-noise so much, that the quantization-noise is almost negligible compared to shadow-noise.
There are different ways to to judge right exposure:
-no highlight-information is gone
-the average brightness is middle-gray
-the main-subject in the picture is exposured well
If you use the first, then it doesn't make a difference which (cine-)gamma-curve you choose.
If you take the second, then it does matter. But because you can do all that gamma-stuff in post, you should concentrate on capturing most relevant information, which almost doesn't deppend on the (cine-)gamma. "Relevant" shall express, that capturing all highlight-information can be a bad idea, if your (darker) main-subject then vanishes in noise.
I said that cine3 has "negative", so the opposite of, black-stretch, so cine3 makes shadows darker. Cine3 also has more highlight-compression than cine1. So it's a very unneutral gamma, but I want to capture pictures neutrally, so I don't like it.
The cine-gammas use more of the available sensor-range than the stds (without knee), so they must look darker, because what a std shows as white the a cine shows as gray, and what a cine shows as white a std shows as clipped.
Using more sensor-range allows to use more of the available light by increasing exposure. But increasing exposure won't increase sensor-noise, so the signal-to-noise-ratio increases.

sorry, guys. got pulled into this discussion once before. not this time.

I see it. Weird that in tests I did using a waveform monitor I could only see difference in the mids. I will have to retest that.

I’m assuming the aperture was the same for all gammas right. Because to my eyes, Cine4 doesn’t really look as bright as std3 in those pictures. It seems just a bit darker than std3 in the shadows. At least less contrasty in the shadows, to try to speak in your terms :)
Also, looking at your samples, I don’t see why Adam Wilt called cine3 “brighter cine” since cine3 seem to have darker shadows than cine1.
I guess one of the things which is making it so confusing is the technicalities which clearly are what is keeping us from getting to a common understanding because we don’t seem to agree on the technical terms here. But yes, it doesn’t “produce” noise which is not there, but what I obviously meant was which gamma produces the cleanest image with the least visible noise?
That would only mean that when exposing to high lights using the stds without knee you would capture less info in the shadows because they would be underexposed. It would not necessarily mean the shadows would be noisy. If using for example std2 which crushes the blacks you shouldn’t see much noise.

But what you are saying is all cine gammas has the same amount of visible noise?
:
Here we got lost again. I’m talking about dynamic range and or latitude or how many stops of info can be recorded and you are talking about something else ;)
How does that explain why you recommend Cine4 for high contrast but use Cine1?
So what you meant by “and take noise” is that the cine gammas show less noise?

By the way, you totally bypassed my reply about gamma having an effect on low light and the F900R having a hypergamma preset for low light while you said it wasn’t related to gamma. Mind commenting on that?

Dominik, although I think this is an useful discussion, let try to be objective rather than getting lost in technicalities which will probably just add more confusion as it’s clear there are some words getting lost in translation here. ;)

The shorter way sometimes is the better one.

Cheers.

Mike


I

Ok Bill, without getting pulled into the debate, do you mind giving your settings?

No rejecting Piotr. Just trying to find a common ground so I can understand where he is coming from and what he means ;-)

Michael...

Lighting situations, especially outside, are wide and varied. You're right about knee and slope. Every situation requires custom tuning. Really can't provide a "one size fits all" for you. Setting up requires judicious use of the histogram and the zebra's. The non-linear nature of the way gamma is applied really demands that over-exposure be carefully controlled. I'd MUCH rather under-expose than over-expose.

Ah, how easy was life when we still used old "chemical" film ;-)
Underexposed slide films (positive) and overexposed negative films. Sooo easy.
I have now clue what to do with video. P.

Peter...

hope this helps your understanding:
http://www.adobe.com/products/photos...near_gamma.pdf

With the very close relationship between the EX's cinegammas and Sony's Hypergammas it might be worth taking a look at Sony's guide to Hypergammas. The curves are very similar.

Sony : Digital Cinematography with Hypergamma : United Kingdom

Do you know how close they actually are to the hypergammas?
Looking at the hypergammas there are only two kinds, for high contrast and for low light with a broadcast safe version and a full dynamic range version of each. The cinegammas don't seem to be like that at all. I have read the article you linked to before, but without knowing how close they are and which cinegamma correspond to which hypergamma its difficult to apply any of that to the EX1.

Of course.
Yes. The shadows are very similar. But the hightlights are brighter. Actually clipping-bright. ;)
The shadows of cine3 are darker, but the mids and highlights are brighter. So integrated over the whole range cine3 is brighter.
Cine3 is the best for low shadow-noise, because it has the darkest shadows of all gammas.
But when I'm talking about that the SNR doesn't change, I try to explain, that the actually captured information is the same. So you can change the gamma in post without (significant) loss.
In terms of SNR the shadows will be more noisy. You can think about std3 without knee is similar to cine1 with 3db extra-gain.
Yes, but you're not only loosing noise but also picture-information. I don't want to loose picture-information, because that's irreversible. You can't undo that in post.
I'm talking about SNR, because dynamic-range/lattitude deduces straight from maximal SNR.
Because the SNR is the same for all cine-gammas, you're capturing almost the same information with each of them. So it's just a matter of taste, whether you want to approximate the final look in the camera or to shoot more linearly done with cine1.
If you switch from a cine-gamma to a std-gamma without changing the exposure, then the shadow-signal-to-noise-ratio doesn't change (because the amount of light falling onto the sensor stays the same).
-But the clipping point gets lower (because of the std-without-knee-3db-extra-gain-thing), so you're loosing highlight-information.
-When compensating for that by lowering the exposure, the SNR drops.
Choose your favourite.
I commented on that. Look for the word "heuristic" in my last post. ;)
Yes, it makes sense to say that low-light-situations often are low-contrast-situations. So it makes sense to use cine1 on that, because cine1 the has strongest overall-contrast of all cine-gammas.