Restore clipped highlight detail in Premiere Pro CS3 with superwhites at DVinfo.net

Daniel Browning · May 27th, 2008, 06:01 PM

The HDV files created by my Canon XH-A1 HDV contain a lot of detail in the superwhites: the 235-255 or 100-110 IRE range. Premiere Pro normally clips that detail, but I found a way to get at it: add the Fast Color Corrector effect and set the output level to 215. This restores the highlight detail in the program window as well as all the output options, including still frame extraction, which has the superwhites mapped to 255.

Unfortunately, adding any other video effect clips the highlights again, even if the Fast Color Corrector is isolated from the other video effects by nesting a sequence.

I would have expected 235 to be a sufficient output level to restore the superwhites, but it actually took exactly 215, not a hair more or less. Sony Vegas shows the IRE clipping at 110 in its YC Waveform, but Premiere doesn't clip until around 117 or so on the same frame.

I tried using Sony Vegas, and it impressed me greatly by saving the superwhites automatically: no extra work necessary. Furthermore, it doesn't clip them in any of the video effects, including curves and levels.

I also tried converting the files to Cineform (Neo HDV trial), but the highlight detail was still clipped when viewed with avisynth+virtualdub.

What does all this mean? Is there any other or better way to restore superwhites in Premiere?

Graham Hickling · May 27th, 2008, 08:25 PM

Interesting post!

Just to clarify ... when you say "I also tried converting the files to Cineform (Neo HDV trial), but the highlight detail was still clipped when viewed with avisynth+virtualdub", are you meaning when you converted the raw files, or when you exported your "FCC plus 215 output" files?

Daniel Browning · May 27th, 2008, 10:04 PM

The raw files that I captured with hdvsplit.

Peter Manojlovic · May 28th, 2008, 10:32 AM

"Premiere Pro normally clips that detail"

I don't think that it's a function of Premiere that does the clipping..Simplified, the legal video signals are being clipped..Whether Vegas or FCP.

Remember, you can theoretically go above 100%, since Chroma allows for more padding (can't remember exactly how much)....

Nonetheless, if you were truly clipping to tape... the import, or editing for that matter couldn't possibly bring back information that's been blown away...
I believe that you're close, but haven't clipped.

As far as bringing down your highlights, i'd personally attack the tonal range first..
Drop the Master Higlights, bring down the Lift, and play with the gammas to taste.
IIRC the RGB color corrector had these options in the slider...

I would only use fast color corrector for reference...

Daniel Browning · May 28th, 2008, 11:08 AM

Peter, for me, the RGB Corrector clips everything over 100 IRE. Are you saying that you can successfully bring 110 IRE (255 RGB) down to 100 IRE in HDV files?

Peter Manojlovic · May 28th, 2008, 12:45 PM

Hey Daniel....

Firstly, let me get back to you on the correct filter. I'm running on memory currently..
Secondly, you can bring 110 IRE down to 20 IRE if you desire. My point being, unless you've applied filters, Premiere's RGB parade (or external monitor) should show you the same as your source footage. Nothing more, nothing less. And if you've blown the highlights at tape, then nothing can be done to bring that information back..Which leads me to believe you haven't blown them yet....

Be careful about your method..Depending on the feel and source, you're remapping all pixels, including blacks...That might not be desired..In your case though, it might be acceptable..
By playing with Higlights and Gammas, you shouldn't be affecting your shadows too much...

I'll get back to you on the filter...

Daniel Browning · May 28th, 2008, 02:29 PM

Quote:

Originally Posted by Peter Manojlovic

Unless you've applied filters, Premiere's RGB parade (or external monitor) should show you the same as your source footage.

In the RGB parade there is a lot of data that appears to butt up against 100. The program monitor shows clipped highlights (without any filters). If I add the FCC 215 output level, the RGB parade drops to just below 90 and the highlights appear again the program monitor.

Quote:

Originally Posted by Peter Manojlovic

Be careful about your method..Depending on the feel and source, you're remapping all pixels, including blacks...That might not be desired..In your case though, it might be acceptable..

Good point. In this case I'm planning to crush a wide range of shadows with an RGB curve, so I'm flexible on that.

Quote:

Originally Posted by Peter Manojlovic

I'll get back to you on the filter...

Thanks.

Peter Manojlovic · May 28th, 2008, 03:48 PM

Yeah, i was right, it was the RGB color corrector...

I've sampled some DV footage. You might find the gain to be very sensitive..
But at least, you don't effect the Pedestal or Gamma drastically..
Regardless, my clipping won't bring back lost detail....No matter how far down i bring it...It was blown out at the tape..

It's always a challenge taping African men (Burundi) against a brightly lit window :).

Daniel wrote:
"the RGB parade drops to just below 90 and the highlights appear again the program monitor."

Then as expected, they weren't totally blown out to begin with. So that's good news.

Daniel Browning · May 29th, 2008, 08:28 AM

I found that RGB Curves can bring down the superwhites easily, but I'm still unable to do it with the RGB Color Corrector, Adjust->Levels, or any other filter.

I shot a quick test video. I would appreciate it if a few people here could throw it on a Premiere project and see how the superwhites are handled on their system.

For me, here's what it looks like with no effects (clipped superwhites):

http://thebrownings.name/video/test/...no-effects.jpg

And here's what it looks like after bringing the superwhites down with FCC or RGB curves.
http://thebrownings.name/video/test/...te-FCC-215.jpg

This is the file captured off tape with hdvsplit. It's XH-A1 HDV 1080p24F.

http://thebrownings.name/video/test/...t-XHA1-24F.m2t

Graham Hickling · May 29th, 2008, 05:03 PM

Daniel, the image looks the same here, with and without Fast Color Corrector applied.

However....I can also get much the same result with Adjust/Levels (by reducing RGB White) and with RBG Color Corrector (by reducing gain).

So when you say "you can't do this" with other filters, what are you meaning exactly? That the suoerwhites disappear? Is so, thats not the case with my computer.

Daniel Browning · May 29th, 2008, 08:11 PM

Quote:

Originally Posted by Graham Hickling

Daniel, the image looks the same here, with and without Fast Color Corrector applied.

However....I can also get much the same result with Adjust/Levels (by reducing RGB White) and with RBG Color Corrector (by reducing gain).

Thank you for checking, Graham. So when you load the m2t in Premiere, you see the pattern detail (dots and diagonal lines) all over the paper towel? Without applying any effects?

For me, the paper towel looks solid white in the Program Monitor, like the first frame I posted (superwhite-no-effects.jpg). I can only get the pattern detail of the paper towel to appear if I apply the Fast Color Corrector or RGB Curves, in which case it looks like the second frame I posted (superwhite-FCC-215.jpg), which is the look that I desire.

Quote:

Originally Posted by Graham Hickling

So when you say "you can't do this" with other filters, what are you meaning exactly? That the suoerwhites disappear? Is so, thats not the case with my computer.

Yes, I mean they disappear in the sense that instead of showing what the paper towel looks like, it just becomes solid white.

Your post has me thinking that something is broken on my installation. Time to put on the debugging hat. :-)

Graham Hickling · May 29th, 2008, 08:50 PM

Sorry, I was unclear in that last email.

What I meant was .... I see the same thing as you with the two images - i.e. no visible detail in the timeline window with the raw footage; detail appearing in the paper towel once fast color corrector is applied and the output level is reduced.

BUT .... what I'm also saying is that I can recover that same detail with RGB color corrector (with gain reduced to 0.9) or with levels (with RGB white output reduced to 220).

Daniel Browning · May 30th, 2008, 12:14 PM

I see now, thanks Graham. I've tried every which-way combination I can think of, but I still can't make Levels or RGBCC work for me. If anyone has the opportunity to try out this little m2t file, I would appreciate it.

Graham Hickling · May 30th, 2008, 12:19 PM

You say you can't recover that detail with RGBCC ... so I'm wondering what happens for you when you reduce gain?

Are you saying that the towel shifts from uniform white to uniform grey, so that none of the texture detail ever appears?

Daniel Browning · May 30th, 2008, 12:38 PM

Quote:

Originally Posted by Graham Hickling

You say you can't recover that detail with RGBCC ... so I'm wondering what happens for you when you reduce gain?

Are you saying that the towel shifts from uniform white to uniform grey, so that none of the texture detail ever appears?

Yes. I add the RGBCC effect to the clip, change nothing except gain to 0.90, and it does exactly as you say: the towel shifts from uniform white to uniform grey; no detail ever appears.

The same thing happens if I use Adjust->Levels and set just the output RGB white to 215. (What's curious is that the same levels adjustment done via FCC *does* work.)

I've also tried creating new project files with different settings. Tried changing bit depth preference, interpret footage, and a few other things. I tried changing a bunch of options in the RGBCC: Tonal Range, Tonal Range Definition, adjust each channel's gain individually, etc. All to no avail.

May 27th, 2008, 06:01 PM	#1
Daniel Browning Major Player Join Date: Jan 2007 Location: Portland, OR Posts: 949	Restore clipped highlight detail in Premiere Pro CS3 with superwhites The HDV files created by my Canon XH-A1 HDV contain a lot of detail in the superwhites: the 235-255 or 100-110 IRE range. Premiere Pro normally clips that detail, but I found a way to get at it: add the Fast Color Corrector effect and set the output level to 215. This restores the highlight detail in the program window as well as all the output options, including still frame extraction, which has the superwhites mapped to 255. Unfortunately, adding any other video effect clips the highlights again, even if the Fast Color Corrector is isolated from the other video effects by nesting a sequence. I would have expected 235 to be a sufficient output level to restore the superwhites, but it actually took exactly 215, not a hair more or less. Sony Vegas shows the IRE clipping at 110 in its YC Waveform, but Premiere doesn't clip until around 117 or so on the same frame. I tried using Sony Vegas, and it impressed me greatly by saving the superwhites automatically: no extra work necessary. Furthermore, it doesn't clip them in any of the video effects, including curves and levels. I also tried converting the files to Cineform (Neo HDV trial), but the highlight detail was still clipped when viewed with avisynth+virtualdub. What does all this mean? Is there any other or better way to restore superwhites in Premiere?

May 28th, 2008, 03:48 PM	#8
Peter Manojlovic Major Player Join Date: Jun 2007 Location: Hamilton Ontario Posts: 769	Yeah, i was right, it was the RGB color corrector... I've sampled some DV footage. You might find the gain to be very sensitive.. But at least, you don't effect the Pedestal or Gamma drastically.. Regardless, my clipping won't bring back lost detail....No matter how far down i bring it...It was blown out at the tape.. It's always a challenge taping African men (Burundi) against a brightly lit window :). Daniel wrote: "the RGB parade drops to just below 90 and the highlights appear again the program monitor." Then as expected, they weren't totally blown out to begin with. So that's good news. Last edited by Peter Manojlovic; May 28th, 2008 at 04:35 PM.

May 29th, 2008, 08:28 AM	#9
Daniel Browning Major Player Join Date: Jan 2007 Location: Portland, OR Posts: 949	Test Video with detail in the superwhites. I found that RGB Curves can bring down the superwhites easily, but I'm still unable to do it with the RGB Color Corrector, Adjust->Levels, or any other filter. I shot a quick test video. I would appreciate it if a few people here could throw it on a Premiere project and see how the superwhites are handled on their system. For me, here's what it looks like with no effects (clipped superwhites): http://thebrownings.name/video/test/...no-effects.jpg And here's what it looks like after bringing the superwhites down with FCC or RGB curves. http://thebrownings.name/video/test/...te-FCC-215.jpg This is the file captured off tape with hdvsplit. It's XH-A1 HDV 1080p24F. http://thebrownings.name/video/test/...t-XHA1-24F.m2t

May 27th, 2008, 08:25 PM	#2
Graham Hickling Trustee Join Date: May 2004 Location: Knoxville, Tennessee Posts: 1,669	Interesting post! Just to clarify ... when you say "I also tried converting the files to Cineform (Neo HDV trial), but the highlight detail was still clipped when viewed with avisynth+virtualdub", are you meaning when you converted the raw files, or when you exported your "FCC plus 215 output" files?

May 27th, 2008, 10:04 PM	#3
Daniel Browning Major Player Join Date: Jan 2007 Location: Portland, OR Posts: 949	The raw files that I captured with hdvsplit.

May 28th, 2008, 10:32 AM	#4
Peter Manojlovic Major Player Join Date: Jun 2007 Location: Hamilton Ontario Posts: 769	"Premiere Pro normally clips that detail" I don't think that it's a function of Premiere that does the clipping..Simplified, the legal video signals are being clipped..Whether Vegas or FCP. Remember, you can theoretically go above 100%, since Chroma allows for more padding (can't remember exactly how much).... Nonetheless, if you were truly clipping to tape... the import, or editing for that matter couldn't possibly bring back information that's been blown away... I believe that you're close, but haven't clipped. As far as bringing down your highlights, i'd personally attack the tonal range first.. Drop the Master Higlights, bring down the Lift, and play with the gammas to taste. IIRC the RGB color corrector had these options in the slider... I would only use fast color corrector for reference...

May 28th, 2008, 11:08 AM	#5
Daniel Browning Major Player Join Date: Jan 2007 Location: Portland, OR Posts: 949	Peter, for me, the RGB Corrector clips everything over 100 IRE. Are you saying that you can successfully bring 110 IRE (255 RGB) down to 100 IRE in HDV files?

May 28th, 2008, 12:45 PM	#6
Peter Manojlovic Major Player Join Date: Jun 2007 Location: Hamilton Ontario Posts: 769	Hey Daniel.... Firstly, let me get back to you on the correct filter. I'm running on memory currently.. Secondly, you can bring 110 IRE down to 20 IRE if you desire. My point being, unless you've applied filters, Premiere's RGB parade (or external monitor) should show you the same as your source footage. Nothing more, nothing less. And if you've blown the highlights at tape, then nothing can be done to bring that information back..Which leads me to believe you haven't blown them yet.... Be careful about your method..Depending on the feel and source, you're remapping all pixels, including blacks...That might not be desired..In your case though, it might be acceptable.. By playing with Higlights and Gammas, you shouldn't be affecting your shadows too much... I'll get back to you on the filter...

May 29th, 2008, 05:03 PM	#10
Graham Hickling Trustee Join Date: May 2004 Location: Knoxville, Tennessee Posts: 1,669	Daniel, the image looks the same here, with and without Fast Color Corrector applied. However....I can also get much the same result with Adjust/Levels (by reducing RGB White) and with RBG Color Corrector (by reducing gain). So when you say "you can't do this" with other filters, what are you meaning exactly? That the suoerwhites disappear? Is so, thats not the case with my computer.

May 29th, 2008, 08:50 PM	#12
Graham Hickling Trustee Join Date: May 2004 Location: Knoxville, Tennessee Posts: 1,669	Sorry, I was unclear in that last email. What I meant was .... I see the same thing as you with the two images - i.e. no visible detail in the timeline window with the raw footage; detail appearing in the paper towel once fast color corrector is applied and the output level is reduced. BUT .... what I'm also saying is that I can recover that same detail with RGB color corrector (with gain reduced to 0.9) or with levels (with RGB white output reduced to 220).

May 30th, 2008, 12:14 PM	#13
Daniel Browning Major Player Join Date: Jan 2007 Location: Portland, OR Posts: 949	I see now, thanks Graham. I've tried every which-way combination I can think of, but I still can't make Levels or RGBCC work for me. If anyone has the opportunity to try out this little m2t file, I would appreciate it.

May 30th, 2008, 12:19 PM	#14
Graham Hickling Trustee Join Date: May 2004 Location: Knoxville, Tennessee Posts: 1,669	You say you can't recover that detail with RGBCC ... so I'm wondering what happens for you when you reduce gain? Are you saying that the towel shifts from uniform white to uniform grey, so that none of the texture detail ever appears?