The Math on Downconversion

[Adam Reuter]

I was originally going to reply to this thread http://dvinfo.net/conf/showthread.php?t=116196 but I think my post deserves it's own place. This camera is tack sharp...at least tackier sharpier than what most of us have been used to. And it introduces FULL RASTER footage into the prosumer HD equation that I think needs to be addressed.

Note: I am not a computer science wizard. I just have enough years with Photoshop and encoding footage under my belt to understand some concepts I will discuss here.

A good mathematical reason that HDV (SD) footage shot with the EX1 looks better than HQ footage is that computers/algorithms are more efficient (i.e. better quality output) with even integers. This is why in Photoshop (or pick your photo editing/viewing application of choice) if you scale (aka downres) your photo to 50% it looks perceptibly better (less jaggies, smoother) than when you zoom out to 33.3% or any uneven number. 25% also looks nice because it’s half of 50%.

It’s even easier to see at 66.7% vs. 50%...the jaggies are clearly evident! Look for fine lines like hair in portraits to REALLY see it. Put that image in motion and that’s the twitter/flickering you’re seeing! I can clearly see this on my HP w2207 LCD screen but if you are on a lower resolution monitor you may not see much difference. And keep in mind that a still image is like "progressive scan". Add interlacing into the equation (which you will even if you shot your footage at 720p) and you have more issues.

HDV resolution is 1440x1080. Standard definition resolution (in NTSC land) is 720x480. 720 is exactly half of 1440. And although half of 1080 is 540 the lines that are downrezzed are still even. And many SD cameras capture at 540 lines (DV I believe has a theoretical max of 525 lines) so it’s VERY close to the optimal number.

Full raster res on the other hand is 1920x1080. If you divide 1920 (horizontal HD res) by 720 (horizontal NTSC SD res) you get 2.666666666… Or approximately 2 2/3. For PAL it’s about the same for PAL (3 1/3). Again remember the Photoshop analogy. Half the res of your photo looks good (no matter what the resolution of your original photo) but 1/3 of that image or 2/3 of that image (66% zoomed out) does not look as nice.

So, why do computers…err…compute better with even integers? 1’s and 0’s baby!

With all of this said, the better your downressing system (be it software or hardware) the better math it can do and the better techniques it has to work around the rounding off of the integers. I guess in the computer world that’s called “error correction”, although I may be wrong about that. This same reasoning is why some recording studios choose to record at 88.2kHz sampling rate instead of the 96kHz that their hardware is capable of. 88.2 = double that of 44.1kHz, the sampling rate of music CDs. Now if the final output of their recording is for video, which is usually 48kHz then there they would use 96kHz or 192kHz if they have it. 96/2 = 48 and 192/4 = 48.

It only makes sense that outputting with a Kona card would look better than most software solutions. These cards cost many times more than the software and are DEDICATED for tasks such as up and downversion from HD to SD and back. If you have that hardware…USE IT!


One last side note: It's because of this downres/upressing battle that I absolutely HATE IT when movies or documentaries that were shot with standard definition cameras upres their footage to film they then downres it from the film back to standard def. All of this scaling creates a worse image than if they had just used the standard def footage to begin with! Supersize Me comes to mind (I know PD150's look better than what was in that film!)

And gosh darn it can these camera makers make up their minds on interlacing fields. Lower field first, Upper field first. This mixing of interlacing options is why a lot of footage you see in documentaries where they use multiple sources looks like crap (motion-wise). Someone didn't set the proper flag for the footage and all hell broke loose. Once 1080/60p is out none of this will be a problem (or you can shoot at 720/60p for now).

[Adam Reuter]

One last thing...for now.

Anyone who has tried using standard definition interlaced footage on the Panasonic HVX-200 can attest to this same issue. The camera does a horrible job at downressing its chips and the flickering in the footage is ridiculous compared to a DVX100. This can be "fixed" by changing settings related to sharpening at whatnot but it kind of defeats the purpose of such a camera.

My question for EX-1 owners: How does a 4:3 center crop look with the EX-1? If you've done it successfully than do you have any tips for workflow?

[Dennis Dillon]

Adam,
I have used my PDW 530 XDSD camera as a reference. After adjusting the blacks to match the 530, I raised the detail on the EX to +50. This made it a great match to the SDXD. I have output the EX SD SDI to an XD SD Deck and made what my clients thought was another disk from a PDW 530.

[Dennis Dillon]

Adam
also Hi SAt

[Gabriel Chiefetz]

Thanks for the interesting explanation, Adam. However, I get mediocre to terrible results when downrezzing 1440x1080 HDV from the Z1 in FCP. Putting aside all UFF/LFF interlace issues, the SD result still has nasty stairstepping. So your assertion that 1440x1080 material downconverts well doesn't match my experience.

I've started doing the downrez in-camera on the Z1, which produces excellent SD. But it makes me wonder if I should just forget HDV acquisition and shoot on a DVX if the deliverable is SD.

It's a rather sorry state of affairs, and it's not good to hear that the EX-1 (and I'm guessing the Z7 as well) will continue the problem or make it worse.

[Ali Husain]

Originally Posted by Adam Reuter

A good mathematical reason that HDV (SD) footage shot with the EX1 looks better than HQ footage is that computers/algorithms are more efficient (i.e. better quality output) with even integers. This is why in Photoshop (or pick your photo editing/viewing application of choice) if you scale (aka downres) your photo to 50% it looks perceptibly better (less jaggies, smoother) than when you zoom out to 33.3% or any uneven number. 25% also looks nice because it’s half of 50%.

It’s even easier to see at 66.7% vs. 50%...the jaggies are clearly evident! Look for fine lines like hair in portraits to REALLY see it. Put that image in

if you point sample then a non-integral downscale will look horrific, as it is not sampling on the actual edge. an integral point sample will merely look bad... depends on the image of course.

high-quality resampling has other problems, but at that point integral vs. non-integral sampling shouldn't make much difference.

you are correct in that trivial resampling algorithms are more efficient than non-trivial ones. :)

Originally Posted by Adam Reuter

With all of this said, the better your downressing system (be it software or hardware) the better math it can do and the better techniques it has to work around the rounding off of the integers. I guess in the computer world that’s called “error correction”, although I may be wrong about that. This same reasoning is why some recording studios choose to record at 88.2kHz sampling rate instead of the 96kHz that their hardware is capable of. 88.2 = double that of 44.1kHz, the sampling rate of music CDs. Now if the final output of their recording is for video, which is usually 48kHz then there they would use 96kHz or 192kHz if they have it. 96/2 = 48 and 192/4 = 48.

the algorithms are simpler to get from 88.2->44.1 than from 96 to 44.1. with high quality resampling, the results are equally good with either 96 or 88.2. it's more paranoia than lack of a high-quality downsampling route that makes people use 88.2.

[Bill Ravens]

A lot of work by very knowledgeable people is going on to resolve the flicker problem. Here's a(practically) perfect way to remove that ugly flickering, especially when down-converting.

You can frameserve, or manually run your intermediate footage thru VirtualDub. Apply the "resize" filter with the following settings:
Choose Lanczos
Tick the "Interlaced" box.

This might seem counter-intuitive for progressive footage, but, trust me on this, until you try it.

If you don't want to resize, set up the resize filter for zero size change.

[Dennis Schmitz]

Originally Posted by Bill Ravens

A lot of work by very knowledgeable people is going on to resolve the flicker problem. Here's a(practically) perfect way to remove that ugly flickering, especially when down-converting.

You can frameserve, or manually run your intermediate footage thru VirtualDub. Apply the "resize" filter with the following settings:
Choose Lanczos
Tick the "Interlaced" box.

This might seem counter-intuitive for progressive footage, but, trust me on this, until you try it.

If you don't want to resize, set up the resize filter for zero size change.

Doesn't work with progressive material

regards Dennis

[Bill Ravens]

It works for me, Dennis...and I shoot progressive, exclusively.
No jaggies, no combing...it just works. can't say it much simpler.
Is your frame mode truly progressive? I think not.
And if it is, did you get the field order right?