Compression theory questions...
 

My question is rather technical and theoretical, but here goes:

I've shot a film in HDV, using a Sony HVR-Z1 with a letus35 adapter.
While I shoot the material is stored on DV-tapes.

The HDV compression algorithm is some kind of MPEG2 compression, right?
What this means is that, apart from inter-frame compression, every frame
is transformed to fourier-space, or at least something similar. I know
something about fourier transforms, but not enough to have a clear idea
of how this works. One way to describe it, I think, is that a frame,
existing in the camera chip as a matrix of pixels, each described with
a X-bit value, or maybe three X-bit values, is TRANSFORMED into a bundle
of sinus-shaped waves, described by a wavelength and an amplitude and a
phase. [If you don't follow me this far, you can close this post with a
clean conscience] I'd guess that other kinds of compression are applied too,
before the images are stored onto tape.

Now, where does the stated 8-bit limitation of the HDV compression come
into the picture? I understand that an image, stored as a pixel map,
each pixel being described by, say, 10 bits, can be said to be "a 10-bit image".
But how can a bunch of sinus waves be described as "8-bit"?

The holy grail for me, of course, is: once the HDV compression has taken
place and the material is stored onto tape, is there some way to extract
more information from the tape than I do now, that is, capturing into
an Adobe Premiere native HDV-project?

For example, will a product like Prospect HD give me better image quality
than capturing as Adobe Premiere native HDV, besides the obvious advantage
when color correcting and applying other effects in 10-bit?

I would be really, really grateful if someone with a robust insight into
these matters would take his/her time to explain this. I've surfed wikipedia
and some other sites but the info I've read somehow always miss the target.

On a more practical note, I'm wondering if I should invest 199 USD to upgrade
from Aspect to Prospect.

Regards,

Nikolaj Marquez von Hage

mpeg2 is more tricky than the process you describe, but whatever it is, there is only one thing to remember: It is a lossy compression scheme.
So it means what you loose at compression cannot come back at decompression.
But there are some codecs that try to uncompress the mpeg2 to a higher level quality by applying filters. What cineform does can be done another way (with virtualdub for example), but why bother with that when cineform will do it in realtime at capture ?
Since this filtering process involve heavy calculation, and calculation include rounding/overflow values at some steps, you can imagine that working on 10 bits when original signal is at 8, will allows to minimize the loss/errors due to rounding.
and this safe margin is here not only at capture but also during editing, so yes , i think if you are looking for best quality without extra work/time , spending 199$ is cheap for the opportunities it gives.

Nikolaj... Let me first admit that your technical description made my eyes glaze over. I'm just not that technical.

That said, the benefit of 10 bit over 8 bit really comes into play when you start pushing colors around in post. It gives you more room to work in. If I'm going to get really serious with a Photoshop image, I'll convert it from 8 to 16 bit for that very reason. You probably already know this, but I just wanted to make sure.

BTW: If you want to upgrade to CS4 or stay current with the latest Cineform software you'll have to upgrade to Prospect since Aspect is going end of life.

Thanks
 

Thanks for sharing your thoughts.
Regards, Nikolaj

Interesting question - having just upgraded from Aspect to Prospect I've been pondering the whiole "10-bit internal" thing also.

One possible analogy: think of a still image bmp ("uncompressed") versus as a jpeg (compressed). The bmp already has a bit-depth BEFORE any jpeg compression algorithm is applied ... in the case of a digital camera that bitdepth I assume gets "applied" in converting the analogue readout of the sensor into digital 1s and Os.

Perhaps similar for HDV - bitdepth being set before the mpeg2 compression occurs?

Graham, that's a plausible hypothesis... interesting. Another idea I had is that
the "bit" part applies to the wave amplitudes somehow? But it gets mathematically
complicated to convert the concept of image quality from wave-space to pixel-space
I guess.

A typical situation where 10-bit is preferred is smooth gradients in an image.
Using a letus35-adapter for shallow depth of focus, that's a common situation for
me: a background wall being out of focus and therefore appearing as a homogeneous,
monocolored surface, slowly changing its color value across the image.

Using Aspect HD (8 bit) I found that the dreaded "color banding" was somewhat more
apparent than when using native HDV capture to Adobe Premiere. Confused (having
only heard praise for the Cineform solution) I discussed it with the Cineform
support. Their explanation was, if I'm not misinterpreting it, that Aspect HD
represents the image as clear and without noise as the HDV material permits.
Native HDV, on the other hand, is more noisy. The noise, paradoxically, may
contribute to perceptually "better" image quality by hiding the banding, that is there.
Cineform actually suggested that I add a small amount of noise to hide the
color banding.

I interpret Giroud as saying that Aspect HD uses "reverse engineering" filters on the
compressed HDV material, to remove, as far as possible, artefacting that results
from the MPEG2 compression - uncompression scheme. That seems to imply that yes,
one can extract more image quality than native HDV contains??

I would say that ultimately, you cannot restore missing data, but you can work on the existing one to guess (interpolation) the missing data.
Usually this is working well, but a filter can create annoying artifact, that is why most filters can be applied at different strength/level to allow getting the best.

but again , there is a world between saying that if you get a black pixel then a white pixel, the missing pixel has good chance to be gray, and what was really in the picture.