4:2:2 Color, is it Worth it?
 

There ARE ways to get 4:2:2 color out of HDV camcorders (by using their HDMI, component or HD-SDI outputs). But is it worth it?

I imagine for fast moving subjects, the answer may be yes. But I'm just not sure if 4:2:2 is worth the considerable extra hassel and expense.

Your thoughts on the subject?

Thanks much!

Well you also get to avoid lots of compression. If you are shooting blue/greenscreen or special effects work or lots of color grading, that can be helpful.

2- What are you shooting? If you're shooting a doc, then the higher quality recording is not that feasible (unless you get something like the Red camera).

The first step of my documentary is to shoot the talking head interviews. Not a lot of movement subject movement. But not a lot of camera movement either. So I COULD capture to a computer. Cineform would give me the color and a reasonable bandwidth/file size.

If you're downscaling to SD you can get close to 4:2:2 as part of the process. If you're delivering on SD DVD it's only 4:2:0. I'm not suggesting it's not desirable to acquire 4:2:2 or better but from a practical point of view and depending on what you're shooting I think there's more important things to worry about.

As for HDV and the motion problem, overrated from what I've seen.

I definitely want to have an HD final cut. I agree about other things being more important, namely SOUND. But I'll get that next.

Is it important? That depends on what you are doing. You are shooting interviews without a lot of movement. Are the interviews going to be shot on green screen for chroma key in post? Are you going to need a lot of color correction in post? If the answer is no, then 4:2:2 is not necessary in your situation. By the way, 4:2:2 has nothing to do with motion... bypassing HDV compression does. In my opinion, the extra color information is not worth it unless you need to do green screen or have to color correct in post.

John, thanks a lot for your insight, it's much appreciated ;).

One of the aspects of HDV compression is using 4:2:0 luminence and color sampling. Reading up on this, it seems that 4:2:2 can actually have motion artifacts as well (just not as many as 4:2:0). Only 4:4:4 color should be truly artifact free, FWICT.

So are we saying that 4:2:2 is really no better than 4:2:0 and might actually be worse? I always though a larger colorspace would be beneficial. Frankly, due to the mathmatic nature of these terms, I wouldn't think it would be possible for 4:2:2 to not be as good..

No, no, no. 4:2:2 IS beter than 4:2:0. It's just that from what I've been able to digest on the subject, it seems that only 4:4:4 would be truly motion artifact free.

Motion artifacts from color? I don't really understand how that is possible.

Okay, glad we are on the same page. 4:4:4 I believe is pixel perfect with essentially no color sampling being processed assuming that the camera is really recording a true 1920x1080 image (for example). Obviously ideal for chroma-keying.

1- There are different 4:2:0 schemes in use, and this makes a difference.

The intuitive approach would be to apply 4:2:0 to the entire frame (both fields at once). But if you are shooting interlaced and there is motion, this doesn't necessarily work well. Chroma from one field (a particular slice of time) will bleed into the other field (a different slice of time). The bleeding in chroma from different slices of time can appear as a visible artifact.

So to solve that, you have to apply the 4:2:0 chroma subsampling on each individual field. But this raises a new problem in that you see combing artifacts in the chroma. So to solve THAT problem, you could blur the chroma vertically (this does not hurt resolution). However, this brings you back to square one. Now the chroma is bleeding between different slices of time. So to fix that, you need a good de-interlacer that detects motion. If there is motion, then the de-interlacer doesn't blur the chroma vertically.

So essentially... if you are shooting interlaced, that really screws up 4:2:0.

See the Wikipedia article on chroma subsampling and
http://www.hometheaterhifi.com/volum...ug-4-2001.html
for some pictures. The home theatre article is described this problem (interlaced chroma problem) and a different problem, the chroma bug if you get confused like I do.

2- In some rare cases, 4:2:2 color has subtle visual differences when compared to 4:4:4. This will happen if you have VERY saturated colors in the shot (which is rare except for colored lights, certain stage lighting).

3- If it's an interview, I wouldn't worry about it. Better to keep it simple, keep the subject comfortable, and spent more time interviewing them.

4- 4:2:2 doesn't have the same motion artifacts that using interlaced material with 4:2:0 (applied to both fields as a whole) does.

You could argue that 4:2:2 has aliasing artifacts that are made worse by motion. Subtle movement will cause the aliases to move and cause distracting crawling motion. But you won't really see this on 4:2:2 done properly, with reasonable filtering (e.g. linear/triangle resampling both on encoding and decoding).

Colour space ratios represent the ratios of colour resolution to luminance resolution and don't have anything to do with motion artifacts - the latter are more a product of intra v inter frame compression. But it's worth thinking about what the colour space numbers actually mean.

4:4:4 means that colour resolution is as good as luminance, period. 4:2:0 means that colour resolution is only half that of luminance - both horizontally and vertically. Is that a problem? Well, not as good for keying etc, but it matches the behaviour of the eye, which is also more sensitive to luminance detail than colour, so it may be seen as a pragmatic way of saving data.

So what about 4:2:2? In this case colour resolution is half that of luminance horizontally (as for 4:2:0), but the same vertically - the system is assymetric.

Which begs the question why. My understanding has always been that 4:2:2 dates from the days when video was inherently interlace. A 4:2:0 approach would lead to the interlace causing samples on lines 1 (field 1) and 2 (field 2), but none on 3&4 etc - hence 4:2:2 makes much sense with interlace TV systems. It halves the number of chrominance samples (relative to 4:4:4), but you don't get the effect of two adjacent lines without samples, and possible related PAL/NTSC coding problems.

But with a PROGRESSIVE TV system the point of 4:2:2 is much less valid. Why have this imbalance between the vertical and horizontal colour resolutions? For progressive TV systems 4:4:4 and 4:2:0 both have validity in their own ways (both have symmetry), but 4:2:2? It may be argued that more colour samples are always a good thing. Maybe, but it comes at a price - higher data rate, or higher overall compression - so hence, yes, FOR TWO SYSTEMS OF THE SAME BANDWIDTH, 4:2:2 may not be such a good idea as 4:2:0. (A 10bit 4:2:0 system may be preferable to 8bit 4:2:2, for example.)

And these numbers only refer to what the recording system is capable of, NOT necessarily what the imaging system can deliver - the colour resolution of a Bayer sensor is less than it's luminance resolution, for example. (And hence more closely resembles the behaviour of the human eye.)

It's not worth getting too hung up on colour space. It's only one parameter out of many that define relative performance of cameras.

[EDIT: Glenn goes into more detail above, but I think the main message is the same. The 4:2:2 v 4:2:0 debate is very different for interlaced and progressive systems.]

For production, you would ideally record 4:4:4. 4:2:2 or lower can be visually different. (And obviously for chroma keying, 4:2:2 is not completely ideal; though in practice 4:2:2 is not the biggest obstacle to good keys, other factors are.)

The wikipedia article partly covers that. The other big reason why 4:2:2 is not visually lossless is because it pushes colors outside the R'G'B' gamut. For example, it can end up creating colors with negative green or blue values. Monitors can't emit negative light- that's a problem.

2- For emission / distribution, chroma subsampling makes sense for DCT-based codecs (MPEG2, MPEG4, etc.). For progressive systems, 4:2:0 of course makes sense.

But a lot of broadcast is being done in 1080i60. So 4:2:0 interlaced with 1080i60 broadcast has poorer chroma than a theoretical progressive system.

David, I have to disagree (with my newly found and probably incomplete knowledge, LOL). It's pretty clear to me from Glen's posts, and other materials, that the lower sampling of chroma versus luma can cause problems with motion.

Firstly, the differences between interlace and progressive in this context must be stressed, and I don't think with a progressive system there is any connection between motion artifacting and colour space.

With interlace, then yes, there may be some motion effect, but I'd hazard that it is a secondary effect compared to the reduced resolution, and also that most motion artifacts are more likely to be associated with the intra/inter frame issue. But maybe my earlier sentence would be better reworded something like "Colour space ratios represent the ratios of colour resolution to luminance resolution and have less to do with motion artifacts - the latter are more a product of intra v inter frame compression. ..."?

A great quote from the link Glenn provides is "The reason this (4:2:0 on a DVD) is done is for efficient use of space on the DVD. Sure, 4:4:4 would be nice, but the DVD might have to be two feet in diameter. So, it's 4:2:0."

It would be great if we could all, always, record 10bit 4:4:4 uncompressed at the highest resolution in a camcorder sized package. That's not likely to be the case. Compromises are inevitable, and colour space issues are but one of many. If 4:2:2 means higher overall compression than 4:2:0, it may not be a better compromise.

But in the first post the question posed is:

To answer that question directly, then if you are referring to a 720p HDV camera (ie JVC) then the speed of motion of the subjects will be irrelevant. If it's a 1080i camera, it may have some relevance, but I'd suggest that if you can get at the HD-SDI output, less compression will have significantly more effect than colour space issues on motion artifacting.

That makes sense, David. However, everything using 4:2:2 that comes to my mind, is running a higher data rate to accommodate the extra chroma resolution, so that additional compression doesn't become necessary.

Examples:

DVCPRO25: 4:1:1
DVCPRO50: 4:2:2

XDCAMHD 35VBR: 4:2:0
XDCAMHD 50VBR: 4:2:2

-gb-

Absolutely true, and I stress everything I said was entirely theoretical, deliberately staying away from actual examples. Hence the earlier comment: "FOR TWO SYSTEMS OF THE SAME BANDWIDTH, 4:2:2 may not be such a good idea as 4:2:0".

DVCPro 50 should obviously be better than DVCPro 25 in quality terms - but that's due to lower overall compression as well as increased colour space. At the expense of double the bandwidth.

I really wanted to get away from statements such as "4:2:2 is better than 4:2:0" given without qualification. Imagine the choice was between 8bit 4:2:2 or 10bit 4:2:0, all else equal. Under those circumstances, the 4:2:0 option may well be the better one - especially for a progressive system!

To hopefully clarify things a little...

There are many different 4:2:0 schemes in use. They mostly differ in whether they operate on each individual fields (let's call this "interlaced sampling 4:2:0") or on both fields at once (let's call this "progressive sampling 4:2:0").

Progressive sampling 4:2:0 reduces chroma resolution (only) 2X in both directions. If interlaced footage is used in this scheme, motion artifacts can result.

Interlaced sampling 4:2:0 reduces chroma resolution 2X horizontally and *4X* vertically. No motion artifacts result if interlaced footage is used in this scheme.
*It gets slightly more complicated in that the chroma has a comb-teeth look to it; a good deinterlacer solves this.

2- Done poorly, chroma subsampling (any of the schemes, e.g. 4:2:2) can suffer from aliasing. Motion can make aliasing look worse. This is a separate issue.

No argument here, I'm shooting XDCAM HD and feel good about the quality. Others have used this system and reported pulling very clean keys.

I think something that sort of gets lost in all of this is the fact that we're talking about HD resolution. That, in and of itself, lends to a nice progressive picture, even at 4:2:0. Many folks that think they need 4:2:2 are basing it on what they've seen for years working with SD resolution. But when you render the same scene with 4X the number of pixels, the chroma resolution isn't all that bad.

-gb-

I think a consideration should be mentioned in regard to resoultion. DVCproHD records as a lower resolution (720p/1080p) regardless of being 4:2:2.
Your chroma rez is only related to your luma rez.

Second point, what about pixel shifting? Does a very low rez sensor eg. 960x540 deliver anything definative in regards to 4:2:2 resolution when it is displayed as 720p/1080p or even lower if it is DVCproHD? It is pixel shifting colour after all.

Third point is that you can not compare bit rate as a definitive for colour space comparison.
More colour space is not always more bitrate. The codec is a very important factor.
Eg.
DVCproHD 720p24: 4:2:2 960x720 40Mb/s
HDV 720p24: 4:2:0 1280x720 19.7 Mb/s (4-5 times more efficient codec)
So who is the clear winner?

Very likely true. And whilst in PAL land there is very little 4:1:1 coding (DVCPro being the obvious exception) I understand that with NTSC DV25 is 4:1:1. The obvious problem being that when going to DVD (4:2:0) the resultant is effectively 4:1:0. With 4:2:2 that doesn't happen, but 4:2:2 is preferred not because of what it is, but because it is NOT 4:1:1, if that makes sense!
Yes, and interesting to speculate whether the overall result would be better if it was 4:2:0 1280x720 - trade off chrominance samples for extra luminance. My understanding is that DVCProHD is what it is due to an initial derivation from DVCPro25, and using 4 chips in parallel. A single chip gives 480i/30, a pair gives 480p/60 (and 4:2:2 cf 4:1:1), and two pairs gives twice as many samples - 2x720x480, or 720x960. Can anyone comfirm or deny that?
Like a Bayer sensor, pixel shifting inherently gives a higher luminance resolution than chrominance - nothing wrong with that, it mimics the eye. But I do feel that if recorded 4:2:2, you may not be getting what you think you are. Although the recording format may be 4:2:2, the *system* is not giving equal lum/chrom vertical resolution.

Trouble is, the more you go in to the subject, the more complex it becomes. I believe that for some applications where bandwidth is limited (such as digital terrestial transmission) chrominance is allocated fewer bits/sample than luminance, and this is a major cause of the contouring that may be seen on areas of nearly uniform saturated colour. What is then needed is lower compression, not a higher colour space. (Which would need even more compression!)

What's needed isn't so much less compression, it's better compression. Given that DCVPro HD is 100Mbits/sec it's possible today to fit stunning quality into that bitrate, modern wavelet codecs can deliver 4:4:4 full raster HD at that bitrate and not raise a sweat.
Not so long ago the problem was the silicon to encode and decode those kinds of data streams were the problem. Today that's no longer the case with off the shelf CPUs.

I agree that compression isn't the concern. The on board processors these cams carry improves every year. For example the "super encoder" on the JVC 200 series delivers twice the frame rate as the older 100 series (720p60 vs. 720p30) and offers better image quality, all in the exact same 19.7Mbs bit rate. On the older DVCproHD codec a jump from 960x720@24p to 960x720@60p results in a jump from 40Mbs to 100Mbs.

Here is a image I have shown before about chroma sub sampling.

Chroma sub sampling does not have to be a bad thing.

4:2:2 is about as good as you could ever hope for when it comes to keying only. 4:2:2 means the chroma pixels are 2x1 pixels in size compared to the luma pixels being 1x1 pixels in sixe. All it means is that the chroma pixels are twice as wide.

true progressive 4:2:0 (XDCAM progressive mode, JVC HDV1, Canon F modes) has chroma blocks that are 2x2 pixels in size. this creates very clean results because while it has larger chroma pixels then 4:2:2, there is a equal amount of pixe overlap in both the horizontal and vertical directions.

4:1:1 was bad because the chroma blocks were 4x1 pixels in size. This worked great for interlaced video but it creates holes that are 4 pixels wide which show up very well in keyed footage.

4:2:0 interlaced is better then 4:1:1 but it is a little complex to deal with because it has to alternate with the fields. When you key 4:2:0 interlaced material it is going to have lines that seem to be out of phase and it will look funky. As soon as you playback this footage however it will be corrected because a display will show only one field at a time either because the display is analog or via bobbing or some other method. So if interlaced footage is meant to stay as interlaced then 4:2:0 is not bad at all. The area that is a problem with interlaced however is when you start to have progressive inside of interlaced.


Think of keying a 4:2:0 progressive 1080 HDV image as keying a 720x540 4:4:4 image and then doubling it's size. but then being able to pull in extra luma detail.

Good keying also comes down to the software you use. The best way to key footage is to interpolate the chroma samples back in. My favorite way to do this is to blur or resample only the chroma channels. This gives me a soft 4:4:4 chroma channel to key with. Yes it isn't any more detailed but it doesn't have the jagged edges either. This method even works well with 4:1:1 material but it works very well with 4:2:0 progressive material.


http://www.dvinfo.net/conf/attachmen...0&d=1159424266

Is there really a perceptual difference between 4:4:4 color and 4:2:2/4:2:0?

In some (rare) cases yes. It can happen if:
A- You have transitions between very saturated colors, like pure green text on a purple background (or yellow/blue or red/cyan). You get dark bands where the chroma transitions are. How dark the bands are is proportional to saturation; so in real world scenes (where colors aren't as saturated) this isn't really a problem.
B- Saturated text on a black or white background.

see the pictures at codecs.onerivermedia.com
In the top right part of that test pattern, none of the 4:2:2 codecs can get the red lines on the black background correct.

So really the only drawbacks to 4:2:2 or 4:2:0 color is that the compressed colors can cause artifacting when pushed to the extreme saturation levels and that obviously keying an uncompressed 4:4:4 piece of footage is gonna be cleaner because there is more information. But basically for real world colors/contrasts there is no perceptual difference between a 4:4:4 and a 4:2:2/4:2:0?

If you are encoding something with very high saturation, then 4:2:2 can exhibit artifacts.

In the big majority of situations, yep. In very saturated lighting, 4:2:2 can fall apart slightly (e.g. red LEDs, concert lighting, neon lighting, etc.).

Whew, that makes me feel better. I did so much hand wringing over the Red's 4:4:4 and HVX's 4:2:2 vs the A1's 4:2:0. In the end I bought the CANON and am very pleased.

Yes good software can help a lot. The Cineform products AspectHD/ProspectHD ect.. do an automatic intelligent chroma upsample on conversion, which is a highly under rated feature of the software IMO. This helps a lot when you use your keying software.
I am dropping a link below because it is really hard to find it on their site.

http://www.cineform.com/technology/H...ysis051011.htm

IMO Cineform's chroma upsampling looks non-standard as it disobeys interstitial chroma siting- the center of the chroma is shifted/moved upwards in the Cineform scheme. A better scheme would blur without shifting the chroma center... i.e. chroma blur 1.0 vertically in Vegas (*not sure if Vegas' filter uses the right luma co-efficients for 709/HD; I doubt it).

(This assumes that the MPEG-2 standard calls for co-sited chroma horizontally, and interstitial chroma vertically. I've never read the MPEG-2 spec, but this is what it calls for according to Poynton's book.. which is likely correct [see poynton.com].)

Well you tell me by looking at the image I posted. Can you see any difference between the 4:2:2 and the 4:4:4? I'm not talking about the blown up window but the real image itself.

You also need to remember there is a huge difference between computer graphics and what a video camera can do. A video camera will never ever be as sharp and detailed as a computer image. Most cameras including every single HDV camera in the world only process 4:2:2 in the DSP. This means no matter what you do you will never ever get 4:4:4 out of your camera. The only cameras that process 4:4:4 in the DSP are the SONY F950 Cinealta cameras. I'm not even sure if the F900 will. The only way above that to get 4:4:4 is to use a camera such as Red but then a single chip bayer sensor has it's own color interpolation issues which means you will never have true perfect 4:4:4 source for less then $100,000.00 on a camera.

While the samples on the Onerivermedia website are a good example of what the different codecs do, at the same time they are not really fair of realworld footage. Again no camera in a decent price range will ever resolve perfect single pixel lines.

Current HDV cameras have so many other limitations such as chip size and optics that the difference between 4:4:4 and 4:2:2 is really insane to think about. Even if you had a super clean source which is something you can only get with computer generated images 4:2:2 will be so hard to notice that you would have to be a super geek to even worry about it.

That sample shot is taken from an interlaced HDV source. Sony FX1 I believe. I have only ever used it on progressive HDV footage and it works very well, especially considering it does it automatically.

I suspect it may slightly degrade quality if you were to author a DVD off it.

For chroma key, you're slightly better off doing a vertical blur that does not move the chroma center.

But those are likely really small technical details. For speed/workflow, it may be nice if you're doing lots of chroma key.

Probably the most likely case where something like that would happen is if you're pan&scanning still photos, or down-converting CG generated imagery.

You can also have problem with real-world imagery if panning across any color edges.

But then again, extremely subtle artifacts (you kind of have to be looking for them). And there are much bigger issues to worry about than tiny details like that.

I tried the following:

1) Keying HDV (4:2:0) and resize output to SD
2) Resize HDV to SD DVCPro50 (4:2:2) and pull keys from SD

Keying 4:2:0 HDV results in cleaner SD images.