Over in the http://www.dvinfo.net/forum/panasonic-avccam-camcorders/491883-panasonic-tm900-users-thread-30.html thread of the "Panasonic AVCCAM Camcorders" forum, Roger Shealy posted a very nice video shot with his Panny TM900. However, the Vimeo embed suffered from blocking artifacts. I asked the question, "Have you tried HandBrake for h.264 encoding?" So, rather than further hijacking that thread, I suggested we continue the discussion here.

The basis for this discussion is the following tutorial (which I've posted here before).

Vegas-to-Vimeo Tutorial - A Better Method on Vimeo

I was involved in this project, but he primary credit goes to "musicvid" of the Sony Creative Software Forums.

So, Roger's question is "what advantages would you expect to see following this method? [i.e. HandBrake for rendering Progressive source]"

Cutting to the chase, I did some testing, comparing Sony AVC, MainConcept & HandBrake renders at low bitrates (where differences in quality are most apparent). The results are here: HD Video for the Web - Guide for Vegas Users (http://www.jazzythedog.com/testing/dnxhd/hd-guide.aspx#LBR)

Now, to address Roger's question directly - Why HandBrake? There are three reasons HandBrake renders are superior to renders directly from Sony Vegas:

1) HandBrake uses the superior x264 codec see: Sixth MPEG-4 AVC/H.264 Video Codecs Comparison - Short Version (http://compression.ru/video/codec_comparison/h264_2010/)
2) HandBrake resizes the image using the Lanczos resizing algorithm (Sony Vegas uses Bicubic or Bilinear).
3) HandBrake deinterlaces using yadif (Sony Vegas uses Interpolate or Blend). Although there is a yadif plugin available: yohng.com Yadif Deinterlace for Sony Vegas (http://www.yohng.com/software/yadifvegas.html)

Obviously, Roger's source is progressive, so point 3) doesn't apply, but 1) & 2) do.

So, that was rather long winded and rambling, but I encourage discussion and other's experiences.

...Jerry

Do you have some good examples of your best attempts of same footage with different codecs posted to Vimeo?

Here's a couple examples (I don't have a Sony AVC example).

First the MainConcept:

Sony Vegas->MainConcept - Version 4 on Vimeo

Next the HandBrake:

Sony Vegas->DnXHD->Handbrake - Version 4 on Vimeo

And as I mentioned earlier, at higher bitrates the difference in encoders is not as visually apparent. I'm particularly interested in low bitrate encoding because I post a lot of videos as locally hosted using JW Player - and I want to get he best quality with the lowest bitrate.

In any case, this entire subject has been discussed in excruciating detail here: Sony Creative Software - Forums - Vegas Pro - Video Messages (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=751170) If you're not a member there, suggest you sign up so you can view the embedded images.

...Jerry

One more thing, HandBrake is free and open source, and can be found here: HandBrake (http://handbrake.fr/)

...Jerry

...Why HandBrake? There are three reasons HandBrake renders are superior to renders directly from Sony Vegas:

1) HandBrake uses the superior x264 codec see: Sixth MPEG-4 AVC/H.264 Video Codecs Comparison - Short Version (http://compression.ru/video/codec_comparison/h264_2010/)
2) HandBrake resizes the image using the Lanczos resizing algorithm (Sony Vegas uses Bicubic or Bilinear).
3) HandBrake deinterlaces using yadif (Sony Vegas uses Interpolate or Blend). Although there is a yadif plugin available: yohng.com Yadif Deinterlace for Sony Vegas (http://www.yohng.com/software/yadifvegas.html)...
As a close follower of various other threads, I can say that I've put these methods to the test, with comparisons to various other h.264 encoders/codecs, including MS (expression), Sorenson and Apple, as well as Sony and MainConcept. x264 is superior in quality for a given bitrate, as well as very fast on the encodes. The resize and deinterlace methods are also demonstrably superior, as Jerry says.

The takeaway for me is that Handbrake provides a superior MP4 with minimal pain. The batching ain't bad, either. I still use other encoding products that are more versatile than HB, but that one workflow is very broadly applicable for current methods of online distribution.

I've incorporated use of Handbrake into the college class I teach on compression and streaming, and use it myself for iOS, Silverlight, and Flash distributions.

HB is not the only product to use the x264 encoder, but it's really very good, not a bad GUI, and free!

Jerry may not have done all the benchmarking and examination of alternative vegas workflows, but he's done a lot of them and has been very involved in getting the word out - thanks!

I'm a big fan of the "Better" method; but I use somewhat different settings. Partly this is because I start with progressive video - all of my recent clips come from a pair of Panasonic cams that shoot progressive. If I must include older or imported interlaced video, I use the Smart Deinterlace plugin from within Vegas 32-bit versions (usually Vegas 9e) so that all my clips on the project TL will be progressive. By sending progressive clips into Handbrake, there is no need for the Decomb filter (nor any other filters) in the Video Filters tab.

I also use Sony .MXF instead of DNxHD. The latter is technically superior, and it does support stereo - the .MXF files use one or the other of the stereo channels, but not both. However, the file size is much smaller with .MXF, and the render is a bit faster, as I recall.

The lack of stereo? Not a problem for me, as I want to post low-bitrate video and I don't want to waste extra bits on audio for minimal benefit. For that reason, I set my Audio tab to 32 kbs mono when I'm going to post to my own website:

http://www.torealize.net/files/HB-AudioBitrateForMyWebsite.jpg

I also have had problems with high-motion areas becoming torn with the settings shown in the Advanced tab of the tutorial video. Instead, I use these settings to avoid that problem - both for sites such as YT, and for my own website:

http://www.torealize.net/files/handbrake-AdvancedTab.jpg

For greater control of bitrate, I use the Average Bitrate option instead of the slider:

http://www.torealize.net/files/Handbrake-videoTab.jpg

For ordinary 720p videos, I set it to 500 kbps for my own website playback (2500 if uploading to YT or Vimeo). You can see a sample of the picture quality at the 500 kbps bitrate here (be sure to click the full-screen button):

Encounter with (http://www.torealize.net/1short.html)

You can also see the same video on Vimeo (worse image quality, even though higher bitrate - as I recall):

(1a) The Encounter on Vimeo

I finally got the JWplayer to give me video that my daughter's iPhone can play by means of HTML5. I also got the Levels to work, so now start with the .MXF file from Vegas which I then encode to 3 bitrate .MP4s in Handbrake - typically 500 kbps for 1280x720 video, 150 kbps for 640x360 pixels, and 80 kbps for 360x208 pixels (someone on this forum once said that some internet cafe WiFi connections are that slow).

And, instead of coding directly into my (free) nVu HTML editor, I use Notepad (the trouble with nVu is that it sometimes allows the coding to continue in long lines that run beyond the edge of the screen - requiring scrolling). Here's a sample from Notepad (from which I then select all, copy, then paste into nVu):

-------------------------------

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html><head>


<title>"Kriya Yoga and physics"</title><meta name="title" content="This is the explanation of Kriya Yoga in terms of quantum physics.">

<meta name="description" content="I finally was able to accept the claims of Yoganandaji by studying quantum physics.">

<link rel="image_src" href="http://www.torealize.net/images/Yogananda.jpg%20">

<script src="http://ajax.microsoft.com/ajax/jquery/jquery-1.4.4.min.js" type="text/javascript"></script>
<script src="js/jquery.simplemodal-1.4.1.js" type="text/javascript"></script>
<script type="text/javascript" src="/jwplayer/jwplayer.js"></script></head>
<body topmargin="0" leftmargin="0" style="margin: 0pt; color: rgb(0, 0, 102); background-color: rgb(255, 255, 255);" alink="#006600" link="#0000cc" marginheight="0" marginwidth="0" vlink="#ff0000">
<div id="container">Loading the player ... <script type="text/javascript"> jwplayer("container").setup({ autostart: true, flashplayer: "/jwplayer/player.swf", volume: 80, height: 270,
width: 480, skin: "/skins/modieus.zip", image: "/images/Yogananda.jpg", levels:
[ { bitrate: 80, file: "http://www.torealize.net/video/2physics360.mp4",
width: 360 }, { bitrate: 200, file: "http://www.torealize.net/video/2physics640.mp4",
width: 640 }, { bitrate: 800, file: "http://www.torealize.net/video/2physics.mp4",
width: 640 }], provider: "http", "http.startparam":"starttime" }); </script> </div>

</body></html>

-------------------------

Notice that I put each bitrate paired right next to the .MP4 filename - per the .PDF that comes with the JWplayer download. I do it this way just to keep it easy to keep track of all the parameters in one place for each bitrate.

In order to use these low bitrates, again I emphasize how important it is to clean up all noise in Vegas with Smart Smoother or some similar noise-reduction plugin before rendering. By doing that and rendering some really low bitrate .MP4s along with the higher bitrate ones, your video should play on almost anything (Flash-enabled browsers or HTML5), play almost everywhere even if the web connection is slow without buffering except for the first few moments, and look really sharp on fast web connections when watched full-screen.

incidentally, I could not get video on the iPhone until I added the skin - I know not why. The skin that I downloaded, modieus, is free for non-commercial work. Just put the .zip file into a folder named "skins" in your root directory, and it will automatically load.

Here's what the above coding looks like:

"Kriya Yoga and physics" (http://www.torealize.net/2physics.html)

Good stuff, Larry - you might also check out the JW Player "modes" parameter.

JW Embedder Modes | LongTail Video | Home of the JW Player (http://www.longtailvideo.com/support/jw-player/jw-player-for-flash-v5/18508/jw-embedder-modes)

btw, one of the real complications in this lack of standardization is that one must have a cadre of devices to test on - ipads, Androids, tablets, etc. etc.

...Jerry

Jerry: thanks for the "modes" tip.

Yes, I wish I had a tablet so I could test on the latest Android. Unfortunately, the phone version of Android only goes up to 2.3.7 - which I just installed on my cheap GT 540 (the HTML5 wouldn't work on it).

However, Android is now near the end of updated a unified version 4 that works on both the phones and the tablets - no longer will Android have two versions. Hopefully, some brave soul will modify it so that it will work on my phone - the way that a fellow in Bulgaria released the ROM that I just installed. As I recall, it was 18 steps and some who have tried this (and similar unofficial) upgrades have bricked their phones. But all went well for me, even though I was disappointed that it still would not play my HTML5.

Fortunately, Android claims that V4 will finally get all HTML5 to work properly. I hope I can get it soon . . .

Sadly Vimeo files will not play on an Android Tablet as of today. Will Vimeo play on an Ipad 2?

If I start with 1080p footage shot at 24 (well 23.976) fps from a Canon DSLR (7D and 60D), what settings should I use? The tutorial said 1080i 145 8-bit, but the only options for 1080p 23.976 that are close are 175 8-bit or 115 8-bit. Which one should I use?

Also why are settings for your own website playback different than for YouTube or Vimeo?

Thanks!

In the DNxHD configuration there is a setting "1080p/23.976 DNxHD 175-8bit" - use that. Then create a custom Quicktime Template that looks like the attached image (i.e. 24.976 progressive).

Of course, when you get to the HandBrake steps, you must, once again use the 23.976 framerate, and, of course, there's now no reason to deinterlace.

"Also why are settings for your own website playback different than for YouTube or Vimeo?"

I assume you're referring to the fact that the locally hosted web videos are at a much lower bitrate than recommended in the tutorial?

When you upload to YouTube or Vimeo, you should render at a higher bitrate and let the YouTube/Vimeo processors reduce the bitrate (and quality) to something that will pseudo-stream properly. In other words, start with a very high quality video and let YouTube/Vimeo degrade it.

When hosting a video on your own website, you must manually set the bitrate to balance of video quality and the ability to progressively download. Thru trial-and-error I've found that about 1200mbs works well for this video. It could be different for other footage - depends on many things - motion in video, transitions, quality of audio, etc.

...Jerry

I tried the "Better" method, which while better than something like .wmv straight from Vegas, still wasn't significantly better. Compared to the Sony AVC method, which I found actually preserved color vibrancy instead of making everything dull and muted/flat, the Handbrake version just wasn't as pleasing - though I did 1080p 23.976fps 175 8-bit instead of the 1080i 60i 145 8-bit recommended setting because I had footage from a Canon DSLR. In fact, the Handbrake's colors seemed pretty similar to a straight .wmv render, both of which were significantly flatter than what I saw after color grading inside Vegas's video preview window.

Sony AVC render method here (though I changed it to 16,000,000 bitrate instead of the recommended 4-6million). Ultimate Sony Vegas Render Settings - Kodak HD forum on Vimeo (http://vimeo.com/groups/kodakhd/forumthread:4319)

Also, it's unclear to me when I should apply the Levels filter for Computer RGB to Studio RGB and when I should not. And how does this differ from the Broadcast Colors filter? Thanks!

Zhong,

If you go back to the first post in this thread you will see the three advantages of HandBrake:

1) HandBrake uses the superior x264 codec see: Sixth MPEG-4 AVC/H.264 Video Codecs Comparison - Short Version
2) HandBrake resizes the image using the Lanczos resizing algorithm (Sony Vegas uses Bicubic or Bilinear).
3) HandBrake deinterlaces using yadif (Sony Vegas uses Interpolate or Blend). Although there is a yadif plugin available: yohng.com Yadif Deinterlace for Sony Vegas

At a high bitrate (16Mbps in your case), item 1) is much less apparent, if at all. Since you start with progressive footage, item 3) does not apply. So, the only advantage you get is with 2). And if you don't resize your frame size, you wouldn't even see that. So... I'm not surprised that you are not seeing a huge improvment.

Re: when to apply the cRGB to sRGB Levels FX, the short answer is use it when the ultimate render is h.264/mp4 (as it is YUV color space). Do not use it for WMV as that is RGB color space. Note: this applies to Sony Vegas only. I understand other editors handle this automatically (although that's not first hand knowledge).

...Jerry

Thanks Jerry! Makes sense.

However, why was the AVC version (at 16,000,000 bitrate...best quality of this trio) so much better with the colors compared to both the WMV (worst quality) and the Handbrake version (second worst)? In fact, the wmv and Handbrake versions, as far as colors go, seemed to be almost the same to me - both significantly worse than the AVC.

Also, now I'm trying to turn my 1080p 23.976 project into a DVD (standard def) file for DVD Architect, which requires rendering as .mpeg2 and .ac3 or .wav for audio.

But the .mpeg2's colors now seem way off again (similar to the wmv or Handbrake colors). How can I make it more similar to the AVC vibrant colors? I mean, the .mpeg2's colors seemed okay on playback from my local hard drive straight after rendering from Vegas, but when I stick it into DVD Architect and press preview, the colors are so much duller again.

Some examples:
dull wmv color: Rachel + Dan: Wedding Highlights (Ring Shot Version) WMV Version - YouTube

vibrant AVC color: Rachel + Dan: Wedding Highlights Film - YouTube

It seems that my Sony Vegas Preview window splits the difference between wmv and avc. Here's a screen shot of it (top left is wmv, bottom right is avc, upper right is Sony Vegas preview):
http://imageshack.us/f/42/colordifferences.jpg/


And here's a pic of the color differences between AVC uploaded to YouTube vs. .mpeg2 rendered at CBR 9,800,000 and put in the DVD Architect Preview Window:
http://imageshack.us/f/705/dvdacolordiff.jpg/

I've used both Sony avc and Mainconcept avc for the web, and both come out perfectly fine for me. My conversions of projects from 720 60p and 1080 24p come out beautifully for DVD also.

I currently use Sony avc and set the bit rate depending on the amount of movement in the video. I follow the guidelines for Vimeo, it's really simple.

The biggest factor is the quality of my original footage. It it's good, then the web version comes out fine.

I was struggling with this issue and asked an award winning videographer what settings they used and they said Sony AVC with a bit rate of around 12 mbps, I think, which seemed very low, but their online videos look stunning.

Improper monitor settings will create a discrepancy I think also, and this has been discussed to death also. When I calibrated my monitor it helped with this issue somewhat.

I render to Sony AVC, then play back the file before uploading, and after uploading I watch the video online, and they look close to identical to me.

Zhong,

Aha, your videos are a great example of exactly why you want to use the cRGB->sRGB Levels FX to h.264 renders while leaving the WMV untouched. Actually, you want to map the h.264 to the 16-235 color space - the cRGB->sRGB Levels FX is a "hammer" to do this job. A better way would be to use your Video Scopes to do this.

Here's the reason: online players such as YouTube, Vimeo, JW Player, etc. will take the 16-235 range and expand it to 0-255 (you'll even see this in WMP or VLC). If your footage exceeds the 16-235 range, the player still attempts to expand them to 0-255 and the whites are blown out & blacks are crushed. Unfortunately, the Vegas Preview does not exhibit this phenomena. A temporary fix would be to apply the cRGB->sRGB FX to the "Video Output FX" on the Preview Panel. However, make sure you disable it before you render or the render will be expanded once again.

Take a look at the attached image. I tried to capture the same frame in both your h.264 & WMV versions. Yes, the h.264 looks "more vibrant", but the whites are blown out. Note the detail in the veil. I think you will agree that detail is lost in the h.264 render.

Now, maybe the "vibrant" look is the one you wish achieve and don't care about the (minor) loss of detail. In that case, go with it. On the other hand, you could map the levels and then use the Color Curves FX to increase the contrast and get the best of both worlds.

I know this is an extremely confusing subject, and I'm trying to do my best to explain it. Hope I'm helping.

...Jerry

I'm confused what you mean by mapping the levels and using Color Curves FX to get the best of both worlds?

I use Color Curves on all my clips, applying an S-curve to my Technicolor Cinestyle shots (which are all of them). Then I play with each clip individually with color corrector, brightness/contrast, etc.

What does mapping the levels mean? And using video scopes instead of the cRGB to sRGB...how?

Are you saying the Sony Vegas preview window displays 0-255? And only if I apply the cRGB to sRGB filter to the preview monitor will it show 16-235?

I'm really confused...when I'm color correcting/grading, should I be viewing my preview monitor at 0-255 or 16-235? I want what I see in Vegas to look identical as much as possible to what I see when I upload to Youtube and identical to what I see when I burn a DVD (mpeg2).

It seems the mpeg2 renders, even at high bitrates, lose the color vibrancy again...just like the WMV. How can I fix this?

My Sony AVC renders -- which I did not render out with the cRGB to sRGB filter on -- look much more similar to my Vegas preview window (without the cRGB to sRGB filter on) than the WMV renders did. However, both AVC (rendered w no levels filter) and WMV (also rendered with no levels filter) look different than what I see in my Vegas preview (w/ cRGB to sRGB filter OFF). I posted a screen shot showing all three in one picture in my earlier post I think.

The current AVC render you see has the amount of vibrancy I liked and that my client preferred. Yes, you're right that it lost detail compared to the WMV, but the WMV looked so flat and dull - people didn't like that. It looked vibrant in my Vegas preview window too (w/ no levels filter), much different than the flat WMV render.

Thanks for all your detailed explanations, Jerry! Learning a lot from you.

Zhong,

First, the phrase "best of both worlds" was probably the wrong phase to use. I should have said, "you can use the Color Curves FX or other techniques to increase the contrast of while retaining the 16-235 color space", sorry.

Second, your questions are good, and yes, this is a rather confusing subject. I think the best way to explain this subject might be to produce a short tutorial/screencast to explain it. After all, if a picture is worth a thousand words, a video should be worth at least 10,000 - correct? This might take me a week or so to get things together as I've got some other projects going on.

...Jerry

Hi;

Jerry, I too would be very interested in your tutorial. I'm shooting with an AVCHD camera that I presume produces YUV420 (4:2:0 ?) video. I edit with Sony Vegas Pro. I've only recently been getting into color correction, using the scopes, etc. I'm pretty confused about the 16...235 thing, monitor gamut, the sRGB 16-235 option on the waveform scope, etc.

I'm not just sitting here waiting for you to reveal all knowledge ... I've spent hours and hours reading on the web, but there is so much information, spread out all over the place. Much of it just raises new questions ... such as this Wikipedia article:

YUV - Wikipedia, the free encyclopedia (http://en.wikipedia.org/wiki/YUV)

In particular, one thing that is bugging me ... is the AVCHD 4:2:0 video produced by my Sony NX70 limited to the 16-235 gamut? Or is that even a valid question? Like the OP, I don't know if I ought to leave my monitors and nVidia cards at 0-255 or use the 16-235 setting ...

Very confused!

Steve

Steven,

The more I try to explain this to people the more I confuse them - that's why a video tutorial with examples may be the best way to explain it. Please be patient with me, it will undoubtably take some time.

I have two AVCHD cameras - a Panasonic TM700 & a Canon HG21. Both shoot in the 16-255 range. So when I have a project that is intended for h.264 render, I routinely use the Sony Levels FX to set the "Output End" to 0.905 which reduces the Luminance to the 16-235 range (although this may vary if the luminance does not reach 255). I use the "Histogram Scope". If you wish to test your NX70 to see what it shoot in, here's a procedure you can use: Survey: What min/max levels does your cam shoot? (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=766876)

Also, here's a recent thread in the Sony Creative Software Forum that may shed additional light on this subject: Color space clarification needed! (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?MessageID=794330&Replies=19)

...Jerry

btw: I'm user "amendegw" over in the SCS forum.

Steve,

... a Panasonic TM700 & a Canon HG21. Both shoot in the 16-255 range. ..

Jerry, thanks for the nice reply and the links. I will study them over the next few days. Also looking forward to the tutorial. Having done only one video tutorial (on Vegas Time-Lapse using Huffy and VeeDub, available on my stevecrye YouTube channel) , I know they are a lot of work, so it will be greatly appreciated!

I just want to make sure that when you wrote "16-255", that was not a typo. Did you mean 16-235? 'Cause if you mean 16-255 then I am *really* confused now! ;-)

Thanks again,

Steve

Hi All;

From one of the posts on the 'Survey" link that Jerry provided a few posts above, this had some great info that helped clarify a lot of things. BTW by the weekend I will do the Survey test for my cams, post both here and there.

---------
Subject: RE: Survey: What min/max levels does your cam shoot?
Reply by: GlennChan
Date: 6/27/2011 8:25:04 PM

There are different issues going on.

First issue: (This may be esoteric and not really matter.)

There are three colorspaces you need to know about.

Y'CbCr... as is commonly used and described in Rec. 601 and 709. (We'll assume that 601 and 709 are the same because it doesn't matter for this discussion, even though they aren't.)

studio RGB, which is RGB with black at 16 and white at 235.
computer RGB, which is RGB with black at 0 and white at 255.

2- Y'CbCr has black at Y'=16 and white at Y'=235. Suppose you are capturing from an analog tape and the calibration is a little off or there is edge sharpening going on. The code values above 235 (Y') will record those values. Later on you could potentially bring those values back into legal range. Had the standard been white at Y'=255, then you can easily get clipping and you'd never be able to bring that information back.

3- Vegas converts Y'CbCr to studio RGB for DV and HDV (in the later versions... I think 6 and above). Sometimes/oftentimes there is useful information above 235 Y'. Mapping 235Y' to 255RGB would clip all the information above 235Y'.
Also, almost all video cameras have useful information above 235Y'.

4- However, this does not mean that all information is necessarily preserved. Yes, 255 Y' will get mapped to 255 RGB'. However, Y'CbCr color space still far exceeds computer RGB. Y'CbCr can represent a lot of crazy colors, some of which can't exist because it would require the existence of negative light. In practice, some cameras will record Y'=255 while the chroma is not neutral, so the equivalent RGB value will have one channel that is above 255 RGB.

Clipping can still occur when converting from Y'CbCr to computer RGB. I believe that some cameras actually do record colors that fall outside computer RGB (though I could be wrong here).

--------------------------------

Second issue:

The values in Vegas depends on what codecs you are using and how they are configured.

For example, if you set Vegas to use the Microsoft DV codec (not recommended), DV footage will decode to computer RGB.... 0-255. And you will get clipping.

If you set Vegas to use 32-bit in project properties and use the full range setting, then HDV will decode to computer RGB. You will have illegal values above 255 RGB, though you can bring this information back since 32-bit floating point numbers are being used.

It may be more useful to say what default codec Vegas uses to decode footage from that camera.

------------------------------

Third issue:
Some/most cameras record values outside the legal range, and/or do not follow standards at all.

Almost all video cameras will record information above 235 Y'.

I can't remember if some of the dSLRs will put black level at Y'=0 and white at Y'=255.
----

Steve

"I just want to make sure that when you wrote "16-255", that was not a typo. Did you mean 16-235? 'Cause if you mean 16-255 then I am *really* confused now! ;-)

Yup, both my TM700 & HG21 shoot in the 16-255 color space. If you look in the Survey: Survey: What min/max levels does your cam shoot? (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=766876) post, that's my TM700 in the third post down and my HG21 in the 7th post down. Incidentally, I don't think you'll see the embedded Video Scope images unless you're a member of the forum, so I'll attach the images here.

For those that haven't read the "Survey" link, here's the methodology:

"Black footage method: Cover the lens with the lens cap or whatever, close the iris, set gain to zero, set the shutter as fast as possible.

White footage method: Open the iris as wide as possible, set gain to max, set the shutter as slow as possible, remove any ND filters, point camera at sky."

...Jerry

btw: I'm continuing to work on-and-off on a video presentation, but it's slow and I don't have a whole lot of time to spend on it.

Hi Jerry + Everyone;

The "A New Look" method et al (which I became aware of a month ago and spurred me to try to understand concepts related to the "16-235" alluded to in the supporting online material) now is finally starting to make more sense.

I just completed the white/black levels tests for my NX70. The whites are pretty much full 255, but the blacks are in the 20-ish range. I found it odd that the histogram for the white was just solid bars, but the blacks looked like little mountain peaks. I guess the NX70's factory settings don't fully crush the blacks (good thing Vegas is ready to rescue) I hovered the cursor over the middle of the groups so that the value can be read in yellow on the far right. For the FullWhite, the histogram just showed solid vertical bars, so I had to streeeeetttccchhhh the vectorscope window across two monitors to get the screen shot attached.

Tomorrow I hope to do the same test for the CX550V, which despite the lack of decent manual controls and some other bells and whistles, seems to have better image quality.

So, I can see how it would help the video to look better on YT if one first adjusts tonality so the blacks go to zero and not below, the whites hit 255 and not higher, and then apply the cRGB >> sRGB FX to "squeeze them in anticipation of YT stretching them out.

What still confuses me is the Vegas preview window display, and the little setting on the Vegas vectorscope for sRGB. Should I leave the vectorscope set for full 0-255 while I adjust tonality and color?

Finally, I do a lot of rendering for Blu-Ray. Do I need to worry about the cRGB>>sRGB for Blu-Ray before rendering? (I have never used it in the past and the BD's seem to have good tonality)

Thanks!

Steve

Stephen, this is a useful result. I'll add it to the list on the SCS forum (http://www.sonycreativesoftware.com/forums/ShowMessage.asp?ForumID=4&MessageID=766876). (Incidentally, there is an important firmware update (http://provideocoalition.com/index.php/atepper/story/sony8217s_nx70_camera_to_receive_its_missing_29.97p_framerate_via_free_firm/) on the way for your cam, if you didn't know)

Looking forward to your CX550V results too.

What still confuses me is the Vegas preview window display, and the little setting on the Vegas vectorscope for sRGB. Should I leave the vectorscope set for full 0-255 while I adjust tonality and color?

I set it to "Studio RGB (16 to 235)". That's perfect because then the 0 and 255 lines then become your black/white limits for both web and TV delivery, but you can still see the illegal values.

I have an external monitor (Windows Secondary Display) set with "Use Studio RGB", so I can see what the luminance should look like when delivered to the web or TV, but at the same time I monitor the Vegas preview window which uses computer RGB, so I can see what detail might be getting clipped from the highlights and lowlights.

Perhaps the easiest way to think about this levels issue, is that Adobe Flash Player (in the majority of cases) does the same thing as a TV-delivered scenario (e.g. DVD). In my opinion that's great. The problem is that users of "0-255" cams such as Canon dSLRs, editing in Vegas, don't realise that their shadows and highlights will get clipped unless they adjust levels to prevent that.

A final thought... One or two of my DV clips lead me to believe that some of the "16-235" cams might give you blacks below 16 in some high-contrast shots. So the "black out" test technique may be flawed. I'll do some testing on that.

Nick,

Thanks for registering in on this discussion, 'cuz...

1) You are much more expert in this subject than I, and
2) I'm having a difficult time conveying this concept and another voice hitting it from a different angle can only help.

...Jerry

Stephen Cry said, 'Finally, I do a lot of rendering for Blu-Ray. Do I need to worry about the cRGB>>sRGB for Blu-Ray before rendering? (I have never used it in the past and the BD's seem to have good tonality)"

I don't do a whole lot of DVD/Blu-Ray work, but I believe the the stardard practice is to confine the color space to 16-235 (as Nick confirms in his post).

Here's what I do for projects I'm not sure about. Nick has come up with a clever test - I call it the "Nick Hope Levels Test" - which illustrates this phenomena.

Nick Hope Levels Test on Vimeo

When the left side of the video looks all white & the right side looks all black, then your color space must be confined to the 16-235 range. I put together a Vegas Project with generated media only (attached). If I want to determine how a Blu-Ray project performs, I merely render this out to Blu-Ray, insert the test into my player and see what the whites & blacks look like.

...Jerry

Hi Nick!

I think I remember you from the creative "Bovine" forums if not, then I have a strong false memory of chatting with you, watching some great underwater clips of yours.

Don't get me started about the NX70 firmware upgrade - far too little. In another thread I'm going to post a list of *MUST* have firmware fixes for the NX70 - annoying things like not being able to use the viewfinder while the LCD is open (which forces me to constantly open and close the LCD in bright sunlight to get anything done), and the need to have the ability to adjust the update speed of the auto-focus to prevent it from trying to focus on noise in high-gain situations.

And for Sony to think that just giving us the ability to set the zoom rocker to "slow" is a fix for the horrible problem is a joke. It is supposed to be a *variable* rocker, after all!

All we want is for this $3200.00 cam to work at least as well as the little MC50 (aka CX550V) ! grrr.

Sorry to rant.

Steve

Hi Everyone;

OK, I've been doing some tests regarding the purported "stretching" of levels done by YouTube when YT processes an uploaded video. My results are (not unexpectedly) confusing.

My test:

I created a short clip in Vegas Pro 10e. With the Waveform scope set to Studio RGB 16-235, I used the Sony Levels FX plugin to squeeze the whites down to 100% and lift the blacks up to 0%. I used the Output Start and Output End sliders to bring down the whites and lift the blacks. I then rendered using Sony AVC to an .mp4. I uploaded two versions of the clip, one with the Levels plugin engaged and one without.

Visually, the clip without the levels plugin had the look of crushed blacks and more "blown-out" highlights. My monitor is set to Computer RGB 0-255, and I use Calibrize to calibrate it. In the clip with the levels adjusted, the blacks were less crushed and the highlights less blown out.

After I uploaded the clips to YT, I then downloaded them and brought then into Vegas for comparison. I took a screen grab of the Waveform scope at about the same point in the clip, in the middle where a woman walks in front of the performers. Those four screen grabs are attached to this post.

All four versions of the clip are available on my website for download:

http://www.huecotanks.com/temp1/DanxLevelsAdjustedToStudioRGB.mp4
http://www.huecotanks.com/temp1/DanxLevelsAdjustedToStudioRGBAfterYT.mp4
http://www.huecotanks.com/temp1/DanxNoLevelAdjust.mp4
http://www.huecotanks.com/temp1/DanxNoLevelAdjustAfterYT.mp4

The YouTube videos (unlisted) are here:
DanxNoLevelAdjust.mp4 - YouTube
DanxLevelsAdjustedToStudioRGB.mp4 - YouTube

My Confusion: Although YT seems to be "stretching" the levels, it is subtle, and not nearly what I expected. I expected the YT-processed "Studio RGB" version of the clip to look more like the unadjusted version of the clip. Instead, the YT processed versions look pretty much like the way they looked before YT processed them.

I welcome comments and analysis from any and all gurus ...

Thanks,

Steve

Attachments : top row = after YT , bottom row = before YT, left = no adjust, right = adjusted

Stephen,

I read your post a couple of times and I'm not sure where you're going with this. You do understand that this phenomena is not specific to YouTube, don't you? It's observed when YT's Flash Player plays an h.264/mp4 format video.

In our testing, we observed the same expansion of color space in every player tested - whether played locally or web based e.g. Windows Media Player, VLC, FlowPlayer, JW Player, Silverlight, HTML5 and several others.

To confuse the issue, if you drag the h.264/mp4 back to the Sony Vegas timeline, the colorspace is perceived as compressed again (and to my knowledge that's the only place you will see this).

So, the before and after YouTube is really not pertinent to the subject. Once footage is rendered to h.264/mp4 all players we've tested will expand the color space observed in Sony Vegas. The YouTube re-render does nothing to the color space that hasn't already been done. Re-renders are not additive - once it gets to the h.264/mp4 format.

Does that help? Or am I misunderstanding what you're doing.

...Jerry

Stephen, don't do anything to the blacks! They're perfect as they are. Anything in your original footage that you see below 16 in the histogram or below 0 in the studio-RGB waveform is rubbish such as noise and sharpening-artifacts that it's quite OK to clip. Look on your histogram. The [shoulder/elbow/knee]* in the luminance is perfectly at around 20, falling to 16 before it hits the noise where the slope flattens out.

So just reel in the whites and you will see more detail in the highlights when it plays back from YouTube, particularly in the players' arms/hands where the levels would otherwise be blown out. Actually this is a perfect clip to illustrate the salvaging of highlights. It's not earth-shattering, it's subtle, but it's definitely better.

I have uploaded a V8 veg file here (http://dl.dropbox.com/u/21489814/Vegas-color-curve-clip-blacks-at-16-roll-off-whites-to-235.veg) (also attached to this post) containing a color curve FX (with explanation) that, IMO, works really well on your clip. You can open the veg in V8 or later and save the FX as a preset. I have attached a screen grab of it.

Even if you're going to process your highlights like this in post-production, I would still try to reduce the exposure a bit when you shoot. As they are, your red and blue channels are hard up against 255. But you might then need to deal with the blacks if they dip below 16 too.

* - insert body-joint of choice

Oh, and as Jerry says, YT don't "do anything" to your levels. Assuming you upload a Y'CbCr codec such as AVC to them, they simply encode it to AVC at lower bitrates. i.e. Y'CbCr to Y'CbCr. The levels "expansion", for want of a better term, happens when your player (e.g. Adobe Flash Player browser plugin) converts that Y'CbCr to RGB for your screen. So when you download an mp4 from YT and compare it against the footage you uploaded, you shouldn't see any significant difference in levels, but just the differences where YT have mangled the detail in your footage with their crappy encoder.

To convert a 16-235 Y'CbCr to 0-255 RGB is, arguably, correct behaviour, since that is what happens in the TV world. If anything it's the cameras that are wrong for shooting outside "legal" levels.

This correction we are talking about is nothing new. It's exactly the same as the correction we should be doing if we want to "legalise" levels for DVD etc..

Hi;

Nick and Jerry, thanks for the insight. I'm sorry it took me so long to get back - suuuper busy at work, no time for any editing.

I'm digesting the new information provided. I hope everyone understands that I am not disputing anything, I am confused and seeking guidance.

As soon as I have something cogent to post, I will get back, probably with some more results. The first thing I'm going to do is take the raw clip, render it without any levels adjustment, bring it back into Vegas, and take a snapshot of the waveform scope. If I understand you correctly, I will see the purported levels expansion - or am I still confused?

I'm beginning to realize that I am probably confusing several things, they are all mixed up in my mind:
* changes to levels introduced by rendering to AVC (what exactly is changed?)
* changes to levels introduced by rendering to other formats (such as when rendering for Blu-Ray or DVD - does this happen?)
* changes to levels when a rendered clip is played in the players listed (all but Sony Vegas are guilty? What about the Sony Picture Motion Browser Player?)

Finally, I have been told by Atomos and others (although Sony is tight-lipped) that my NX70's raw real-time HDMI output is "4:2:2 8-bit" . When I plug the NX70 into my LED flatscreen TV, what I see in real time (in terms of levels and color balance) does not look much different than what I see when I subsequently play what I was recording (after transferring the clip) using Sony PMB, although it does seem to have less compression artifacts. I've read that AVCHD compression in the camera results in 4:2:0 files . No matter how many times I read the Wikipedia articles on 4:2:0 vs 4:2:2 vs 4:4:4 I get more confused ... help.

Sorry for all the questions, I really appreciate the help. All this stuff is clear to gurus, but beginners like me struggle to put it all together.

Steve

The first thing I'm going to do is take the raw clip, render it without any levels adjustment, bring it back into Vegas, and take a snapshot of the waveform scope. If I understand you correctly, I will see the purported levels expansion - or am I still confused?

No, in that case you shouldn't see a change in the levels. The Vegas scopes reflect what you see in the preview window.

Whether you see levels "expanded' during decoding and playback depends on your computer's graphics card and its settings, and the particular media player and its settings. It varies from computer to computer. Even on my computer one of my 24" monitors behaves differently to the other. And GOM Player on my computer behaves differently to GOM Player on other computers. A tool like Takecolor can help you measure the behaviour.

What I can say is that an AVC file that displays black at 16 and white at 235 on Sony Vegas' scopes will end up displaying black at 0 and white at 255 in Adobe Flash Player (YouTube, Vimeo and JW Player) in the majority of cases. That behaviour is the same as it should be on Blu-ray and DVD playback.

I have two follow up questions to this excellent discussion.

I wish to clarify what are the necessary steps to get the most accurate playback from video i post on the net.

The reason I ask this is there are 7 places on a typical Vegas editing station where color can be adjusted.

1) on the monitor itself (theater mode, srgb mode, standard mode, user mode...)

2) third party color correction software (i use the Huey pro)

3) on the graphics card drivers (color profiles, sliders for gamma, saturation etc)

4) within video player internal settings (Vlc wmplayer qtplayer etc...)

5) within Vegas itself by globally turning on or off Options > Preferences >Preview Device > Adjust levels from Studio RGB to Computer RGB

6) applying color correction FX filters on directly Vegas media bin files, or on the time line;

7) applying presets discussed in this thread on the project output preview player.


Seems to me there is much opportunity to over correct and get a false final render for posting on the web.


The current steps I am using is

1) monitor is set to user mode and calibrated with the HeueyPro;

2) graphic driver is set to defaults

3) all video players at default settings.

4) the global switch "Adjust levels from Studio RGB to Computer RGB" is checked on

5) I apply color correction FX on the clips in the media bin to color match.

6) apply the setting recommended in this post on the output player preview window.

7) render out to the web.


As a bonus question, newer cameras and camcorders can be set to record in x.v.color space which is wider than srgb. Most computer monitors at best only can display srgb. How should such footage be handled on the vegas time line?

Thanks

Ray,

I'm pretty sure you'll be just fine with that procedure, however, I do have some comments.

I don't use an external monitor, but I'm pretty sure that "4) the global switch "Adjust levels from Studio RGB to Computer RGB" is checked on" is correct and only affects the way you observe the preview - not the render itself.

I've never applied color correction as a Media FX, but I don't see way that procedure shouldn't work.

Normally, I apply the Sony Levels FX as the final Event FX - making sure the footage conforms to the 16-235 color space as seen with the Sony Video Scopes. Which, of course, means that it would not be applied as the "Video Output FX" on the preview window.

As far as the bonus question is concerned, I think the key is to limit the color space to the 16-235 range and you should be fine.

...Jerry

Thanks Jerry

Recently I shot and posted a music video for my brother George, which you can view here.

George-Painted Corners Video.mp4 - YouTube (http://youtu.be/0yLUgePvuKA)

I followed all of the steps above & used the handbrake better method of rendering the video to the web.

However, upon playback from youtube or on facebook, the video is darker and the "darks" darker than how it was shot and how it was edited. I even applied some luminance gain on the output, to where the light reflections off of the cymbals and dobro in the background just started to clip.

However, playing back on youtube the experience is darker than what I edited in Vegas, even played back on the monitor it was edited on.. So, I am not sure where the problem is.

Thanks

Ray, it looks fine on my 24" Dell.

Ray,

Triplets!!! I think that looks very nice as well. What camera did you use?

...Jerry

Edit: Thinking a little about your "dilemma" - did you try adding the cRGB->sRGB Levels FX template (rather than adjusting the sliders to conform to 16-235)? In other words, if you applied the sRGB->cRGB correction to your external monitor, in order to reverse the process try applying the cRGB->sRGB to your footage in Vegas. (if I understand what you're doing).

Jerry

I used my HDR-AX2000. I shot it in 1920x1080/24p FH mode and down sized it via handbrake. I added some luminance and saturation via the HSL FX on the output, then applied the Sony level FX with the Computer RGB to Studio RGB preset last. Also under the project settings I selected the "32bit floating point video levels" for the pixel format.


Reviewing the RGB parade vector scope the dark pixels start at about value 38 and there are only a few pixels in 235-240. range.

Thanks everyone for the feedback.

P.S. While editing I realized 24p was too slow a frame rate - Musicians hands & feet move surprisingly fast