VBR makes sense when you have to put 3 hours MPEG-2 in a DVD.rafael

Currently, the quality fluctuation at bitrates of 100 Mbps and higher generates MPEG-2 Long GOP footage with a quality flaw due to the large differences between the I-Frame and P-frame, though I notice this quality jump nearly every six frames in Sony Vegas (100 or 140 Mbps). Other user comments along the lines of "I-Frame Only at 220-280 Mbps looks cleaner" indicate other users may notice the quality drop and or variation.

If VBR mode allows for specifying a constant quality per frame even if the bitrate varies and increases somewhat, VBR is not only sensible but critical to the quality of the Nano output files. Right now, I'm find that I need to record I-Frame only 220 Mbps to achieve a quality that should be available at LongGOP bitrates at half that rate.

Currently, our MPG files are CBR.

In our upcoming release, our next firmware release, they will be VBR.

Excellent news, though I'd prefer this in a user-programmable option. Is there a quality tuning effort underway? In MPEG-2 algorithm documentation, there is mention of constant quality (Q) as well as independent parameters to balance the relative degradation of I, B and P frames. Is CD looking at these?

Dear Gints,

We found for for our MPG files, the VBR was more compatible with other devices than CBR. I think, for these files, VBR is far more common.

Could you please send me a link to the MPEG-2 documenation you mentioned. I would like to read it.

Currently, the quality fluctuation at bitrates of 100 Mbps and higher generates MPEG-2 Long GOP footage with a quality flaw due to the large differences between the I-Frame and P-frame, though I notice this quality jump nearly every six frames in Sony Vegas (100 or 140 Mbps). Other user comments along the lines of "I-Frame Only at 220-280 Mbps looks cleaner" indicate other users may notice the quality drop and or variation.

If VBR mode allows for specifying a constant quality per frame even if the bitrate varies and increases somewhat, VBR is not only sensible but critical to the quality of the Nano output files. Right now, I'm find that I need to record I-Frame only 220 Mbps to achieve a quality that should be available at LongGOP bitrates at half that rate.

Thank you Gints for reminding what this thread has been about since the very beginning : the evident quality problems due to shimmering noise in some nanoFlash modes.

Introducing additional VBR L-GoP modes (or even replacing the current CBR modes from 100 Mbps up for L-GoP recording) that's being discussed now, is just one of the ideas brought forward by the thread participants, as a potential remedy to the disappointing L-GoP quality, due to the quality fluctuation within any given GoP. The fluctuation is seen as shimmering especially on large screens, and can outweigh the advantages that the nanoFlash high bitrates bring about (like the reduced mosquito noise, better detail, and of course doubled chroma resolution).

Whether and how the VBR is able to eliminate this quality fluctuation can only be tested and assessed in the Convergent Design labs - we the end-users are in no position to decide on that, but we (myself, for one) are ready to test the VBR modes in our shooting scenarios, and NLE systems. Also, I do appreciate that introducing all new encoding modes would be a huge task for CD, and may not be viable as their number one priority at the moment.

So, to sum it up, the whole VBR idea is not even a "feature request" from nanoFlash users, but merely a suggestion on how its PQ could possibly be improved.

But, other methods of fine-tuning the encoding chips should also be considered by CD - in my own humble opinion, "opening up" the max bitrate may not help with the fluctuation in question. If one goes back to my first posts in this thread, my examples make it evident that the highest bitrates are even more disappointing in this respect than - say - the 50 Mbps mode.

Speaking of which: the advent of the newest PMW-500 camera with just 50 Mbps 422 CBR, and the fact the XDCAM HD422 codec has never been used in any Sony camera at higher a bit-rate than 50 Mbps, may indicate that this in fact is the sweet spot for this particular codec - and not much can be done to improve it by "brutal force" of increased data rates... Trying to catch up with the HDCAM may be pointless, as they are different animals - even though they also use MPEG 2 L-GoP, as well.

The bottom line of my reasoning here is: bitrates higher than 50 Mbps in 4:2:2 L-GoP may not be capable of further improving the "noise vs. detail" aspect of PQ, unless they are accompanied by further chroma resolution increase.

Dan, would the XDCAM HD chip be capable of 4:4:4, if only at limited data rate, like 100 Mbps?

;)

Piotr

PS. We can dream, can't we...

Hi Piotr,
Although I believe that VBR won't adds nothing, I agree very much with your post.
What I don't understand is the last paragraph:

"The bottom line of my reasoning here is: bitrates higher than 50 Mbps in 4:2:2 L-GoP may not be capable of further improving the "noise vs. detail" aspect of PQ, unless they are accompanied by further chroma resolution increase.
Dan, would the XDCAM HD chip be capable of 4:4:4, if only at limited data rate, like 100 Mbps?"

Why do you want more chroma?
If the processor or the "compression formula" are not able to cope properly with 422, you think will work better with a a 30% more info to cope with?
Aside that 422 still being the standard, more chroma just would add more problem. What we have to think is how to manage properly those 422.
For me the problem is simply the Closed/Long GOP structure.
More key frames and higher data rate is the only way to get a better picture in MPEG-2.
With the NANO, data rate is not an issue.
Probably is a matter of tweaking the SONY formula.
SONY has designed a format for the people that used Betacam (good enough if you don't give a closed look), but seems that the codec (Processor?) doesn't work when you try to get the most from your camera.
Cheers,
rafael

Dear Piotr,

The nanoFlash is not setup for 4:4:4 recording.

This requires dual-link or HD-SDI 3G, and a camera capable of outputting 4:4:4. Your camera, and most others, do not output 4:4:4. Usually only very high-end cameras are capable of 4:4:4.

Amazing as it seems, the Canon HV30/40 and maybe others output 4:4:4 over HDMI.

This would require fast CompactFlash cards.

At this moment, I do not know if the Sony Codec supports 4:4:4.

For me the problem is simply the Closed/Long GOP structure.
More key frames and higher data rate is the only way to get a better picture in MPEG-2.
With the NANO, data rate is not an issue.
Probably is a matter of tweaking the SONY formula.
SONY has designed a format for the people that used Betacam (good enough if you don't give a closed look), but seems that the codec (Processor?) doesn't work when you try to get the most from your camera.
Cheers,
rafael

Rafael,

You are absolutely right - I've always called for L-GoP "fine-tuning" throughout this thread.

My statement about 4:4:4 was meant as a kind of provocation for some more brain storm in this thread, you must have noticed the "blink" character. I expected Dan's answer to be what it is :)

Piotr

Hi Piotr,
The problem is that if we start to talk about 444 or 10b, we get lost on discussions again.
In short: When the MPEG-2 developers started to play with the ideas of 444 and 10b, they realized that was better to develop a new format: MP4.
8b 4.2.2 YUV is a solid formula. That's what I wanted when I purchased the NANO.
The point is to get the most that we can from the NANO instead of dreaming.
Would be great if Dan would say something about how "tuneable" is the NANOs processor.
As Dan points, some NLEs could have some issues working with these files, but some NLEs probably would manage them without problems.
That would be a good starting.
Cheers,
rafael

Hi Rafael,

I agree 100%.

If I let my mind drift for a while was just because of the lack of a - shall I say - more constructive response from CD to our inquiries on "how tunable the nanoFlash encoder is"...The only firm answer so far being that Closed GoPs are used.

It took several weeks before I finally got the confirmation that indeed, 12-frame-long GoP is used in the 1080 50i/24p/25p modes.

With the competition around the corner now, let's hope this will get some more insight from CD engineers - but let's also remember there are still some promised features waiting for implementation!

Piotr

OK, with all this information out now,
am I to assume that the sweet spot for mpeg recording is now 50mbs not 100mbs?

And the sweet spot for I frame only recording is 220?

Ron,

Assuming you're shooting with an EX1/R/3, yes - that's right.

However it all depends on the actual S/N of your source camera. For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!

Well - no wonder; just the GIGO formula at work... The goal now being how to possibly tune the L-GoP nanoFlash encoding so that it does NOT augment the noise.

Piotr

..... For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!..............
Piotr

Dear Piotr,
I am really surprised that it took you so long to try that option of test! I remember suggesting that months ago just as others did.....
I am finally happy for you! Some how we all got involved in your ( as you said) paranoia! ;-)

I love my NF and it does deliver more then I've expected! I am so pleased, that I even consider buying the 3D single unlocked unit. That in combination with the one I have would open way more oportunities for a freelancer!
Now that we may see some price war ( I hope, but I doubt) maybe the right time to buy it... CD may provide some special discount for exsisting costumers!, you never know....
Good luck to you and Thank you for this Thread!! It was exhausting but very informative in any direction!
Cheers mate

Hi Ron,
50 Mbps may be the sweet spot for the SONY XDCAM 422 customers.
With XDCAM, SONY has targeted his old Betacam customers. And SONY knows about the needs in term of archiving of his customers.
50 Mbps is a trade quality-files size.
But we are not XDCAM 422 users and the file size is not our main concern, but the picture.
We expect much more from the Long GOPs.
rafael

Dear Ron,

For Broadcast work, our 4:2:2, 50 Mbps, CBR Sony XDCam 422 Long-GOP footage is widely used and accepted.

We say that the 4:2:2, 100 Mbps, CBR XDCam 422 Long-GOP footage is the sweet spot as while this option uses more space, the quality is visually better, without using too much space.

Our 280 Mbps I-Frame Only is the best quality of all of our options.

For those wanting I-Frame Only, such as for Avid users, I personally recommend 140 Mbps or above.
Avid also support our 50 Mbps 4:2:2 Long-GOP.

Dear Piotr,
I am really surprised that it took you so long to try that option of test! I remember suggesting that months ago just as others did.....
I am finally happy for you! Some how we all got involved in your ( as you said) paranoia! ;-)

I love my NF and it does deliver more then I've expected! I am so pleased, that I even consider buying the 3D single unlocked unit. That in combination with the one I have would open way more oportunities for a freelancer!
Now that we may see some price war ( I hope, but I doubt) maybe the right time to buy it... CD may provide some special discount for exsisting costumers!, you never know....
Good luck to you and Thank you for this Thread!! It was exhausting but very informative in any direction!
Cheers mate

Dear Luben,

I'm also very happy for you having so many freelancer opportunities:)

Just for the records, though: I did test all possible PP combinations, including PP off, before - if I commented obout it today was just to illustrate my answer to Ron - that there's no single best setting of the nanoFlash alone. Everyone must consider it together with his/her acquiring device.

With EX1, using rich colors in combination with certain gamma curves tends to increase the noise, and - along the GiGo rule - it gets augmented by the nanoFlash... No doubt about it, and the need to fine tune the nanoFlash high bitrate L-GoP encoding so that the increased storage space is accompanied with real quality gains....

Cheers

Piotr

However it all depends on the actual S/N of your source camera. For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!

Yes, I would suggest to all the people out there getting your camera to do gyrations with picture profiles to invest in a DSC Chart (they aren't cheap), shoot the chart at the start of each scene or shoot you do and don't yank the color in the camera but do it in POST. Some of those picture profiles though issuing relatively good color are pulling on the color space in a severe manner and is probably resulting in noise exaggeration.

I felt this for a while abut the EX but am only now coming to a more solid conclusion on it.

Further to that bring out the lights so the noise floor is further reduced.

Now that the AJA mini is recording 10 bit, the discussion on whether the nanoFlash only recording 8 is enough has been revived. Before, the consensus has been that with cameras like the EX series the last 2 bits would not add any useful information but noise...While this is most probably right, I'd like to ask both Convergent Design and fellow users more knowledgeable than myself for some input on the following issue:

As you know, after a long and exhaustive testing the EX1/nanoFlash combo I have finally sort of reluctantly settled with recording 100 Mbps max (for L-GoP) or 220 Mbps min (for I-Fo), and preprocess with Neat Video plugin (whose Vegas version is working great, the only caveat being it takes ages).

However what I have noticed is that after noise removal, some areas of the picture (the very same that suffer from noise the most - mid-bright, uniform color) tend to show another ugly shortcoming: the color banding!

The logical explanation that I have is that before cleaning, the noise acts as dithering the color information; it's "dirty" but it's not banding, even with just 8 bit sampling. After noise removal, the same 8 bit is not enough - hence the banding.

What I've been doing with clips of the greatest importance to me (in terms of the PQ) is apply the Neat Video de-noiser, and render out into a 10 Bit codec (like the Sony YUV) - the banding is gone!

However I still am at loss as to a complete workflow; after all the delivery format I'm using is 8 bit anyway, so the final render must be to an 8 bit codec. So far, it looks promising as after doing all the editing stuff required (CC, levels, etc), my Vegas renders of such 10 bit clips into - say - a 35 Mbps, 8 bit, 4:2:0 mpeg-2 or 20 Mbps AVC for Blu-Ray do not show the banding - i.e. it's not coming back once eliminated.

I'm open for any suggestion re: the optimal workflow, but for now, I'd like to revive the 8 vs. 10 bit discussion on the nanoFlash. I'm not sure of details, but I think I've read the nanoFlash simply truncates the last 2 bits of the HD-SDI stream, and that the alternative method of actually dithering full 10 bit information into 8 bit recording could be possible...

Dan - do you think it could help with color banding like this mentioned above? And (I know it's a log shot) - wouldn't actually recording the full 10 bit (at lower bitrates - I know of the bandwidth limits) help with both noise not being augmented, and - after removing the noise generated in camera head - prevent the banding completely?

"Now that the AJA mini is recording 10 bit, the discussion on whether the nanoFlash only recording 8 is enough has been revived. Before, the consensus has been that with cameras like the EX series the last 2 bits would not add any useful information but noise...While this is most probably right,".

Piotr,
That makes no sense at all, and to discuss the advantages of 10b over 8b is a waist of time.
The signal out of the SDI, with or without noise, is 10b.
Do you think could be any benefit on crunching it to 8b?
The ideal would be to record it to 10b Uncompress. Any other option will degrade the picture, and an 8b codec will degrade the picture more than an 10b codec. I'm talking of course about decent codecs.

About NeatVideo, you said:
"What I've been doing with clips of the greatest importance to me (in terms of the PQ) is apply the Neat Video de-noiser, and render out into a 10 Bit codec (like the Sony YUV) - the banding is gone! "
Sure, NeatVideo must be rendered in 10b. Internally works in RGB, and unless you render in Floating point, you will get all your Highlights clipped. That happens when you render in 8b RGB: No room for "SuperWhites".
In FC is necessary set "render all YUV in High Precision".

"However I still am at lost as to a complete workflow; after all the delivery format I'm using is 8 bit anyway, so the final render must be to an 8 bit format. So far, it looks promising as after doing all the editing stuff required (CC, levels, etc), my Vegas renders of such 10 bit clips into - say- a35 Mbps, 8 bit, 4:2:0 mpeg-2 or AVC for Blu-Ray do not show the banding - i.e. it's not coming back once eliminated".
Acquire as good as you can, and always render with the top quality you can, exporting to the codec that retains the most from the process done. Compress to the delivery format in the last step.
rafael

Dear Piotr,

In reference to dithering:

We fully implemented dithering, then we tested it in our lab.

Dithering did not help the quality of the image, and it interfered with the efficiency of the codec.

So, we decided not to include dithering in our production firmware releases.

You mentioned the 8-Bit versus 10-Bit debate.

Of course, we know that 10-Bit is desirable, if one has a very good camera, with a very clean 10-bit output. These cameras do tend to be the expensive cameras.

And one should know, that while HDMI 1.3 or above supports 10-bit, all HDMI cameras that we know of have 8-Bit HDMI output.

HD-SDI, of course, can support 10-Bit.

The proof is in the images. While a camera may technically put out a 10-Bit HD-SDI source, one should carefully examine the images to determine if the 10-Bit actually helps or not.

This is not meant to detract from a good clean 10-Bit camera. We use the Sony codec module which produces does an amazing job, and it is only 8-Bit.

My comments are meant to say that one typically needs an expensive camera to get a good clean 10-Bit output.

One way to test a camera to determine if the 10-Bit is clean, is to use an expensive 10-Bit professional HD-SDI monitor. Just setup the camera at static scene, or a color chart, then examine the image on the monitor. Nothing should be moving and no color changes should be present. Then vary the lighting.

(Of course there are other ways to test the quality of the cameras output0.



As one piece of the puzzle, I would like to discuss codec efficiency.

Adding the extra 2 bits requires a higher bit-rate just to compensate for the two extra bits.

Doing the math:

220 Mbps at 10-Bit is similar in performance to 176 Mbps in 8-Bit

145 Mbps in 10-Bit is similar in performace to 116 Mbps in 8-Bit

220 Mbps in 8-Bit requires 275 Mbps in 10-Bit

280 Mbps in 8-Bit requires 350 Mbps in 10-Bit


The above examples do not take into consideration any of the other differences between the codecs, just the difference when having to process the extra two bits or not.


My reading of the Apple ProRes white paper, is that ProRes is always 10-bit (or higher).
I carefully read the white paper and did not find any mention that they would use only 8-Bits for an 8-Bit source. Of course, this level of detail may be missing from the white paper.

http://images.apple.com/finalcutstudio/docs/Apple_ProRes_White_Paper_July_2009.pdf

Hi Dan,
I disagree very much when you say "My comments are meant to say that one typically needs an expensive camera to get a good clean 10-Bit output".
Also I disagree very much when you suggest a 10b monitor.
The difference between 8b - 10b, (as its happens with the difference 420 - 422) may not be very obvious on your screen, but will arise in postproduction.
Cheers,
rafael

Dear Rafael,

I am trying to understand your comments.

Do you know of a low cost camera that puts out a good clean 10-Bit signal over HD-SDI?

One where the quality of the 10-bit image is not affected by the noise level of the camera?

If so, which one or ones?

My comment about using a good 10-Bit monitor was meant to provide a reasonable way to evaluate a camera's output, without any recorder, or NLE affecting the test.

How would you determine the quality of a cameras' 10-Bit output, without resorting to expensive test gear?

I respect you opinions and I agree that the benefits of modern technology help in post and in multiple generations.

Dear Dan,
I was in a hurry when I wrote that and didn't explained my self.
What I wanted to say is that the possible noise in an 10b SDI is not an excuse to record it at 8b.
MY comment about the monitor, in this moment, I don't remember why i wrote it.
Some times I post too late.
Best,
rafael

Dear Ron,

For Broadcast work, our 4:2:2, 50 Mbps, CBR Sony XDCam 422 Long-GOP footage is widely used and accepted.

We say that the 4:2:2, 100 Mbps, CBR XDCam 422 Long-GOP footage is the sweet spot as while this option uses more space, the quality is visually better, without using too much space.



For a newby like me it is quite hard to decide now what setting to use. I am going on an expedition. Not data storage capcity, but power capacity will be our challange. When downloading to a nexto, 50Mbps footage is twice as fast as 100Mbps, so it saves battery capacity (of the nexto). In what shootings is 100 visually better? I will make a nature documentary, slow pans, no fast movements (maybe the birds wings) etc. Is 100Mbps also better in such circumstances or can I stick to 50 and thus be more efficient with storage and power consumption?

Cees, which camera are you using?

If it's an EX1(R)//EX3, than my recommendations are:

- for L-GoP: 100 Mbps max if you're planning extensive post-processing, 50 Mbps if you're just cutting
- for I-Frame Only: as high as possible (i.e. 280 Mbps if your cards allow, otherwise 220 minimum).

The reason being that the EX series cameras are indeed noisy, and L-GoP at bitrates higher than 50 Mbps actually augment this noise (or strictly speaking, do not mask it enough, like the more compressed, i.e. lower bitrate, formats do).

This observation has been confirmed by Dan who has found time to watch my sample footage of L-GoP @50 vs. @180 Mbps, and admitted some picture areas of the latter are unacceptable straight from the nanoFlash. (Thanks, Dan !).

Of course, when footage at even 180 Mbps bitrate is subject to heavy grading and multiple re-compressions, it should hold up better than the 50 Mbps recording. Also, the final encoding to a delivery format (at a bitrate usually not exceeding 50 Mbps anyway), should result in the excessive noise being suppressed. I'm saying "should", because I never actually tried it in my workflow; I'm sticking to the rules outlined earlier in this post.

If you want to use nanoFlash files without postprocessing, however, either use 50 Mbps Long GoP, or at least 220 Mbps I-Frame Only.

Piotr

Dear Cees,

Our 50 Mbps 4:2:2 is exactly the same image quality as the Sony PDW-700 and PDW-F800 cameras, and is widely accepted as Broadcast Quality.

Here is my opinion:

Our 50 Mbps looks very good. It is intended to be viewed at normal frame rates.

If one wants "extra insurance", or ability to capture great images where there is an excessive amount of detail, or lots of movement in the image, or lots of camera movement, or when each and every image may be used as a still frame, then I would go with 100 Mbps or one of our other higher bit-rate options.

If I was making a movie, and power was not a consideration, as it is for you, then I would use higher bit-rates.

You have one of the challenges that makes the nanoFlash so desirable.

You need high quality, but you are limited by power. You can easily choose 50 Mbps and help solved the power problem.

My recommendation: Run some tests, check out the 50 Mbps footage.

Background:

The Next DI devices have a built-in Liithium Ion battery. After xx Gigabytes of copying CompactFlash cards the battery is going to be exhausted.

50 Mbps footage is one half the size of 100 Mbps footage, so one gets twice the footage on one battery charge.

Cees, are you aware that Nexto has small, external batteries for the Nexto?
You could get a few of these and solve your power problems. They only weigh a few ounces.

And they have an AA Battery pack option.

Also, if at all possible, do the copying of files when you have AC power.
(I realize that this may not be possible on your upcoming trip).

Piotr, it is an EX3 I use.

Dan, Indeed I won't have access to AC power. I do have extra external batteries with my nextos. If it turns out I can keep up with charging my batteries (nextos, EX3 batteries, SLR equipment), I will use 100Mbps. As soon as power consumption seems to go faster than recharging all the batteries (I will use portable solar panel and the outboard engine of our boat), I will move to 50 Mbps LongGop, at least knowing this still is broadcast quality. So I have to care less about the nexto batteries. There will not be a lot of camera movement, not a lot of movement in the image (if the tribal dancing starts I will change to higher bitrates). Excessive detail may occur when shooting close ups of birds (feathers).

It is indeed the kind of challenge that makes me very happy with my nano!!

Thanks to you both for your prompt reply.

Cees

- for L-GoP: 100 Mbps max if you're planning extensive post-processing, 50 Mbps if you're just cutting
- for I-Frame Only: as high as possible (i.e. 280 Mbps if your cards allow, otherwise 220 minimum).

The reason being that the EX series cameras are indeed noisy, and L-GoP at bitrates higher than 50 Mbps actually augment this noise (or strictly speaking, do not mask it enough, like the more compressed, i.e. lower bitrate, formats do).


Piotr, You started this thread and the lengthy discussion. The above does not represent the entire issue, as you already know, but I'll write it again. LongGOP above 100 MBps has excessive variation between I and B-frames, and this variation grows with increasing bitrate resulting in nearly no benefit between 100 MBps and 180 Mbps LongGOP. This detail variation is seen as shimmering seen in image including detailed subjects as grass or wet sand - not just noise from Sony EX cameras. I-Frame Only solves the inter-frame problem at 220+ MBps, but I suspect this is an I/B frame quality balancing issue in LongGOP more than inadequate bitrate.

Piotr, You started this thread and the lengthy discussion. The above does not represent the entire issue, as you already know, but I'll write it again. LongGOP above 100 MBps has excessive variation between I and B-frames, and this variation grows with increasing bitrate resulting in nearly no benefit between 100 MBps and 180 Mbps LongGOP. This detail variation is seen as shimmering seen in image including detailed subjects as grass or wet sand - not just noise from Sony EX cameras. I-Frame Only solves the inter-frame problem at 220+ MBps, but I suspect this is an I/B frame quality balancing issue in LongGOP more than inadequate bitrate.

But Gints, I fully agree - I never backed out of this conclusion of ours. However, considering CD does not confirm it (and obviously, isn't going to do anything about L-GoP fine tuning), to questions like that of Cees I am answering with what the work-around is, rather than beat the dead horse again and again.

Have you noticed, Gints, the we are the only two persons in this thread who can see the obvious imperfections in the nanoFlash L-GoP structure? Well - 2 users is not enough to make CD admit it, not to mention fix it, for us :(

But Gints, I fully agree - I never backed out of this conclusion of ours. However, considering CD does not confirm it (and obviously, isn't going to do anything about L-GoP fine tuning), to questions like that of Cees I am answering with what the work-around is, rather than beat the dead horse again and again.

Have you noticed, Gints, the we are the only two persons in this thread who can see the obvious imperfections in the nanoFlash L-GoP structure? Well - 2 users is not enough to make CD admit it, not to mention fix it, for us :(

Dear Piotr,

I am going to step out of character and respond to your accusations:

On Post 165 of this thread I posted:


Dear Piotr,

The Sony Long-GOP consists of I Frames, B-Frames, and P-Frames.

They are not all created equal in terms of quality.

If you want the highest quality, we recommend 220 Mbps or 280 Mbps I-Frames Only.

Our 280 Mbps I-Frame Only has been thoroughly tested by a major network for one of their high production value prime-time shows using the most sophisticated of test equipment.

The quality met their specifications and needs.

As I calculate it, you are looking at two frames, zoomed in 310%.

Please feel free to ship us the DVD, if you wish.

Thus your comment that Convergent Design "Does not confirm it" is false.


In Post 1 of this thread you stated:

...Now, I have deliberately chosen a back-lit scene like this, and a picture profile not designed to minimize the chroma noise, so that the results are more readily visible. But frankly, I never suspected the difference would be so great, and not in favour of the nanoFlash...

And you have continued presenting images throughout this thread where your camera or lighting conditions were not optimized to produce a quality image, and then faulted the nanoFlash for not removing the noise from the images.

Then, after 25 pages of posts, on Post 361, you posted:

Ron,

Assuming you're shooting with an EX1/R/3, yes - that's right.

However it all depends on the actual S/N of your source camera. For instance, I've just run a series of new experiments - and with PP OFF (yes - not just DETAIL, but also the matrix, etc at the factory settings), even with my EX1 as the source, the 100 Mbps nanoFlash L-GoP looks gorgeous!

Well - no wonder; just the GIGO formula at work... The goal now being how to possibly tune the L-GoP nanoFlash encoding so that it does NOT augment the noise.

Piotr

Previously, I asked for details on how you were creating these images and you refused.

Others advised you to turn off your picture profile.

In your most recent post you state:

(and obviously, isn't going to do anything about L-GoP fine tuning)

I am going to politely point out that you are not fully aware of our internal developmental plans, nor do you fully know what we are currently working on, nor are knowledgeable about what we are going to do next.

Your accusations are unfounded, unwarranted, and unwelcome.

Dear Dan,

I'm very sorry that - unintentionally - I made you "step out of character". Really, didn't mean to...

But I must defend myself, as I don't want anyone following this thread to think that what I have been writing in it, has been aimed against CD or you personally.

So, first of all, I must say I got very sad reading that "my accusations are unfounded, unwarranted, and unwelcome". The reasons you made me so sad are:

1. First of all: nothing written in this thread has been meant as "accusations".

2. Secondly, even though in our latest e-mail exchange you agreed that my footage at 180 Mbps shows defects in the image - at the same time you kept blaming the lighting conditions and other factors, not related to how nanoFlash encodes L-GoP. Even though the very same scene I sent you recorded at 50 Mbps was clean!

This made me believe CD is not acknowledging the issue as described by myself and Gints in this thread.

So, even though indeed, as you put it: "I am not fully aware of CD's internal developmental plans, nor do I fully know what CD are currently working on, nor am knowledgeable about what CD are going to do next"

- I simply thought the message from you was clear that the excessive noise has been my camera fault. If however CD is seeing the problem, and plan to take a closer look at it - this is great news, and I'll be the first to thank you for this extra effort.

3. Last but not least, what makes me sad is that you didn't appreciate the fact I moved this discussion from the public thread to our private email exchange, in order to avoid anything that could be perceived as criticism towards Convergent Desing, and/or the nanoFlash. As a loyal customer, I though it was important in the specific situation we're facing, with the challenge from CD's competitors... In one of my emails, I wrote: "I hope you agree that - especially with the current challenge by the competition - it's important fro Convergent Design that my lengthy thread on DVInfo was concluded with some constructive outcome... By "constructive outcome", I meant some well grounded consensus between users like myself and Gints, and Convergent Design, on the nature of the issue and how it can be resolved or worked around.


If - even with the above clarification - you still feel personally offended, please accept my sincere apologies.

Piotr

This has been a very interesting and productive thread.
After almost 400 posts, my conclusion is that would be desirable to try to fine tune the LGOP structure of the NANO.
CD has been making an incredible effort trying to give to the NANO all the operational options needed for a wide range of cameras and NLEs and for a variety of scenarios and jobs.
Once those operational needs are satisfied, it would be great if we could pay a bit of attention yo the NANO encoding.
We know how to improve the MPEG-2 compression, but we don't know if the NANO meet the options to do so.
In the end what we want is the NANO to be the best 8b recording solution.
With most affordable cameras to come outputting 8b Uncompress, the NANO will still being the perfect recording companion.
Best,
Rafael

To finalize the question I posted in this thread. Yes: I have wishes that I would like to be optimized on the nano. And yes, I find it ridiculously valuable as it is. IT IS GREAT DEVICE. And so is the support from CD, answering my questions early mornings, late at night and during the whole of the weekend. Never experienced anything like that before. I am sure Poitr also realizes that. He himself is by the way also very helpful at any moment and obviously having very valuable knowledge. So both of you, I learned a lot this weekend. Thanks a lot!

Cees

IT IS GREAT DEVICE.

Of course it is, Cees - mine is permanently attached to the camera (using Olof's plate and bracket), and I never shoot without it!

And so is the support from CD, answering my questions early mornings, late at night and during the whole of the weekend. Never experienced anything like that before. I am sure Poitr also realizes that.

Yes, I do realize that, and have always been very grateful to CD and Dan in particular...

Cheers,

Piotr

Have you noticed, Gints, the we are the only two persons in this thread who can see the obvious imperfections in the nanoFlash L-GoP structure? Well - 2 users is not enough to make CD admit it, not to mention fix it, for us :(

This is probably because we represent the operational cost-conscious segment of CD's market, and I hope our comments make a difference. I thought 80 GBytes/hour of footage was a lot at 140 Mbps (shoot SxS 35 Mbps in parallel.) , and this footage needs at least two backups. The rest of CDs customers will just use 220-280 MBps.

However, considering CD does not confirm it (and obviously, isn't going to do anything about L-GoP fine tuning),

There is a possibility that CD may not be able to do anything about it as they do not make the Sony codec. MPEG shimmering is a common problem and usually indicates inadequate bitrate. After tuning for constant quality, it is also possible that LongGOP may need to operate at double the bitrate, anyway. We continue this discussion because we *suspect* that LongGOP should be at least twice as efficient as I-Frame only.

Any news on L-GoP optimization, Dan?

Thanks,

Piotr

Piotr, It's taken me this long to accept that recording I-Frame only at 220-280 Mbps is the way to go. Yes, I was disappointed that 6-8x data rate is needed to exceed the 35 Mbps Sony EX1, but that is the best portable high quality video system available. We did not expect that the higher data rates would expose sensor noise on the Sony EX1 and that much of the high bitrate is channeled to code this noise as accurately as possible. Still, better inter-frame data balancing improvements would be welcomed in a future release. Nano could be that ultimate MPEG-2 LongGOP recording machine.

Hi Gints & Piotr:
This is where one truly hits the wall with MPEG 2. MPEG 4 is actually much more efficient at encoding at lower data rates. MPEG 2 has to use high data encoding rates to obtain even slight improvements in quality, whereas MPEG 4, Quicktime, MJEPEG, Cineform and Cineon do not. Quicktime allows for 8,10 & 12 bit color precision encoding. MJEPEG will do 8 or 10 bit encoding precision, and Cineform will do anything. The only way to get SD & HD MPEG 2 to look really great is to use those famous $50,000.00 Zoran hardware, realtime MPEG 2 encoders, which use special chip optimizations for CBR & VBR encoding @ key DVD authoring data rates. (Usually, 4, 6 and 8 Mbps for SD) (18 to 24 Mbps data rates for Blu-ray) These boards can give you the big screen HD quality you seek @ 18 Mbps data rates ! I have no idea of what *Optimization settings* the Sony hardware MPEG 2 XDCAM 4:2:2 encoder is set at to give you the optimized MPEG 2 results you are looking for ? (Probably Long GOP 50 Mbps would be one of the targets this chip in the XDR & Nano is optimized at to give you a really great picture) I have observed Long GOP 100 to be no sweet spot as far as I am concerned in regard to a very clean BIG screen or projected picture, yet I have seen XDCAM Long GOP 50 projected , as well as on a 50 plus inch flat screen, and you see a remarkably clean image without the granularity & flashing you guys are seeing.

A word to the wise; If you are using FCP to post your Nano/XDR clips, then please be ware you are at a certain disadvantage, which can actually cause the anomalies you are seeing on large screens. What do I mean ? Simple. FCP is processing all of your effects @ 8 freaking bits ! Now with Avid (And **MANY** Avid Media Composer editors simply do not know this as well !) - AMC gives you the *Option to process and render @ 16 bit precision !!! Yup !!! This is what I wrote ! Many folks don't even know how to turn it on !!!

**This setting is in the >Media Creation>Render Settings Tab !

1. Don't check the Same as Source box, because your source is only a sucky 8 bits !
2. Don't select *Automatic* because this tells Avid Media Composer to analyse the video clips and automatically process all effects at the same color precision as your clips (Sucky 8 again !)
3. DO SELECT 16 BIT PROCESSING ! Even Avid goes into some detail in their instruction manual as to how 8 bit sources can benefit from turning on 16 bit processing.

Two things will now happen to what you post:

A) Your processing (Read Rendering) times will take 3 times longer than before.
B) Your resulting output will look demonstrably smoother, there will be *MORE* color ! There will be *LESS GRANULARITY* ! You guys *WILL NOT* see those anomalies you are seeing anymore (Providing you shoot @ Long GOP 50 Mbps) You simply won't believe your eyes. - And what you observed for I-Frame recording @ 220 and 280 Mbps will still apply, but wait until you see what that now looks like !

Try my solution - You will like it !

Mark, this page shows info that is different than what you posted.
Apple - Final Cut Studio - Tech Specs and System Requirements (http://www.apple.com/finalcutstudio/specs/)
I'm not trying to challenge your info I'm just trying to learn, Am I reading what they are listing wrong? I would like to know because I just got the Nanoflash and am trying to get the most out of it.

Piotr, It's taken me this long to accept that recording I-Frame only at 220-280 Mbps is the way to go. Yes, I was disappointed that 6-8x data rate is needed to exceed the 35 Mbps Sony EX1, but that is the best portable high quality video system available. We did not expect that the higher data rates would expose sensor noise on the Sony EX1 and that much of the high bitrate is channeled to code this noise as accurately as possible. Still, better inter-frame data balancing improvements would be welcomed in a future release. Nano could be that ultimate MPEG-2 LongGOP recording machine.

Hi Gints,

Same here - I invested in a couple more 64 GB cards, and whenever shooting something "serious enough" (meaning intended for heavy grading etc.), I'm using I-Frame Only 220 Mbps.

At other occasions when L-GoP is enough, I almost exclusively use the 50 Mbps bitrate.

Mark,

I'm not a Final Cut or Avid user, but you're right that a lot can be improved in post. Vegas Pro has an option of 32bit floating point pixel processing, and I'm using it whenever my footage needs de-noising. For that, I'm using the Neat Video plugin, which gives excellent results (not just de-noising, but also sharpening the footage to my liking). The price is much longer rendering times, so - having invested in the nanoFlash - I'd rather get clean pictures to start with, and of more manageable sizes than the 220 Mbps is producing...

Mark, this page shows info that is different than what you posted.
Apple - Final Cut Studio - Tech Specs and System Requirements (http://www.apple.com/finalcutstudio/specs/)
I'm not trying to challenge your info I'm just trying to learn, Am I reading what they are listing wrong? I would like to know because I just got the Nanoflash and am trying to get the most out of it....Hi Joe:
I don't see anything in those listed specs which would indicate FCP can do internal 16 bit processing of files and effects, or color correction. It's the internal processing (and Non-transcoding) of your clips which will get you the most out of your Nano video. AMC does all of what is listed in FCP anyway, although I would have to say FCS is a better finishing system, and Avid is a better cutting application.

Thanks for the info, I guess I don't understand INTERNAL processing , I'll have to look it up. Also I thought all Nano files are 8 bit. I really want to figure this out. I'm having trouble accepting that FCP can't retain the original quality of the Nano's clips. Thanks for any help.

Thanks for the info, I guess I don't understand INTERNAL processing , I'll have to look it up. Also I thought all Nano files are 8 bit. I really want to figure this out......FCP must *Transcode* (Read Re-Compress) the Nano 8 bit codec to use it. Avid Media Composer will not Re-compress (Transcode) your Nano 8 bit clips even if you tell it to, it will automatically import them "as is," in their *Native format. This is a real biggy. Also, AMC has an option to perform all rendering, effects, & color correction rendering using 16 bit image processing. What I do is import my XDR clips (Same technology as the Nano Clips) into AMC. perform all editing,transition effects, titling, nested effects, including multi-layer digital compositing in the native format, then transcode to Avid's DNxHD 175 X (The X stands for 10 bit color precision), then do all my color correction of my 8 bit clips in 10 bit. Then I perform a little trick which makes your 8 bit stuff look like 4:4:4 12 bit ! No I'm not exaggerating here ;-) Although I acknowledge my claim sounds ridiculous ! I mark an IN point at the beginning of my edited sequence (Timeline), then an OUT point at the end of my sequence. After this I right click on the timeline and select unrender In/Out. Next I make sure I've turned on 16 bit render processing and I right click again on my timeline and select render In/Out, then Voila ! Wow ! Everything outputs so incredibly superior to anything I've ever seen before ! Try it-It's really incredible !

1. Edit in native 8 bits.
2. Color correct in 10 bits.
3. Render and process for output in 16 bit !

WOW !!




I'm having trouble accepting that FCP can't retain the original quality of the Nano's clips. Thanks for any help.[/QUOTE]

I'm wondering about a few items regarding workflow in FCP Studio and this discussion of the best image quality:

1. Working with a FCP Uncompressed Timeline and round trip to Color for color correction - or working in a Pro-Res 10bit 4:4:4 timeline

2. Color can be set to work in 32 floating point which basically further divides the 8,10,12, or 16 bit down into more units yet for grading

3. In mid-September, FCP put out an update that added better support of XDCAM HD422 - I haven't had time to figure out what the upgrade has done to benefit our CD files yet though. Maybe someone else can chime in.

But I think this level of workflow is probably limited to the lucky folks who are working with filmout or high end broadcast.

Mark, wow. Great informative post on the practical distinctions between mpeg 2 and 4 and on the avoidance of 8 bit rendering.

I can reassert Mark's observation about Final Cut. It is setup to do 8 bit conversions and rendering in several places within a given project and many filters are 8 bit as well.

I would recommend that anyone regardless of whether you are using high bitrate footage from a nanoFlash or otherwise turn on 10bit or high precision rendering in FCP if you want transitions and titles to look good. Anyone will see the difference, not just people in the biz.

Hi Gints & Piotr:
A word to the wise; If you are using FCP to post your Nano/XDR clips, then please be ware you are at a certain disadvantage, which can actually cause the anomalies you are seeing on large screens. What do I mean ? Simple. FCP is processing all of your effects @ 8 freaking bits ! Now with Avid (And **MANY** Avid Media Composer editors simply do not know this as well !) - AMC gives you the *Option to process and render @ 16 bit precision !!! Yup !!! This is what I wrote ! Many folks don't even know how to turn it on !!!
e detail in their instruction manual as to how 8 bit sources can benefit from turning on 16 bit processing.
!
Final Cut renders in 32b Floating point when needed.
Just set "Render all YUV in High Precision".
Rafael

.....FCP must *Transcode* (Read Re-Compress) the Nano 8 bit codec to use it. ing trouble accepting that FCP can't retain the original quality of the Nano's clips. Thanks for any help.[/QUOTE]
Absolutely wrong.
As long as you do not render, you can edit and export the FULL original NANO footage.
If you render, of course, your only option is 50Mbps 422. The official SONY format supported by FC.
rafael

Piotr and Mark,

I found peace with this exercise : 220-280 Mbps I-frame MPEG2 for HD is a scaled-up DV25. 1080i60 has an image size that is 6x that of 480i60. 25 Mbps * 6 = 150 Mbps. Let's disregard the NTSC pixel shape and assume 720x480 square pixels. Moving from 4:1:1 to 4:2:2 is a scale factor of 1.3333, bringing us to 200 Mbps. This disregards the differences in frame coding, but ...

I'm wondering about a few items regarding workflow in FCP Studio and this discussion of the best image quality:

1. Working with a FCP Uncompressed Timeline and round trip to Color for color correction - or working in a Pro-Res 10bit 4:4:4 timeline

2. Color can be set to work in 32 floating point which basically further divides the 8,10,12, or 16 bit down into more units yet for grading

3. In mid-September, FCP put out an update that added better support of XDCAM HD422 - I haven't had time to figure out what the upgrade has done to benefit our CD files yet though. Maybe someone else can chime in.

But I think this level of workflow is probably limited to the lucky folks who are working with filmout or high end broadcast.
John,
The NANO footage workflow should be the same than for any other 8b footage.
- Edit native and render/export with the higher possible quality.
No advantage on transcoding before editing.
rafael

The only difference DV25/50, with MPEG-2, is that MPEG-2 is scalar, and supports GOPs.
DV are Intraframe with fix data-rate.
Both use DCT compression.
rafael