Sony X70 4K - Lowest bit rate in the industry!

[Craig Seeman]

As per David's comments on scaling data rate to frame size, Jan Ozer wrote this interesting article some years back.

"Microsoft’s Ben Waggoner, a respected compressionist, uses the power of .75 rule. Here’s a snippet of an email he sent to me explaining the rule.

Using the old "power of 0.75" rule, content that looks good with 500 Kbps at 640x360 would need (1280x720)/(640x360)^0.75*500=1414 Kbps at 1280x720 to achieve roughly the same quality."

The Essential Key to Producing High Quality Streaming Video

[Paul Hardy]

Originally Posted by Jody Eldred

The bottom line is not algorithms and equations, but how the images ultimately LOOK, and is your client and audience happy with them. All else is purely academic.

That hits the nail on the head - well said!

[Bob Searl]

That's what we all want...great or at least equivalent image quality with small file sizes. Hopefully, Sony will live up to it's design vision and the XAVC long GOP (MXF version) will outperform the rest of the XAVC-S version codecs and at historically low(er) Bit Rates which will directly help keep the file sizes manageable.

They have been describing this vision for two years. The promise of the XAVC chip has all been part of their plan. Better chips, better battery life, and better algorithms have been promised. Let's hope that this is a harbinger of what the X70 can deliver--at 4-2-0 no less. The jury is in deliberation and the verdict is near!

Traditional views of Bit Rates alone no longer tell the story.

[Dave Contreras]

Looks like my upgrade license shipped today with an expected arrival date of July 6th! woohoo!

[David Dixon]

Mine shipped as well, and I didn't order from B&H until four days ago.

Edit: mine will also arrive Friday, and B&H has already kindly emailed the rebate form.

[Bob Searl]

Mine is supposed to arrive on Friday AM from B&H!!!

[Cliff Totten]

Originally Posted by Bob Searl

That's what we all want...great or at least equivalent image quality with small file sizes. Hopefully, Sony will live up to it's design vision and the XAVC long GOP (MXF version) will outperform the rest of the XAVC-S version codecs and at historically low(er) Bit Rates which will directly help keep the file sizes manageable.

They have been describing this vision for two years. The promise of the XAVC chip has all been part of their plan. Better chips, better battery life, and better algorithms have been promised. Let's hope that this is a harbinger of what the X70 can deliver--at 4-2-0 no less. The jury is in deliberation and the verdict is near!

Traditional views of Bit Rates alone no longer tell the story.

Allot of people believe that XAVC is an actual "codec". However, it's really just using the industry standard MPEG h.264 compression. There is nothing in XAVC- I, S or L that goes beyond the standard h.264 algorithm. Yes, Sony does have some patented VBR encoding tricks and some look ahead optimization. However, it does NOT "exceed" the h.264 standard and does not include any new mathematics that go outside the official h.264 spec. (Something that ALL companies that employ h.264 have full access to...not just Sony)

Sony's XAVC-S, I and L are all mostly folder structure and container specifications. The Sony XAVC standard specifies metadata storage too. But these, of course don't affect the picture quality of the image.

So, to sum up: "XAVC-x" is a "format/container standard" that uses the MPEG h.264 standard as its "codec" of choice.

60Mbp/s is still way too low. This Pro XDCAM camera should NOT be FAR below all other 4k cameras on the market today. It's a shame that even the cheapest industry consumer 4k cameras ALL exceed the X70 with 100Mbp/s.

[Cliff Totten]

Originally Posted by Craig Seeman

As per David's comments on scaling data rate to frame size, Jan Ozer wrote this interesting article some years back.

"Microsoft’s Ben Waggoner, a respected compressionist, uses the power of .75 rule. Here’s a snippet of an email he sent to me explaining the rule.

Using the old "power of 0.75" rule, content that looks good with 500 Kbps at 640x360 would need (1280x720)/(640x360)^0.75*500=1414 Kbps at 1280x720 to achieve roughly the same quality."

The Essential Key to Producing High Quality Streaming Video

Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording. Not even any other Sony camera today used 60Mbp/s. (No,..not even the cheapest entry level Handycams)

The industry has basically chosen 100Mbp/s or higher for even the cheapest 4k consumer models. Even Sony's tiny "Action Cam" records 4k at 100Mbp/s.

This rule might work ok for simple "viewing" purposes. However, in post production and editing, the lower the bitrate, the faster it will break when color correction or gamma bending/stretching is applied.

Color grading is FAR more demanding of a codec than just simple "viewing" of the video.

Again, this is why no other 4k camera in the industry records at 60Mbp/s.

The Sony PXW-X70 is all buy itself at the bottom of the list.

[David Heath]

Originally Posted by Cliff Totten

Allot of people believe that XAVC is an actual "codec". However, it's really just using the industry standard MPEG h.264 compression. There is nothing in XAVC- I, S or L that goes beyond the standard h.264 algorithm.

But there is no "standard" MPEG H264 compression. Any standard defines how to decode an H264 signal, it only defines coding in so far it has to meet the decode criteria. Hence XAVC is a subset of H264 - it does define how the signal is to be coded, and all the tricks to be used. The result has to be H264 decoder compliant, yes, but saying "it's XAVC" says so much more than saying "it's H264", and that goes way beyond wrappers and containers.

I like to think of it as baking a cake. To meet consumer standards to call a product "a cake" means it must meet certain criteria (and in the UK, it has big tax implications whether it's a cake or a biscuit, but I digress..... :-) ) But there are many, many different ways to bake a cake, and many different ingredients which may or may not be used. Think of H264 as "a cake", and XAVC as "a Victoria sponge".

If you want an illustration, just look at https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels and the table labelled "Software encoder feature comparison". It compares various software encoders - which will all produce an H264 compliant file - but they don't all use anything like all the possibilities that the standard is capable of, and consequently results will vary widely at the same bitrate. That's why you cannot talk of "the standard h.264 algorithm" in relation to a CODER.

It's like going into a shop and asking for "a cake". Do it a few times and you could come out with widely differing things - and some a lot better than others!!

Originally Posted by Cliff Totten

Yes, Sony does have some patented VBR encoding tricks and some look ahead optimization. However, it does NOT "exceed" the h.264 standard and does not include any new mathematics that go outside the official h.264 spec. (Something that ALL companies that employ h.264 have full access to...not just Sony)

Other companies may have access to such mathematics and "tricks" - but that's not to say their products use them. That may be for a variety of reasons. If a defined subset of H264 (as XAVC is now) it's spec becomes fixed for a variety of reasons, or it may be that a given piece of equipment has a coder whose performance is limited by power, technology or whatever. The advantage Sony have with XAVC is that it's relatively recent, hence have been able to fix the latest levels and "tricks" in the spec.

Same with the varieties of XDCAM - they were MPEG2 at heart, but saying "MPEG2" only told you half the story. "XDCAM" defined a lot more. (Compare the relevant performance of HDV and XDCAM as codecs, for example.)

As for "is 60Mbs enough?", then all I'll say is that 60Mbs of XAVC is highly likely to be better than 60Mbs of other forms of (more basic!) H264 encoding! From a manufacturers point of view, then whilst one side may want to up the bitrate for quality reasons, another side may argue for keeping it down for reasons of cost (filesizes, speed of memory needed etc). Which is why you have to consider the market any product is in, it's cost, and what it's target customers are most likely to go with in terms of cost/performance.

[Cliff Totten]

David, I suspect you already know most of this but I will say it for the others reading who might not know.....

You are certainly right that CODING h.264 is different than DECODING it. And yes, manufacturers are free "play" with the CODING aspect all they want....with the limitation that whatever they CODE must be DECODED by a standard h.264 library.(circa 2003 or around there)

For instance, let's say Sony engineers came up with a new trigonometry calculation that improved edge sharpness 15%. Let's say they also modified the CABAC algorithm and CODED their XAVC with 3 new geometry equations to improve entropy block estimations by 10%. How wonderful!,..they just improved their XAVC/H.264 codec by a significant margin!!!

However,....

Once an h.264 library tries to DECODE it...what happens? The DECODING library tries to run the calculations against the "official" parameters that it's programmed with (dating back to 2003 or so) and suddenly hit's these new "Sony" mathematics and says: "What the heck are these calculations?....they are not in my DECODING library and I don't have the SOLUTION for them." And of course, the file is then completely unplayable.

Every calculation in XAVC MUST not include anything that is not in the MPEG h.264 library.

Another way to look at it: We all CODE and DECODE English every day. Here is a sentence I will CODE for everybody:


The brown dog ran around the "bruniwafllop" and then ran up the "gronoklipop" and saw his friend who barked at him "grunubvicaton" and then ran away.


Nobody is not able to DECODE my sentence because I CODED "proprietary" words (only I know) that are not in your translation DECODING library.

In the end, if Sony adds one single calculation that is NOT in MPEG h.264's pre-established library...than their video is simply no longer "h.264" and will not be DECODED by h.264 standards.

You are right about Sony pushing the limits on ENCODING. I don't think anybody does a better job than Sony in real time h.264 ENCODING. I don't know how many or what they exactly are but I know Sony has patented several techniques for doing it. I think they are VBR look-ahead tricks and some important things on real time, two pass filtering. (That's why when you hit "stop" button, the card light is still going for a second or two..it's a Sony write delay for improved VBR CODING)

In the word "CODEC" (Code & DECODE) the two processes must work 100% together and communicate with the same established h.264 library created in 2003. (or something like that)

I was on a global chat/presentation with Hugo Gagioni (Sony CTO) about XAVC. He stated to me that XAVC is 100% h.264 all the way.

Sony is definitely trying to market XAVC as best they can without talking too much about the h.264 part.

Bottom line: "XAVC" uses a sophisticated modern hardware chip set CODING process...that files that MUST be played back with an MPEG h.264 library that was created over 10 years ago.

[Paul Hardy]

Originally Posted by Cliff Totten

Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording. Not even any other Sony camera today used 60Mbp/s. (No,..not even the cheapest entry level Handycams)

The industry has basically chosen 100Mbp/s or higher for even the cheapest 4k consumer models. Even Sony's tiny "Action Cam" records 4k at 100Mbp/s.

This rule might work ok for simple "viewing" purposes. However, in post production and editing, the lower the bitrate, the faster it will break when color correction or gamma bending/stretching is applied.

Color grading is FAR more demanding of a codec than just simple "viewing" of the video.

Again, this is why no other 4k camera in the industry records at 60Mbp/s.

The Sony PXW-X70 is all buy itself at the bottom of the list.

Except the GoPro Hero4 - that's 60mbps @ 4K :-)

[Cliff Totten]

lol,..yes, I stand corrected! ;-)

So the professional market Sony "XDCAM" PXW-X70 sit's at the bottom of the 4k camera industry bit rate list along side it's 60Mbp/s buddy,...the GoPro Hero 4.

Still,...very embarrassing, indeed!

[Mark Watson]

Just get rid of your X70 and find another 4K camera with the more respectable bit rate, problem solved.
Sony said they were going to put 4K in the camera, and they did.
Don't buy a camera without the features you need and hope they will come out with a free firmware upgrade later.

Just sayin'

Mark

[David Heath]

Originally Posted by Cliff Totten

For instance, let's say Sony engineers came up with a new trigonometry calculation that improved edge sharpness 15%. Let's say they also modified the CABAC algorithm and CODED their XAVC with 3 new geometry equations to improve entropy block estimations by 10%. How wonderful!,..they just improved their XAVC/H.264 codec by a significant margin!!!

However,....

Once an h.264 library tries to DECODE it...what happens? The DECODING library tries to run the calculations against the "official" parameters that it's programmed with (dating back to 2003 or so) and ........

I don't disagree with what you're saying in general, but aren't you forgetting about the concept of levels, which I believe go a long way to counter exactly the problem you describe?

So decoders are split into various "levels" - a given level of decoder will decode all levels below it, but not above. See https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

By standardising their H264 implementation relatively late, Sony (with XAVC) have been able to take advantage of the latest tricks, those corresponding to (I believe) level 5.2.

So in your above example, it's quite true if you tried to decode XAVC with a version of a decoder which only supported level up to 4.2 it wouldn't end well. Which is why if you want to use XAVC any NLE etc has to be "XAVC compliant". This is one reason why even if it will deal fine with another H264 variant, it may not handle XAVC or any other level 5.2 encoded H264.

If you like, a level 5.2 capable decoder is a version of your dictionary reprinted with the definitions of "bruniwafllop", "gronoklipop" and "grunubvicaton"! ( :-) )

It's worth then thinking about other forms of encoding based on H264. Can manufacturers not just incorporate the latest "tricks" in their latest products? The problem is that you could then end up with a state of affairs where consumers wouldn't know if their existing equipment could or could not work with a given product.

To take AVC-Intra as example, then I believe it's spec DEFINES it as being based on H264 level 4. If Panasonic was to now bring out a new camera and take advantage of level 5 "tricks" then it would be a recipe for confusion if it was still to be described as AVC-Intra. A broadcaster with a large AVC-Intra based post infra structure would suddenly find that any files produced by the newer camera wouldn't be usable by their existing system. Imagine they employing a freelancer with a newer camera - "is your camera capable of AVC-Intra recording?" - "yes" - ........ "your files appear to be faulty......."

And the same would apply to XAVC if some even newer techniques were to be developed - a "level 6" if you like. If such as Sony wanted to use them, the result wouldn't be XAVC. That's the whole point - "AVC-Intra" or "XAVC" etc defines which level is used, as well as a host of other factors.

Originally Posted by Cliff Totten

Bottom line: "XAVC" uses a sophisticated modern hardware chip set CODING process...that files that MUST be played back with an MPEG h.264 library that was created over 10 years ago.

No, not true. Try decoding XAVC with an original H264 decoder from 2003 and you won't get very far! It has to have the capability of "knowing about" H264 level 5.2, which software from 2003 wouldn't have. (Though the converse would be true - a new version of a decoder should be able to play back a file created with a 2003 encoder.) And all this is before we even start to think about different H264 profiles........

Now just how much difference level 5.2 abilities etc do make I can't quantify. Would it, for example, at least make up for a drop from 100Mbs to 60Mbs, if the 100Mbs material was level 4 compliant? I can't give a direct answer to that, but as regards your comment that "Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording.", then whilst it may be factually accurate, it doesn't really tell the whole story. If those companies are using encoders which comply to a lower level, any 60:100 comparison tells you nothing in isolation. (Other than that 60Mbs will be far more tolerant of media write speed, and give smaller file sizes. :-) )

[Paul Anderegg]

And we al know XAVC of the same flavor is always fully compatible with the various NLE's. :-)

Paul