DV Info Net

DV Info Net (https://www.dvinfo.net/forum/)
-   Sony 4K Ultra HD Handhelds (https://www.dvinfo.net/forum/sony-4k-ultra-hd-handhelds/)
-   -   Sony X70 4K - Lowest bit rate in the industry! (https://www.dvinfo.net/forum/sony-4k-ultra-hd-handhelds/528510-sony-x70-4k-lowest-bit-rate-industry.html)

Mike Griffiths June 4th, 2015 02:30 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
This my first venture into Sony, previously use Panasonic. While the X70 is great, Im totally bemused by the lack of coordination and supreme lack of marketing skills that parts of Sony show. Do they just want to p**s people off or are they just stupid?
Now I've got that ability to load straight into FCPX, I'll live with the idiotic white balance controls and I'll forgo 4K until they come to their senses.
Still a great price for such a highly specced camera. Can't say the same for marketing. (Should we really call it marketing? wouldn't 'screwing up' be a better phrase?)

Clayton Moore June 4th, 2015 08:33 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Andy Wilkinson (Post 1887822)
Yes, Canon and Sony are leaving the middle ground wide open at the moment and that's where JVC with the LS300 and (I think especially) Panasonic with the DVX200 will gain significant sales in the next 12 months.

I was all set to buy either a "cheap" 4K camera (initially I thought the X70, then I leaned towards the JVC HM200) and "dabble with 4K" - or go the whole hog and buy a C300MkII…. but more and more I'm getting close to pre-ordering the DVX200 as it meets so many of my needs for the type of shooting I do. I also feel uncomfortable about paying the high price tag of the C300MkII when it still has a few compromises from my perspective and the 4K camera landscape is changing so very fast.

Although we have yet to see images from it, the DVX200 is likely to offer a decent (enough) step into 4K without breaking the bank. Still wish it did 150Mbps in 4K though, not 100Mbps, although I like the idea of 200Mbps for HD, when needed. 120 fps in HD could be useful for some slomo effects too. The X70 is now off my list with its lowly 60Mbps 4K codec limit.

Sounds like good reasoning to me. The Elephant in The Room question is not weather the C300MkII is a great camera, its about value. Is it worth an "added 10K" over other cameras like the AJA or Black Magic or the JVC etc. TEN THOUSAND MORE .... seriously ??

In your example, the pending DVX200 is not the same kind of camera per se, but its a 4k image with a single sensor and a built-in V-log profile with a gamma curve that mirrors the Panasonic VeriCam 35 for more then 10,000 less then the Canon. You cant afford not to look seriously at it.

Clayton Moore June 4th, 2015 08:38 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Im sure there is a Delete button here somewhere - LOL

David Dixon June 4th, 2015 10:41 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
I took that to mean the upgrade will not be sold *online* and user installed. For some reason in China it will require physically bringing the camera into a service center for the 4K upgrade.

Ricky Sharp June 4th, 2015 12:22 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
In another thread (covering the firmware 2.0 update), it's been stated that one cannot record 4K internally while sending a 4K signal out via HDMI for external recording. I cannot confirm that myself. But if true, that's another strike against the X70's 4K solution.

After reading the 2.0 update guide, there's a decent set of features not available when recording 4K (e.g. S&Q motion)

It really does seem that the unit is being completely "maxed out" when having to deal with 4K footage. Look at all the features one loses if wanting to record at that resolution.

I'm very glad I decided to stick to HD for a few years to come. I'm finding the X70 to be a very capable cam for 1080p especially for its price.

Cliff Totten June 4th, 2015 04:27 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
So really, all of the 1080 features work as normal with the firmware and paid 4K upgrade.

Sony disabled some features to open up CPU headroom room for 4K tasks. I'm perfectly fine with this! If Sony gives the X70 at 4K, 100Mbp/s in the future, I'd GLADLY disable "Face Tracking", IP streaming, WiFi, Android control and a few other features I wont use. If I need that stuff, I'd drop back down to 1080.

Oh,..the 4k to 1080 live down convert on the SDI...if it will save CPU for 100Mbp/s?...disable the SDI completely and I'll be fine. It's only 3 gig anyway and cant do 4K.

What I really care about is having "good" 4k. as both, an AX100 and X70 ownner. There is a HUGE difference between 60Mbp/s and 100Mbp/s when color grading both bit rates.

The PXW-X70 needs 100Mbp/s badly. Sony can leave all the other fun fluffware features untouched in 1080 mode.

Buy the way, the AX100 today will not let you record internally at 4k and HDMI at the same time either. (you get one or the other, not both)

Paul Hardy June 5th, 2015 07:53 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Ok guys - I'm trying to get a handle on genuinely how bad 60mbps actually is from those far more in the know than I.......

Am I correct in assuming that 4K 25p will in theory be a slightly higher bitrate per pixel than the AVCHD 1080/50p at 28mbps (28mbps divide by 2 to account for fps times by 4 = 56mbps)??

Ricky Sharp June 5th, 2015 08:04 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Paul Hardy (Post 1888479)
Ok guys - I'm trying to get a handle on genuinely how bad 60mbps actually is from those far more in the know than I.......

Am I correct in assuming that 4K 25p will in theory be a slightly higher bitrate per pixel than the AVCHD 1080/50p at 28mbps (28mbps divide by 2 to account for fps times by 4 = 56mbps)??

Kind of. The frames per second doesn't change the bit-rate though. But it will change the compression ratio. i.e. at 28 Mbps, 25p footage would be half as compressed as 50p footage.

Edit: Hmm.. I think I see where you were going with your calculation though. Assuming the 28 Mbps 50p footage looks decent, then 60 Mbps 4K 25p footage should not contain any more compression artifacts.

Rough calculations taking into account that both sets of footage would be 8-bit 4:2:0

Your AVCHD 1080/50p @ 28 Mbps would have a compression ratio of 42:1
The UHD/25p @ 60 Mbps should have a compression ratio of 39:1

For the compression ratio, I took the number of bits required to store one second worth of the footage in 8-bit 4:2:0. Then divided by the recording bit rate.

Is the AVCHD footage at 50p really just using 28 Mbps though? I had thought it would be more like 35.


For comparrison sake, 10-bit 4:2:2 XAVC footage seems to have these compression ratios (assuming my math is correct):

25p: 20:1
30p: 24:1
60p: 47:1

Cliff Totten June 5th, 2015 10:56 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
As an extremely rough "rule of thumb" with h.264:

If you have 1080 at 30p encoded at 24Mbps "AVCHD", you get a certain quality. If you multiply that times "4" for UHD resolution, you get "roughly" the same quality equivalent in UHD. (Yes, I know with Long GOP and block size, this is a bit debatable)

So....

100Mbp/s UHD is similar to 25Mbp/s in 1080p

60Mbp/s UHD is similar to 15Mbp/s in 1080p

"Similar to..." simply means the codec's ability to deal with motion and sharpness per pixel blocks. I'm fairly certain the block size and math stays the same with H.264 no mater it's resolution. In other words, h.264 doesn't really get significantly more "efficient" as the frame size gets larger or smaller, (1080 to UHD using the same baseline,Main or high profiles with CABAC enabled )

This is not just "math"...it's also very visually evident when you compare with your eyes...especially on zoom/crops. Color grading the two bit rates is also VERY different as well.

So yeah, UHD at 60Mbp/s is really only 15Mbp/s per 1080 quadrent...very low indeed and this is why No Panasonic, Canon, JVC or any other Sony camera uses 60Mbp/s today!!!! (and with very good reason)

At 60Mbp/s, the PXW-X70 is really sitting all by itself at the BOTTOM of the industry UHD bitrate list. It is using a bitrate that even the cheapest consumer camcorders stay far away from.

Paul Anderegg June 5th, 2015 09:17 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Where can I buy this 4K upgrade? I clicked the Sony link in the firmware 2.0 download section and it took me to Service Plus homepage, can't find where to purchase the damn thing.

I am arranging a demo HM200 with JVC, so would like to put the 2 cameras up against each other possibly.

Paul

David Dixon June 5th, 2015 09:48 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Paul, it's not released yet. This week's free firmware 2.0 will be required for the 4K update which is rumored for June 15. Undocumented fixes in 2.0 include that XAVC-L is finally natively compatible with FCPX, and it offers a fix for a rare problem of the camera locking up when shooting with 2 cards simultaneously.

Paul Anderegg June 6th, 2015 05:59 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Oh, OK, my demo would be for June 30, so let's see how that works out.

Paul

Zenes Petrusin June 7th, 2015 10:45 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Hello,

has some technical answers, i own service manuals of CX900, AX100 and X70. All this camcorders is basicaly indentical inside, but has some little difference. Main difference is CPU, CX900 and AX100 use same cpu called IC6000 MSZ4G4G02C , this is CPU, CAMERA DSP, AV SIGNAL PROCESS, LENS CONTROL, MODE CONTROL, HDMI PROCESS. X70 has this CPU called IC6000 MHZ4G8G11A, internal specification say of this cpu more performance, more features as in CX900, AX100. Sony says about bitrate "We are looking to support a higher bit-rate recording mode than 60 Mbps for 3840x2160 XAVC-L in the future."

I think all is marketing and want sell more ax100, then add more bitrate.

And this is more specification about cmos chip in CX900 AX100 X70, btw. this chip can do 4K video mode (4096H × 2160V, 60 frame/s)

http://www.sony.net/Products/SC-HP/n.../imx183_e.html

Dave Blackhurst June 7th, 2015 03:04 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Since 100Mb/s is already "in" the AX100 (and looks excellent), as well as the "consumer" AX33 and X1000 Actioncam (looks OK, but limited by the tiny sensor, IMO), there really shouldn't be any reason the X70 doesn't get that rate when 4K is added, IMO.

Interesting speculation that Sony "could" squeeze even a bit more out of the processor/sensor combo... UHD, 60P at higher bitrates... hmmmm. Oh to have a "fly on the wall" at the Sony "mad scientist laboratory"!

Cliff Totten June 7th, 2015 03:20 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Dave, +1

I might buy into the fact that the X70 needs a higher CPU than the AX100.

4K Codec aside, it's possible that the X70 truly IS doing more calculations per second than the AX100. I'm sure picture profiles, dual card writing with proxy, WiFi control with streaming encoding all eat more CPU cycles.

In fact, I have both models right here next to me and I have been doing back and forth testing this weekend. The X70 certainly eats Sony VF100 batteries SIGNIFICANTLY faster than the AX100. I have noticed this with three different Sony batteries. I'd guess that the X70 drains batteries 35% (or more) faster when shooting 1080 than the AX100 does in 4K. I was quite surprised by this.

Could this X70's higher processor have more cores or run at higher clock speeds?

I have said this before, if the X70 needs WiFi control, face tracking, proxy and dual card recording and IP streaming shut off to achieve reliable 100Mbp/s 4K?...I think it's absolutely worth it. How often does anybody control their X70 with the mobile app? How many people do IP streaming? How often do you use the cameras codec?...every day you shoot, right?

Without a solid recording codec, you have nothing in any camera.

Paul Hardy June 10th, 2015 11:42 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
So Sony have just released the RX10 mk2 & the RX100 mk4 Pocket camera - both with 100mbps 4k recording & we have to pay £499 for ****ing 60mbps. Seriously now considering selling my X70's & buying some JVC's

Ron Evans June 10th, 2015 01:31 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
The new NX100 uses the older NP760 batteries which are much larger and maybe is a better form factor for the X70. Again not sure the logic of Sony product management/marketing is at the moment. Seems to me a more feature rich version of the NX100 would be a better model than the X70 and therefore not need the X70 in the lineup at all.

Ron Evans

Jody Eldred June 27th, 2015 09:37 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Paul Hardy (Post 1887410)
Don't forget that although 60mbps is low - it doesn't quite equate to 15mbps at Full HD due to the compression algorithms (IIRC the maths from a sony seminar when HD first came out - it would actually be a little higher than 15mbps as an equivalent!)

Actually - No, who am I trying to kid??, it's uselessly low & It should be a minimum of 100mbps.

Grab the torches & pitchforks and we'll all meet up outside the Sony HQ!!


I know the math, but, respectfully, do you know the images? I am shooting 60 Mbps 4K with the X70 and it looks OUTSTANDING. I viewed a lot of my footage on a 4K monitor with a respected colleague and we did not observe said compression artifacting. (I own an F55 and have shot many 4K Raw and S-Log projects, so I have a very good basis for comparison.) The bottom line is not algorithms and equations, but how the images actually LOOK, and is your client and audience happy with them. All else is purely academic.

Here is some of that 4K footage (obviously compressed for YouTube):


https://www.youtube.com/watch?v=vx3t...ature=youtu.be


Jody Eldred June 27th, 2015 09:48 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Cliff Totten (Post 1888508)
As an extremely rough "rule of thumb" with h.264:

If you have 1080 at 30p encoded at 24Mbps "AVCHD", you get a certain quality. If you multiply that times "4" for UHD resolution, you get "roughly" the same quality equivalent in UHD. (Yes, I know with Long GOP and block size, this is a bit debatable)

So....

100Mbp/s UHD is similar to 25Mbp/s in 1080p

60Mbp/s UHD is similar to 15Mbp/s in 1080p

"Similar to..." simply means the codec's ability to deal with motion and sharpness per pixel blocks. I'm fairly certain the block size and math stays the same with H.264 no mater it's resolution. In other words, h.264 doesn't really get significantly more "efficient" as the frame size gets larger or smaller, (1080 to UHD using the same baseline,Main or high profiles with CABAC enabled )

This is not just "math"...it's also very visually evident when you compare with your eyes...especially on zoom/crops. Color grading the two bit rates is also VERY different as well.

So yeah, UHD at 60Mbp/s is really only 15Mbp/s per 1080 quadrent...very low indeed and this is why No Panasonic, Canon, JVC or any other Sony camera uses 60Mbp/s today!!!! (and with very good reason)

At 60Mbp/s, the PXW-X70 is really sitting all by itself at the BOTTOM of the industry UHD bitrate list. It is using a bitrate that even the cheapest consumer camcorders stay far away from.

Math is one thing, but have you SEEN the 4K images? Don't you think that calling out Sony as having a worthless codec and a poor quality camera simply on calculations and not actual experience is perhaps a bit specious?

:-P

I am shooting 60 Mbps 4K with the X70 and it looks OUTSTANDING. I viewed a lot of my footage on a 4K monitor with a respected colleague here in Los Angeles and we did not observe said compression artifacting. (I own an F55 and have shot many 4K Raw and S-Log projects, so I have a very good basis for comparison.) The bottom line is not algorithms and equations, but how the images ultimately LOOK, and is your client and audience happy with them. All else is purely academic.

(I must remind everyone here: this is a $2,000 camera, plus the upgrade. If you want more firepower, it's available in the FS7, the F5, and the F55. But you're going to pay 4x, 8x, 15x as much, plus lenses. The X70 is absolutely amazing, especially for the price.)

Here is some of that 4K footage (obviously compressed for YouTube):


https://www.youtube.com/watch?v=vx3t...ature=youtu.be


David Heath June 27th, 2015 10:40 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Ricky Sharp (Post 1888483)
Rough calculations taking into account that both sets of footage would be 8-bit 4:2:0

Your AVCHD 1080/50p @ 28 Mbps would have a compression ratio of 42:1
The UHD/25p @ 60 Mbps should have a compression ratio of 39:1

For the compression ratio, I took the number of bits required to store one second worth of the footage in 8-bit 4:2:0. Then divided by the recording bit rate.

Is the AVCHD footage at 50p really just using 28 Mbps though? I had thought it would be more like 35.

Sorry, but that argument is missing something vital - namely that going from 25p to 50p will almost certainly mean keeping the GOP-interval constant with respect to TIME - and hence the number of frames per GOP will change.

Numerical compression ratios don't tell the whole story in isolation - raise the number of frames in each GOP, and a higher (numerical) simple compression ratio will give equivalent quality per frame.

So it's likely that the time interval between I-frames will be 1/2 second in each case - which implies 12 frames per GOP for 25p, and 24 for 50p. Since the datarate in the difference frames is small compared to I-frames - and the number of I-frames/sec is 2 in each case - it follows the overall datarate won't need to increase very much between 25p and 50p for equivalent quality. Hence why "only" 28Mbs for 50p AVC-HD.

As for the ratios of HD:4K, then again (for equivalent quality) data rates don't scale up linearly with no of pixels. More pixels means more potential redundancy in the image for the compressor to exploit, so no, 4x as many pixels shouldn't mean 4x the datarate. (You can test the theory in Photoshop - start off with a high res image, then have a few downscaled versions, and compress them all to JPEG with the same quality setting in each case. I think you'll find that the compressed file sizes don't scale linearly with the image sizes.)

Craig Seeman June 27th, 2015 10:54 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
As per David's comments on scaling data rate to frame size, Jan Ozer wrote this interesting article some years back.

"Microsoft’s Ben Waggoner, a respected compressionist, uses the power of .75 rule. Here’s a snippet of an email he sent to me explaining the rule.

Using the old "power of 0.75" rule, content that looks good with 500 Kbps at 640x360 would need (1280x720)/(640x360)^0.75*500=1414 Kbps at 1280x720 to achieve roughly the same quality."

The Essential Key to Producing High Quality Streaming Video

Paul Hardy June 27th, 2015 11:14 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Jody Eldred (Post 1890825)
The bottom line is not algorithms and equations, but how the images ultimately LOOK, and is your client and audience happy with them. All else is purely academic.

That hits the nail on the head - well said!

Bob Searl June 27th, 2015 03:30 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
That's what we all want...great or at least equivalent image quality with small file sizes. Hopefully, Sony will live up to it's design vision and the XAVC long GOP (MXF version) will outperform the rest of the XAVC-S version codecs and at historically low(er) Bit Rates which will directly help keep the file sizes manageable.

They have been describing this vision for two years. The promise of the XAVC chip has all been part of their plan. Better chips, better battery life, and better algorithms have been promised. Let's hope that this is a harbinger of what the X70 can deliver--at 4-2-0 no less. The jury is in deliberation and the verdict is near!

Traditional views of Bit Rates alone no longer tell the story.

Dave Contreras July 1st, 2015 01:52 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Looks like my upgrade license shipped today with an expected arrival date of July 6th! woohoo!

David Dixon July 1st, 2015 03:28 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Mine shipped as well, and I didn't order from B&H until four days ago.

Edit: mine will also arrive Friday, and B&H has already kindly emailed the rebate form.

Bob Searl July 1st, 2015 03:33 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Mine is supposed to arrive on Friday AM from B&H!!!

Cliff Totten July 2nd, 2015 12:40 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Bob Searl (Post 1890856)
That's what we all want...great or at least equivalent image quality with small file sizes. Hopefully, Sony will live up to it's design vision and the XAVC long GOP (MXF version) will outperform the rest of the XAVC-S version codecs and at historically low(er) Bit Rates which will directly help keep the file sizes manageable.

They have been describing this vision for two years. The promise of the XAVC chip has all been part of their plan. Better chips, better battery life, and better algorithms have been promised. Let's hope that this is a harbinger of what the X70 can deliver--at 4-2-0 no less. The jury is in deliberation and the verdict is near!

Traditional views of Bit Rates alone no longer tell the story.

Allot of people believe that XAVC is an actual "codec". However, it's really just using the industry standard MPEG h.264 compression. There is nothing in XAVC- I, S or L that goes beyond the standard h.264 algorithm. Yes, Sony does have some patented VBR encoding tricks and some look ahead optimization. However, it does NOT "exceed" the h.264 standard and does not include any new mathematics that go outside the official h.264 spec. (Something that ALL companies that employ h.264 have full access to...not just Sony)

Sony's XAVC-S, I and L are all mostly folder structure and container specifications. The Sony XAVC standard specifies metadata storage too. But these, of course don't affect the picture quality of the image.

So, to sum up: "XAVC-x" is a "format/container standard" that uses the MPEG h.264 standard as its "codec" of choice.

60Mbp/s is still way too low. This Pro XDCAM camera should NOT be FAR below all other 4k cameras on the market today. It's a shame that even the cheapest industry consumer 4k cameras ALL exceed the X70 with 100Mbp/s.

Cliff Totten July 2nd, 2015 12:49 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Craig Seeman (Post 1890837)
As per David's comments on scaling data rate to frame size, Jan Ozer wrote this interesting article some years back.

"Microsoft’s Ben Waggoner, a respected compressionist, uses the power of .75 rule. Here’s a snippet of an email he sent to me explaining the rule.

Using the old "power of 0.75" rule, content that looks good with 500 Kbps at 640x360 would need (1280x720)/(640x360)^0.75*500=1414 Kbps at 1280x720 to achieve roughly the same quality."

The Essential Key to Producing High Quality Streaming Video

Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording. Not even any other Sony camera today used 60Mbp/s. (No,..not even the cheapest entry level Handycams)

The industry has basically chosen 100Mbp/s or higher for even the cheapest 4k consumer models. Even Sony's tiny "Action Cam" records 4k at 100Mbp/s.

This rule might work ok for simple "viewing" purposes. However, in post production and editing, the lower the bitrate, the faster it will break when color correction or gamma bending/stretching is applied.

Color grading is FAR more demanding of a codec than just simple "viewing" of the video.

Again, this is why no other 4k camera in the industry records at 60Mbp/s.

The Sony PXW-X70 is all buy itself at the bottom of the list.

David Heath July 2nd, 2015 03:15 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Cliff Totten (Post 1891406)
Allot of people believe that XAVC is an actual "codec". However, it's really just using the industry standard MPEG h.264 compression. There is nothing in XAVC- I, S or L that goes beyond the standard h.264 algorithm.

But there is no "standard" MPEG H264 compression. Any standard defines how to decode an H264 signal, it only defines coding in so far it has to meet the decode criteria. Hence XAVC is a subset of H264 - it does define how the signal is to be coded, and all the tricks to be used. The result has to be H264 decoder compliant, yes, but saying "it's XAVC" says so much more than saying "it's H264", and that goes way beyond wrappers and containers.

I like to think of it as baking a cake. To meet consumer standards to call a product "a cake" means it must meet certain criteria (and in the UK, it has big tax implications whether it's a cake or a biscuit, but I digress..... :-) ) But there are many, many different ways to bake a cake, and many different ingredients which may or may not be used. Think of H264 as "a cake", and XAVC as "a Victoria sponge".

If you want an illustration, just look at https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels and the table labelled "Software encoder feature comparison". It compares various software encoders - which will all produce an H264 compliant file - but they don't all use anything like all the possibilities that the standard is capable of, and consequently results will vary widely at the same bitrate. That's why you cannot talk of "the standard h.264 algorithm" in relation to a CODER.

It's like going into a shop and asking for "a cake". Do it a few times and you could come out with widely differing things - and some a lot better than others!!
Quote:

Originally Posted by Cliff Totten (Post 1891406)
Yes, Sony does have some patented VBR encoding tricks and some look ahead optimization. However, it does NOT "exceed" the h.264 standard and does not include any new mathematics that go outside the official h.264 spec. (Something that ALL companies that employ h.264 have full access to...not just Sony)

Other companies may have access to such mathematics and "tricks" - but that's not to say their products use them. That may be for a variety of reasons. If a defined subset of H264 (as XAVC is now) it's spec becomes fixed for a variety of reasons, or it may be that a given piece of equipment has a coder whose performance is limited by power, technology or whatever. The advantage Sony have with XAVC is that it's relatively recent, hence have been able to fix the latest levels and "tricks" in the spec.

Same with the varieties of XDCAM - they were MPEG2 at heart, but saying "MPEG2" only told you half the story. "XDCAM" defined a lot more. (Compare the relevant performance of HDV and XDCAM as codecs, for example.)

As for "is 60Mbs enough?", then all I'll say is that 60Mbs of XAVC is highly likely to be better than 60Mbs of other forms of (more basic!) H264 encoding! From a manufacturers point of view, then whilst one side may want to up the bitrate for quality reasons, another side may argue for keeping it down for reasons of cost (filesizes, speed of memory needed etc). Which is why you have to consider the market any product is in, it's cost, and what it's target customers are most likely to go with in terms of cost/performance.

Cliff Totten July 2nd, 2015 04:56 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
David, I suspect you already know most of this but I will say it for the others reading who might not know.....

You are certainly right that CODING h.264 is different than DECODING it. And yes, manufacturers are free "play" with the CODING aspect all they want....with the limitation that whatever they CODE must be DECODED by a standard h.264 library.(circa 2003 or around there)

For instance, let's say Sony engineers came up with a new trigonometry calculation that improved edge sharpness 15%. Let's say they also modified the CABAC algorithm and CODED their XAVC with 3 new geometry equations to improve entropy block estimations by 10%. How wonderful!,..they just improved their XAVC/H.264 codec by a significant margin!!!

However,....

Once an h.264 library tries to DECODE it...what happens? The DECODING library tries to run the calculations against the "official" parameters that it's programmed with (dating back to 2003 or so) and suddenly hit's these new "Sony" mathematics and says: "What the heck are these calculations?....they are not in my DECODING library and I don't have the SOLUTION for them." And of course, the file is then completely unplayable.

Every calculation in XAVC MUST not include anything that is not in the MPEG h.264 library.

Another way to look at it: We all CODE and DECODE English every day. Here is a sentence I will CODE for everybody:


The brown dog ran around the "bruniwafllop" and then ran up the "gronoklipop" and saw his friend who barked at him "grunubvicaton" and then ran away.


Nobody is not able to DECODE my sentence because I CODED "proprietary" words (only I know) that are not in your translation DECODING library.

In the end, if Sony adds one single calculation that is NOT in MPEG h.264's pre-established library...than their video is simply no longer "h.264" and will not be DECODED by h.264 standards.

You are right about Sony pushing the limits on ENCODING. I don't think anybody does a better job than Sony in real time h.264 ENCODING. I don't know how many or what they exactly are but I know Sony has patented several techniques for doing it. I think they are VBR look-ahead tricks and some important things on real time, two pass filtering. (That's why when you hit "stop" button, the card light is still going for a second or two..it's a Sony write delay for improved VBR CODING)

In the word "CODEC" (Code & DECODE) the two processes must work 100% together and communicate with the same established h.264 library created in 2003. (or something like that)

I was on a global chat/presentation with Hugo Gagioni (Sony CTO) about XAVC. He stated to me that XAVC is 100% h.264 all the way.

Sony is definitely trying to market XAVC as best they can without talking too much about the h.264 part.

Bottom line: "XAVC" uses a sophisticated modern hardware chip set CODING process...that files that MUST be played back with an MPEG h.264 library that was created over 10 years ago.

Paul Hardy July 2nd, 2015 11:24 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Cliff Totten (Post 1891408)
Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording. Not even any other Sony camera today used 60Mbp/s. (No,..not even the cheapest entry level Handycams)

The industry has basically chosen 100Mbp/s or higher for even the cheapest 4k consumer models. Even Sony's tiny "Action Cam" records 4k at 100Mbp/s.

This rule might work ok for simple "viewing" purposes. However, in post production and editing, the lower the bitrate, the faster it will break when color correction or gamma bending/stretching is applied.

Color grading is FAR more demanding of a codec than just simple "viewing" of the video.

Again, this is why no other 4k camera in the industry records at 60Mbp/s.

The Sony PXW-X70 is all buy itself at the bottom of the list.

Except the GoPro Hero4 - that's 60mbps @ 4K :-)

Cliff Totten July 2nd, 2015 11:47 PM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
lol,..yes, I stand corrected! ;-)

So the professional market Sony "XDCAM" PXW-X70 sit's at the bottom of the 4k camera industry bit rate list along side it's 60Mbp/s buddy,...the GoPro Hero 4.

Still,...very embarrassing, indeed!

Mark Watson July 3rd, 2015 01:42 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Just get rid of your X70 and find another 4K camera with the more respectable bit rate, problem solved.
Sony said they were going to put 4K in the camera, and they did.
Don't buy a camera without the features you need and hope they will come out with a free firmware upgrade later.

Just sayin'

Mark

David Heath July 3rd, 2015 05:30 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Cliff Totten (Post 1891426)
For instance, let's say Sony engineers came up with a new trigonometry calculation that improved edge sharpness 15%. Let's say they also modified the CABAC algorithm and CODED their XAVC with 3 new geometry equations to improve entropy block estimations by 10%. How wonderful!,..they just improved their XAVC/H.264 codec by a significant margin!!!

However,....

Once an h.264 library tries to DECODE it...what happens? The DECODING library tries to run the calculations against the "official" parameters that it's programmed with (dating back to 2003 or so) and ........

I don't disagree with what you're saying in general, but aren't you forgetting about the concept of levels, which I believe go a long way to counter exactly the problem you describe?

So decoders are split into various "levels" - a given level of decoder will decode all levels below it, but not above. See https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

By standardising their H264 implementation relatively late, Sony (with XAVC) have been able to take advantage of the latest tricks, those corresponding to (I believe) level 5.2.

So in your above example, it's quite true if you tried to decode XAVC with a version of a decoder which only supported level up to 4.2 it wouldn't end well. Which is why if you want to use XAVC any NLE etc has to be "XAVC compliant". This is one reason why even if it will deal fine with another H264 variant, it may not handle XAVC or any other level 5.2 encoded H264.

If you like, a level 5.2 capable decoder is a version of your dictionary reprinted with the definitions of "bruniwafllop", "gronoklipop" and "grunubvicaton"! ( :-) )

It's worth then thinking about other forms of encoding based on H264. Can manufacturers not just incorporate the latest "tricks" in their latest products? The problem is that you could then end up with a state of affairs where consumers wouldn't know if their existing equipment could or could not work with a given product.

To take AVC-Intra as example, then I believe it's spec DEFINES it as being based on H264 level 4. If Panasonic was to now bring out a new camera and take advantage of level 5 "tricks" then it would be a recipe for confusion if it was still to be described as AVC-Intra. A broadcaster with a large AVC-Intra based post infra structure would suddenly find that any files produced by the newer camera wouldn't be usable by their existing system. Imagine they employing a freelancer with a newer camera - "is your camera capable of AVC-Intra recording?" - "yes" - ........ "your files appear to be faulty......."

And the same would apply to XAVC if some even newer techniques were to be developed - a "level 6" if you like. If such as Sony wanted to use them, the result wouldn't be XAVC. That's the whole point - "AVC-Intra" or "XAVC" etc defines which level is used, as well as a host of other factors.
Quote:

Originally Posted by Cliff Totten (Post 1891426)
Bottom line: "XAVC" uses a sophisticated modern hardware chip set CODING process...that files that MUST be played back with an MPEG h.264 library that was created over 10 years ago.

No, not true. Try decoding XAVC with an original H264 decoder from 2003 and you won't get very far! It has to have the capability of "knowing about" H264 level 5.2, which software from 2003 wouldn't have. (Though the converse would be true - a new version of a decoder should be able to play back a file created with a 2003 encoder.) And all this is before we even start to think about different H264 profiles........

Now just how much difference level 5.2 abilities etc do make I can't quantify. Would it, for example, at least make up for a drop from 100Mbs to 60Mbs, if the 100Mbs material was level 4 compliant? I can't give a direct answer to that, but as regards your comment that "Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording.", then whilst it may be factually accurate, it doesn't really tell the whole story. If those companies are using encoders which comply to a lower level, any 60:100 comparison tells you nothing in isolation. (Other than that 60Mbs will be far more tolerant of media write speed, and give smaller file sizes. :-) )

Paul Anderegg July 3rd, 2015 05:46 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
And we al know XAVC of the same flavor is always fully compatible with the various NLE's. :-)

Paul

David Heath July 3rd, 2015 06:47 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Paul Anderegg (Post 1891471)
And we al know XAVC of the same flavor is always fully compatible with the various NLE's. :-)

Paul

But it may be for the reasons I say, it will rely on any NLE having the level 5.2 H264 ability for decoding amongst other things. And may need a given standard of hardware as well.

If a given NLE version states it's "XAVC compliant", then given adequate hardware that should be that. But no one should expect XAVC to work with earlier versions of the software.

Unfortunately it's a fact of life of going for " the latest and best" whether it be 4K, a new codec or whatever. The price you have to pay is a period of waiting for other factors to catch up.

And it reinforces what I said above about why a manufacturer can't tweak the spec of any existing codec like AVC-Intra or XAVC once finalised. It may be bad enough if you are running a system based on AVC-Intra to find it won't run XAVC - but just imagine the confusion if it would run AVC-Intra files from some cameras but not other (newer) ones.

Paul Anderegg July 3rd, 2015 06:56 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
I meant Sony just "modified" the XAVC-L in the X70 to make it work......obviously, people are messing with the internals of the levels, and ARE in fact messing up compatibility.

Paul

Cliff Totten July 3rd, 2015 08:16 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by David Heath (Post 1891468)
I don't disagree with what you're saying in general, but aren't you forgetting about the concept of levels, which I believe go a long way to counter exactly the problem you describe?

So decoders are split into various "levels" - a given level of decoder will decode all levels below it, but not above. See https://en.wikipedia.org/wiki/H.264/MPEG-4_AVC#Levels

By standardising their H264 implementation relatively late, Sony (with XAVC) have been able to take advantage of the latest tricks, those corresponding to (I believe) level 5.2.

So in your above example, it's quite true if you tried to decode XAVC with a version of a decoder which only supported level up to 4.2 it wouldn't end well. Which is why if you want to use XAVC any NLE etc has to be "XAVC compliant". This is one reason why even if it will deal fine with another H264 variant, it may not handle XAVC or any other level 5.2 encoded H264.

If you like, a level 5.2 capable decoder is a version of your dictionary reprinted with the definitions of "bruniwafllop", "gronoklipop" and "grunubvicaton"! ( :-) )

It's worth then thinking about other forms of encoding based on H264. Can manufacturers not just incorporate the latest "tricks" in their latest products? The problem is that you could then end up with a state of affairs where consumers wouldn't know if their existing equipment could or could not work with a given product.

To take AVC-Intra as example, then I believe it's spec DEFINES it as being based on H264 level 4. If Panasonic was to now bring out a new camera and take advantage of level 5 "tricks" then it would be a recipe for confusion if it was still to be described as AVC-Intra. A broadcaster with a large AVC-Intra based post infra structure would suddenly find that any files produced by the newer camera wouldn't be usable by their existing system. Imagine they employing a freelancer with a newer camera - "is your camera capable of AVC-Intra recording?" - "yes" - ........ "your files appear to be faulty......."

And the same would apply to XAVC if some even newer techniques were to be developed - a "level 6" if you like. If such as Sony wanted to use them, the result wouldn't be XAVC. That's the whole point - "AVC-Intra" or "XAVC" etc defines which level is used, as well as a host of other factors.

No, not true. Try decoding XAVC with an original H264 decoder from 2003 and you won't get very far! It has to have the capability of "knowing about" H264 level 5.2, which software from 2003 wouldn't have. (Though the converse would be true - a new version of a decoder should be able to play back a file created with a 2003 encoder.) And all this is before we even start to think about different H264 profiles........

Now just how much difference level 5.2 abilities etc do make I can't quantify. Would it, for example, at least make up for a drop from 100Mbs to 60Mbs, if the 100Mbs material was level 4 compliant? I can't give a direct answer to that, but as regards your comment that "Yet, it's a statistical fast that no other company, Panasonic, Canon or JVC use 60Mbp/s for 4k recording.", then whilst it may be factually accurate, it doesn't really tell the whole story. If those companies are using encoders which comply to a lower level, any 60:100 comparison tells you nothing in isolation. (Other than that 60Mbs will be far more tolerant of media write speed, and give smaller file sizes. :-) )

I think you might be accidentally interchanging "profiles" with "levels".

"Levels" determine the restrictions for frame rate, frame size, chroma sub-sampling, and overall bit rate and stuff like that.

"Profiles" determine the amount of MPEG "tools" (calculation types) that the video is allowed to have from the established MPEG tool list. This is where the actual analysis and compression is done.

Many of the changes that have happened over recent years involve raising the "level " constraints. Each level is less and less "restricted", the higher it goes. A level 5.1 video does NOT mean that it's more "complex" than a level 5.0 video. It just means that it's less "constrained" and is allowed to exist in larger parameters if necessary. Larger frame sizes and faster frame rates means more calculations per second and certainly requires more CPU horsepower and data throughput.

Theoretically, MPEG could release a new level "6.0". This level could allow for 8k frame sizes with a new frame rate at 240p. with 2 gigabit maximum bit rate. The core tools in the chosen the MPEG "profile" are then simply scaled proportionally higher into the new "level" parameters. (Same calculations, just more of them per second) The h.264 CODEC tool sets aren't exactly "changing" it's just being scaled to a larger, higher level. Yes, "6.0" will prolly never happen, (because HEVC is taking it over from here on in). I'm just saying for the sake of discussion.

Changing "levels" is less difficult than changing "profiles". The heart of the CODEC is really in the "profiles" because that is where the meat and potatoes of the mathematics live. The "levels" are mostly just "scaling" the mathematics that is being done to fit into in the proper "level" restrictions.

Think of "levels" increases as a mathematical "restriction" or limit that has been raised. It allows the "profile" math to function inside larger parameters.

Again, the majority of the compression tool sets (all those crazy mathematics that do the magic...or "profiles") was done 10+ years ago. Throughout all the years, the "levels" have been raised higher and higher to allow those "profiles" and tools to exist and operate in bigger spaces. (higher levels) The changes in the past 10 years have mostly been minor. Things like frame packing 3D and multiview and lot's of size and bit rate increases.

Interesting enough,..h.264 is being replaced with h.265 (HEVC). H.265 is built directly on top of h.264. It contains many of h.264's existing tools and math but expands them much further and adds in new tools and even higher new levels limitations. MPEG could never have called this h.264 - Profile @"Super-Ultra-High" and Level 6.0. Why? Because it "breaks" the original h.264 tool and profile set. It not only exceeds 5.2 level limitations (no big deal) but it also "alters" the original CODEC profile math. Once you do that, you have just created a new codec.

Sorry if I stated the exact same thing in 3 different ways. I tend to be redundant sometimes. ;-)

All of this is a wonderful discussion but it doesn't change the fact that the Sony PXW-X70 "XDCAM" 4k/UHD camcorder sit's at the bottom of the industry list (along side the GoPro 4) in terms of bit rate. ;-)

60Mbp/s is just WAY too low. Sony, you are better than this. Show some respect for yourself and please fix this.....

Cliff Totten July 3rd, 2015 10:13 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Quote:

Originally Posted by Paul Anderegg (Post 1891478)
I meant Sony just "modified" the XAVC-L in the X70 to make it work......obviously, people are messing with the internals of the levels, and ARE in fact messing up compatibility.

Paul

Interesting about the X70. If you inspect the original X70 firmware video files that it created...the METADATA was literally missing!!! It was "blank" and completely empty! No media flags at all.

Some NLE's require and read that metadata before you can decode it.Sony Vegas is one. If you completely strip the metadata from a video file, it cant play it.

Sony fixed this in the last X70 camera firmware. Now the files have the proper metadata included...total screw up on Sony's part. And, the fact that it too Sony 6 months to fix it?...wow, that is sad.

Remember, unpacking the file wrapper is an important part of playback. Unpacking .mp4 and playing the video is different than unpacking it in .mxf. (even though the h.264 codec inside is completely the same)

So yeah,..no h.264 codec problem there. It was totally an .MXF container screw up.

CT

Christopher Young July 4th, 2015 09:27 AM

Re: Sony X70 4K - Lowest bit rate in the industry!
 
Cliff

Sorry to correct you on this. You are not far off but I knew what the issue was back in early April but because of confidentiality requests from Sony I couldn't at the time mention what the problem was. I am frequently in a feedback loop with SCS on the software side. This stems from years back having been involved on their Beta test team. Now that the issue of X70 footage being accepted by most NLEs is resolved I feel I can quote verbatim information from a heads up I received in early April.

-------------------------

"Update: We found out that the camera team was aware of this and that it is a problem with the SMDK (development kit). They are building a new release that among other things will: "Remove assertion failure when parsing invalid descriptive metadata."

Which is what we were experiencing – invalid metadata from the older SMDK. Once we get that it will have to be implemented and then go through QA again to make sure there are no regressions. So, not a quick turn-around but hopefully this update will fix the problem.

-------------------------

So the metadata wasn't 'blank' its just an unfortunate fact that in the case of the X70 its development was done with an older version of the SMDK and some of the metadata involved with that meant that the later software developments would see this data as invalid. This causing the importation of the X70 files to be aborted.

I'll also add that David is very much on track with his comments about the later implementation of levels such as 5.2. and the compliance issues surrounding the use of these levels across various platforms and software. In consumer equipment where manufactures don't have to worry too much about cross platform, equipment and software compatibility manufactures have a freedom to introduce changes fairly quickly. Changes that cannot happen as quickly when SMPTE ratified standards such as AVC-Intra and XAVC are involved. All changes to ratified SMPTE standards have to be tested, accepted and ratified again by SMPTE before they can be brought to the broadcast market. Well that's generally the case if the manufacture wants their equipment to be accepted as 'meeting' the standard.

As David alluded to this ensures that there is full compatibility across all equipment and software platforms. This of course all takes time and is one of the reasons that changes and development in cameras such as the X70, the baby of the broadcast family, will lag behind their cousins that run in that juggernaut that is the retail consumer camera market which is driven by very competitive 'one jump ahead of the competition' marketing.

Chris Young
CYV Productions
Sydney


All times are GMT -6. The time now is 03:11 AM.

DV Info Net -- Real Names, Real People, Real Info!
1998-2025 The Digital Video Information Network