AVCHD video at higher bitrates - Page 3 at DVinfo.net

Go Back   DV Info Net > High Definition Video Acquisition > AVCHD Format Discussion

AVCHD Format Discussion
Inexpensive High Definition H.264 encoding to DVD, Hard Disc or SD Card.


Reply
 
Thread Tools Search this Thread
Old December 5th, 2010, 05:46 AM   #31
Regular Crew
 
Join Date: Sep 2008
Location: Beersheba, Israel
Posts: 111
David:

Thank you for staying in this discussion so long… Now back to your arguments.

1. I was disappointed in you when you said that “If the processor can't cope it will stutter or drop frames - not just introduce a few more artifacts”. No David, it is not that simple. Decompressing on the fly (i.e. while replay) is much more complicated process. In the case of near 100 percent overload, yes, frame dropping and stuttering may take place, but what if the load is less than 100%?

2. To question “What basis do you have for saying that higher bitrate AVC will need less processor power than lower bitrate AVC?” I say it based on the compressing rate: the higher bitrate, the less the compression rate of the media.

3. When you argued about percentage of lossless and lossy algorithms within AVC compression design, you unfortunately made me disappointed even more: to say that AVC has a lot of lossy compressions is one hell of an exaggeration.

4. You were right when expressed doubt about whether the difference between two fragments were real and not introduced by statistical noise. So, I have prepared another pair of the fragments cut from an original footage (AVCHD, 24 Mbit/s) and the rendered clip (VC-1 format, maximum bitrate 82 Mbit/s).

Look closely at the window frames of the mosque and the sky near the lamppost. As you can see, the rendered at 80 MBps movie has less artifacts and more subtle details.
Attached Thumbnails
AVCHD video at higher bitrates-pic-1-quality-vbr-.jpg   AVCHD video at higher bitrates-frame-1.jpg  

AVCHD video at higher bitrates-frame-2.jpg  
Arkady Bolotin is offline   Reply With Quote
Old December 5th, 2010, 08:05 AM   #32
Inner Circle
 
Join Date: Mar 2003
Location: Ottawa, Ontario, Canada
Posts: 3,942
It seems to me that the main difference is brought out by the way the stills are extracted. Both streams are long GOP but trans coding to a higher bit rate will yield more information each frame for the still extraction to take. There are big differences between how my various NLE's extract stills with Vegas being at the bottom of the list. I don't have VLC to compare. PMB does the best which has a specific feature for still extraction which may well use a different approach.

The video playback may well be the same unless the transcoding employed some upscaling like the Toshiba Super scaling used to upscale SD to HD which produces similar results to the stills shown. I have Cineform and also use Canopus HQ neither of these shows any difference on video playback to the original AVCHD.

Ron Evans.
Ron Evans is offline   Reply With Quote
Old December 5th, 2010, 08:49 AM   #33
Regular Crew
 
Join Date: Sep 2008
Location: Beersheba, Israel
Posts: 111
That is a valuable point, Ron. Really, a process of still extraction may significantly contribute to appeared difference.

My answer is this. Since I use the same software (VLC media player) to extract and play back the clips, the same mechanism, which might yield more information during still extraction from the clips, should also be in play during the clip replay. Thus, whatever the reason, with the increase of bitrate we would have the improved footage.

Regarding VLC itself, I can say it has a plethora of internal codecs, including H.264/MPEG4 AVC encoder (x264) that is far advanced that Windows 7 Media Player has.
Arkady Bolotin is offline   Reply With Quote
Old December 5th, 2010, 01:40 PM   #34
Inner Circle
 
Join Date: Feb 2007
Location: Apple Valley CA
Posts: 4,866
The experience I had with burning BR to a regular DVD was that if you exceed the bitrate that the media/hardware was capable of playing back at, it would TRY, but then slowly grind to a halt, then just give up. The BR templates in Vegas have two "default" bitrates, 25 and 8 Mbps - to get best results, I shot for what I understand is the maximum bitrate for a regular "red" DVD disk, which is 18Mbps.

What is important to note is that a disk burned at 8Mbps looked like poo, may as well have been SD - it suffered from significant "lost" information in order to "fit" the 8Mbps stream. A 25Mbps disk would attempt playback and look good, only to "choke", but the "optimized" bitrate will both play back and look quite good (the source AVCHD was from Sony consumer cams at 17Mbps max). Once the bitrate was lowered to within the specs, playback was fine and smooth, without trouble.

If you have a storage media capable of higher bitrates, along with playback equipment capable of that bitrate, per my earlier postings, you then can reduce the needed compression, and in theory increase the amount of detail and the smoothness of the individual frames.

I think the question has been "is it possible to 'restore/extract' information encoded in the lower bitrate data stream upon decompression if when you recompress you use a HIGHER bitrate"? Arkady has experimented, and it would seem possible. I suspect that the "math" would under some circumstances make this possible, depending on the efficiency and accuracy at many points along the chain. AVCHD seems to be pretty robust, and the manufacturers are definitely squeezing better and better image quality out of it...

From a practical standpoint, I'm pretty sure you won't see spec changes until the hardware is fully capable of RELIABLE playback. Simple reason for this is if you release a spec for a CODEC/format, but no one can play it back reliably, you've got trouble. The 1080/60p from the latest 700 series consumer Panasonics is an example - looks amazing, but it's taken a while for people to figure out how to process it!

From a "practical" standpoint, I'd think entities like law enforcement, advanced imaging sciences, and espionage would likely already be aware of the possibilities, and have the $$$ to spend on such "enhancement" capabilities. The increase in detail and smoothness would probably not even be noticed by 99.99% of the viewing population, but we'll see it one of these days as the technology trickles down.
Dave Blackhurst is offline   Reply With Quote
Old December 5th, 2010, 03:10 PM   #35
Inner Circle
 
Join Date: Feb 2007
Location: Apple Valley CA
Posts: 4,866
A spanner in the works (wrench in the gears for us in the US)

It suddenly occurred to me that there's a "variable" here that will drastically affect the possible improvements...

MOVEMENT - we all are aware that motion is a problem, and that Codecs tend to "break down" when there's a lot of fine motion in the frame.

Arkady - your samples look like they are from frames with minimal movement, where the compression is LEAST likely to have to discard data. In order to squeeze in the added data from movement between frames, the CODEC has to toss increasingly large amounts of information.

IOW, if you have a relatively static shot (or a series of frames) with little movement (a "still" basically), the codec can almost compress 100% - the repetitive data comprises a large component of the adjacent frames... I'd expect in this case the probability of restoring "lost" detail might be pretty high.

Now, lets take a series of frames with say 25% non-redundant information (I'm just tossing out random percentages, as I don't know where breakdown occurs) - the codec is still able to compress relatively well, maintaining the bitrate AND the level of detail, but by nature, more will be "lost" because the non-repetitive information must take up more "space" or bits.

Bump that to maybe 50% moving/non redundant, I'm guessing the bitrate is maxing out, and beginning to strain to retain details...

Let's say at 75%,the CODEC is now past the point of diminishing returns, and is now tossing data just to stay under the threshold "max bitrate", you start to get "noise", "distortion", etc.

if 100% of the frame is moving/non-redundant - (fast pans, bouncy handheld footage), you've got a big hairy MESS, as the CODEC is trying to empty a bathtub with a teaspoon... the "data" overwhelms the ability to compress it in real time!

I'm going to postulate that you wouldn't be able to "recover" ANY additional detail as the motion component of footage increasingly "stresses" the compression algorithm - there's probably a "curve" that could be plotted as to how much "overhead" the Codec has at a given percentage of movement in the frame, and at some point you literally "hit the ceiling/wall" as the motion increases, and at some point the image quality begins to degrade beyond acceptable.


Just thought this was worth adding to the mix, moral being, don't whip pan, and be conscious of what motion in the frame is doing, and oh yeah, keep in mind shallow DoF "helps" as blurry, non-detailed backgrounds require less data than highly detailed, infinite DoF shots...
Dave Blackhurst is offline   Reply With Quote
Old December 5th, 2010, 03:54 PM   #36
Regular Crew
 
Join Date: Sep 2008
Location: Beersheba, Israel
Posts: 111
Dave, your last comments were right on the nail! Even I myself if tried to wrap up the moral of this discussion, hardly would do better.

Yes, it’s absolutely correct that the H.264/AVC video format (used by AVCHD) is capable of nearly lossless coding. In practical terms it means, that under right circumstances we can expect the exact (or almost exact) restoration of the compressed data.

Therefore, it makes possible to decrease the compression rate of the recoded AVCHD video (by rendering it into some particular format) before and play it back afterward.

But, the problem here, as Dave elegantly put, is reliable media for the playback of such high-bitrate video. Ask yourself how you can run 80-Megabit-per-second video and I am sure the only answer you would find right now is a pretty fast hard drive coupled with a powerful processor. Zero distribution options, no archiving.

So, until that wonderful moment when industry will come with a commercially profitable solution how to playback high-bitrate video, it will stay only in our computers.

P.S. Just saw you last addition. Unfortunately, I have no more time to write, but, in general, yes, you have guessed right: I used a tripod when I filmed these scenes (and no panning, no zooming).
Arkady Bolotin is offline   Reply With Quote
Old December 5th, 2010, 04:20 PM   #37
Inner Circle
 
Join Date: Jan 2006
Posts: 2,699
Quote:
Originally Posted by Arkady Bolotin View Post
3. When you argued about percentage of lossless and lossy algorithms within AVC compression design, you unfortunately made me disappointed even more: to say that AVC has a lot of lossy compressions is one hell of an exaggeration.
But that's not what I said. Look back and you'll see the words are "It may include some lossless algorithms, but it includes a lot of lossy ones as well!" - the spec has lossy algorithms as well as lossless ones. How lossy the compression itself is depends on factors such as bitrate - if it's very high there'll be little loss, if low (as when AVC is used for web video) the losses will be very large. What you can't say is "by its results the H.264/AVC compression can be classed as lossless one".

Look, the latest pictures you've posted show visible artifacts - thats the whole reason you've linked to them. Surely that itself is proof that H.264/AVC is lossy?
Quote:
Originally Posted by Arkady Bolotin View Post
4. You were right when expressed doubt about whether the difference between two fragments were real and not introduced by statistical noise. So, I have prepared another pair of the fragments cut from an original footage (AVCHD, 24 Mbit/s) and the rendered clip (VC-1 format, maximum bitrate 82 Mbit/s).

Look closely at the window frames of the mosque and the sky near the lamppost. As you can see, the rendered at 80 MBps movie has less artifacts and more subtle details.
If they aren't the same frame, original and recompressed, there is really little point in comparing them. Any differences seem pretty slight anyway.
Quote:
Originally Posted by Dave Blackhurst
I think the question has been "is it possible to 'restore/extract' information encoded in the lower bitrate data stream upon decompression if when you recompress you use a HIGHER bitrate"? Arkady has experimented, and it would seem possible. I suspect that the "math" would under some circumstances make this possible, depending on the efficiency and accuracy at many points along the chain. AVCHD seems to be pretty robust, and the manufacturers are definitely squeezing better and better image quality out of it...
They are squeezing better and better out of it by improvements to the CODERS. Making them more and more complex, and making use of more of the defined toolkit. But the DECODERS have to be more or less defined. Any transcode means decoding to uncompressed, then recompression - which inevitably must mean a further loss. Once the original coder has lost something, that's it. The act of inital decompression must get you the best that's possible - anything else you then do to it can only preserve it or make it worse.
Quote:
Originally Posted by Arkady Bolotin View Post
Yes, it’s absolutely correct that the H.264/AVC video format (used by AVCHD) is capable of nearly lossless coding. In practical terms it means, that under right circumstances we can expect the exact (or almost exact) restoration of the compressed data.
No, that's not correct. AVC-HD may give good results under normal usage, but it's nowhere near lossless.

Look, if that argument was correct (that AVC-HD gave nearly lossless coding) then how can you gain anything anyway by reencoding!?! If AVC-HD is lossless, why not just stick with this "lossless" solution? How can you improve on the original if it's as you say?

It's human nature to want to discover a quick magic fix, but here all I can see is extreme optimism. Sorry.
David Heath is offline   Reply With Quote
Old December 5th, 2010, 06:14 PM   #38
Inner Circle
 
Join Date: Mar 2003
Location: Ottawa, Ontario, Canada
Posts: 3,942
Quote:
Originally Posted by Arkady Bolotin View Post
That is a valuable point, Ron. Really, a process of still extraction may significantly contribute to appeared difference.

My answer is this. Since I use the same software (VLC media player) to extract and play back the clips, the same mechanism, which might yield more information during still extraction from the clips, should also be in play during the clip replay. Thus, whatever the reason, with the increase of bitrate we would have the improved footage.

.
The still extraction mechanism may be very different from the realtime playback mechansim as is the case with Sony MBS and some of the in camera still functions in AVCHD cameras. Still ectraction has the opportunity to analyse over many frames and potentially increase resolution based on whether there is any or slight movement in the image.

Ron Evans
Ron Evans is offline   Reply With Quote
Old December 6th, 2010, 01:57 AM   #39
Trustee
 
Join Date: Mar 2005
Location: Coronado Island
Posts: 1,452
Arkady
Regarding the two mosque shots:
To my eye, the original shot (24mbs) looks better.
The 80mbs image has a little less contrast and that may give the appearance of more detail in the darks, but the same detail is in the 24mbs shot as well if you look closely.
If you decompress AVCHD and recompress again to AVCHD, even at a higher data rate, you will loose additional data.
Unfortunately, that's the bottom line.
__________________
Bob
Robert Young is offline   Reply With Quote
Old December 6th, 2010, 06:31 AM   #40
Regular Crew
 
Join Date: Sep 2008
Location: Beersheba, Israel
Posts: 111
My initial intention was to avoid (by all means) a technical lecture on AVCHD and video compression essentials. Besides, this thread is rather over.

However, after the last comments by David and Robert, I see that such lecture is necessary (or at least it would be helpful).

Therefore, people, prepare for the hard stuff. If someone has already felt a jolt of boredom, sorry, you can skip this and consider the discussion closed.


H.264/AVC compression (used by AVCHD) was developed over a period of about four years. The roots of this standard lie in the ITU-T’s H.26L project initiated by the Video Coding Experts Group (VCEG).

The H.264/AVC has been developed to address primarily the following needs:
1. Use more than 8 bits per sample of source video accuracy.
2. Use higher resolution for color (i.e., to use 4:2:2 or 4:4:4).
3. Use very high bit rates.
4. Use very high resolution.
5. Achieve very high fidelity – even representing some parts of the video losslessly.

The main principle of video compression applied in H.264/AVC is this. Each picture is compressed by partitioning it as one or more slices; each slice consists of macroblocks, which are blocks of 16x16 luma samples with corresponding chroma samples. Each macroblock is also divided into sub-macroblock partitions for motion-compensated prediction. The prediction partitions can have seven different sizes – 16x16, 16x8, 8x16, 8x8, 8x4, 4x8 and 4x4.

The hierarchy of a video sequence, from sequence to samples is given by:

sequence(pictures(slices(macroblocks(macroblock partitions(sub-macroblock partitions(blocks(samples)))))).

Thus, the basic unit of the encoding or decoding process is the macroblock.

Slices in a picture are compressed by using the following coding tools:
1. "Intra" spatial (block based) prediction
2. "Inter" temporal prediction – block based motion estimation and compensation
3. Interlaced coding features
4. Picture Adaptive Frame Field
5. MacroBlock Adaptive Frame Field
6. Lossless representation capability
7. 8x8 or 4x4 integer inverse transform
8. Residual color transform for efficient RGB coding
9. Scalar quantization

However, the main tool is Lossless Entropy coding. Entropy coding is the coding technique that replaces data elements with coded representations, which can result in significantly reduced data size without data loss.

A coded (compressed) by H.264/AVC picture contain different slice types, and may come in two basic types – reference and non-reference type.

What’s going on during replay of the video?

It’s important to remember that the fidelity of the playback is constrained by the processing power, the memory size, and other parameters of the AVCHD (H.264/AVC) decoder. Picture size, frame rate, and bitrate play the main role in influencing those parameters. Particularly, the higher bitrate, the higher the fidelity can be.

If the processing power (or the memory size) isn’t up to the task (or the bitrate is constrained to a low limit), the fidelity of playback deteriorates (by throwing away some of the macroblocks or even the picture slices). This is, of course, has nothing to do with the faithfulness of the compressed video, which can be restored (decompressed) in some other way.


I hope this crush introduction into coding technique answers all questions raised before in the course of the thread.

For the purpose of the lecture, I used mainly the papers “The H.264/AVC Advanced Video Coding Standard: Overview and Introduction to the Fidelity Range Extensions” by Gary Sullivan et al (Microsoft Corp., 2004), and “Subjective quality evaluation of H.264/AVC FRExt for HD movie content" by T. Wedi and Y. Kashiwagi (Joint Video Team document JVT-L033, 2004).
Arkady Bolotin is offline   Reply With Quote
Old December 6th, 2010, 10:26 AM   #41
Inner Circle
 
Join Date: Feb 2007
Location: Apple Valley CA
Posts: 4,866
Arkady -
I think you are perhaps missing one aspect that others have mentioned, To quote Yoda, "there is no try, only do or do not" when it comes to playback - if the equipment or media is not up to handling the data stream, it falls apart or crashes entirely.

I think what you're saying is that at a given bitrate, regardless of whether it can be played back, there might be additional image information that can be extracted. My "wrench" post pretty well explains the answer is "maybe", and why that would be true.

Let's say a "raw" data stream would represent for the sake of argument 100Mbps - but you have to get that down to 17Mbps to be able to play it back - there will be a significant reduction required. It the data is sufficiently redundant, that reduction is relatively easy - the more "new" and different data, the more difficult the tack becomes, and the less likely you will find any "restorable" data.

I saw the little bits of noise in the Mosque shot, particularly around the light, that's one of those "artifacts" that personally drives me up the wall - the sky is a smooth tone, and doesn't have a flock of mosquitos. As for the window detail, I thought the 80 Mbs sample was a tad cleaner - this is one of those "judgement calls" on image quality - I've seen people prefer an image with crushed blacks that tend to give a more contrasty overall image, but there is less shadow detail - I myself lean the other way, prefering more gradations of shadow, but the overall image will look less contrasty - just a personal preference.
Dave Blackhurst is offline   Reply With Quote
Old December 6th, 2010, 11:36 AM   #42
Regular Crew
 
Join Date: Sep 2008
Location: Beersheba, Israel
Posts: 111
Dave:

I agree completely with you – the phrase of mine “If the processing power … isn’t up to the task” is an example of bad wording. It’s supposed to be read as “If the H.264 decoder’s power … isn’t up to the task…”

Philosophically speaking, that entire dispute around high bitrates can be reduced to one question: Given a particular AVCHD-based original video what would be the optimal bitrate for the rendered movie? In other words, how we can choose the right bitrate for rendering?

This is an area open for experiments and taste contests.
Arkady Bolotin is offline   Reply With Quote
Old December 6th, 2010, 12:19 PM   #43
Trustee
 
Join Date: Mar 2005
Location: Coronado Island
Posts: 1,452
Quote:
Originally Posted by Arkady Bolotin View Post
Philosophically speaking, that entire dispute around high bitrates can be reduced to one question: Given a particular AVCHD-based original video what would be the optimal bitrate for the rendered movie? In other words, how we can choose the right bitrate for rendering?
It sounds like maybe you are referring to an optimal bitrate for obtaining the best preview imagery within an editing system. That is a reasonable goal, and again there are existing solutions. The most common being to convert the raw AVCHD to a high bitrate DI if needed.
When you say "the optimal bitrate for the rendered movie", this means something different to me.
The movie (edited sequence) is usually rendered to a specific delivery format for distribution- Blu Ray, web, DVD, etc., and those bitrates are determined by the pre existing specs of the various formats.
__________________
Bob
Robert Young is offline   Reply With Quote
Old December 6th, 2010, 12:44 PM   #44
Regular Crew
 
Join Date: Sep 2008
Location: Beersheba, Israel
Posts: 111
Robert:

Yes, industry standards are one thing, but the optimal bitrate is another. Even within the same standard (say, Blu-ray disc), you can choose different bitrates (in the certain range) for rendering.
Arkady Bolotin is offline   Reply With Quote
Old December 6th, 2010, 01:28 PM   #45
Inner Circle
 
Join Date: Jan 2006
Posts: 2,699
Quote:
Originally Posted by Arkady Bolotin View Post
My initial intention was to avoid (by all means) a technical lecture on AVCHD and video compression essentials. Besides, this thread is rather over.
I had intended not to bother with any more answers, but..... well, one last time.....
Quote:
Originally Posted by Arkady Bolotin View Post
However, the main tool is Lossless Entropy coding. Entropy coding is the coding technique that replaces data elements with coded representations, which can result in significantly reduced data size without data loss.
Just what evidence can you quote to support the claim that that is the *main* tool? It's certainly not my understanding. It's one tool, but only one amongst many - and most are likely to involve some loss.

Think about it. Uncompressed HD video is around 1Gbs at 8 bit depth. AVC-HD is about 20Mbs (average). That's roughly a 50:1 average compression ratio! There is simply no way any system can achieve that and still be truly "lossless" on normal pictures. It's a tribute that it manages to achieve it at all, without the lost data being visibly missed under normal viewing.
Quote:
Originally Posted by Arkady Bolotin View Post
If the processing power (or the memory size) isnít up to the task (or the bitrate is constrained to a low limit), the fidelity of playback deteriorates (by throwing away some of the macroblocks or even the picture slices). This is, of course, has nothing to do with the faithfulness of the compressed video, which can be restored (decompressed) in some other way.


I hope this crush introduction into coding technique answers all questions raised before in the course of the thread.
No, far from it. The first paragraph implies that whatever bitrate the original video is compressed at, the original can always be reconstructed by a powerful enough computer. If true, the expectation would be that playback quality would vary widely with the power of the hardware it's replayed on.

But that is not the case in practice. Quality is independent of the power of the decoding computer - until a lower threshold is approached, when playback first becomes stuttery, then fails completely. The compression quality is a function of the individual coder and bitrate - not the replaying equipment.

At AVC-HD bitrates, H264 will have to discard information at the coder, and once lost, it's lost.
David Heath is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > High Definition Video Acquisition > AVCHD Format Discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 09:22 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network