Looking at the cost of buying the quite exciting Sony EX3 and a Flash XDR box, you're getting upto around £11,000 or so, and close to the realms of serious gear like Panasonic HPX2100 etc. (well sort of 'cos I know you'd need a lens on top of that). Anyway just wondering what sort of quality you could expect from the EX3 and the XDR, close to (or better than?) DVCProHD from the likes of the HPX2100, Varicam, even Sony 750?
Any thoughts?
nd presumably this combo should be acceptable by broadcasters as full HD for programme acquisition?
Steve

In many cases the EX3 can already be of higher quality then DVCPROHD. It is only those few rare cases when DVCPROHD has the advantage. If you add the Flash XDR to the mix you get the option of I frame only recording at a bitrate much higher then DVCPROHD. If you choose long form recording of 100 mbits or I frame only at 160 mbits this pretty much blows away any recording format out there including HDCAM. About the only thing better is HDCAM SR but even then the long form 100 mbit should be pretty on par with HDCAM SR except for the lack of 4:4:4 RGB recording.

I'd have trouble believing that an EX3 would provide better pictures than a Varicam, any real evidence that it might would be very interesting to see.
I was wondering (and I suppose doubting) whether EX1/3 +Flash XDR would even get close?
Steve

I don't know the answer to your question, but I would be careful to make clear the difference between talking about codecs and talking about cameras.

The sentence - "In many cases the EX3 can already be of higher quality then DVCPROHD."

Could be misleading as the EX3 is a camera and DVCPRO HD is a codec.

People wil argue all day long that DVCPRO HD is better than MPEG-2 at 35 Mbps.

Ask yourself what is important to you? Resolution? Overall image impact?

I would guess that the XDR unit will help, but I would have to see how much.

I don't know the answer to your question, but I would be careful to make clear the difference between talking about codecs and talking about cameras.

The sentence - "In many cases the EX3 can already be of higher quality then DVCPROHD."

Could be misleading as the EX3 is a camera and DVCPRO HD is a codec.

People wil argue all day long that DVCPRO HD is better than MPEG-2 at 35 Mbps.

Ask yourself what is important to you? Resolution? Overall image impact?

I would guess that the XDR unit will help, but I would have to see how much.

Hi Tim-
We have images on our website that compare the XDR MPEG2 50 Mbps 4:2:2 Long-GOP with DVCProHD. We used the Canon XL-H1 as our source in both cases. We captured directly into XDR and into the DVCProHD CODEC on Final Cut Pro.

In our view, the 50Mbps MPEG2 Long-GOP is clearly superior to DVCProHD. Part of the reason lies in the full-raster (1920x1080) vs the sub-sampling (1280x1080) employed in DVCProHD. The 100Mbps Long-GOP is very near uncompressed in quality.

It is always difficult to compare different cameras with different CODECs, but we think the EX1 + XDR in either 100Mbps Long-GOP or 160 Mbps I-Frame is going to produce incredible video, comparable to much more costly setups.

We will be testing this combination in the coming weeks.

I agree, I am not trying to dispute.

I was mearly trying to point out that cameras and codecs are different.

Thomas' reply sounds as if the EX-1/3 are better than the DVCPRO codec.

The HPX series records DVCPRO HD 4:2:2 at 100 Mbps

As opposed to the EX1/3 which records Mpeg-2 4:2:0 at 35 Mbps

I wouldn't think a 4:2:0 color space would beat a 4:2:2 color space at any resonable bitrate.

On a side note:

Mike I went to the site and looked at the images.

I must say, I am looking forward to seeing some video before drawing any conclusions. Stills show some things, but video is a different animal.

Plus without seeing it on a production monitor or nice T.V., it is tough to make a call over the internet.

I wouldn't think a 4:2:0 color space would beat a 4:2:2 color space at any resonable bitrate.
But that assumes equivalent luminance rasters, which here define "4". And that's not true if we compare the native EX1/3 codec with DVCProHD. The former is 1920x1080, DVCProHD is 1280x1080, so hence chrominance rasters of 960x540 versus 640x1080. And certainly for progressive recording I'd expect 1920x1080 4:2:0 to outperform 1280x1080 4:2:2, all else equal.

But I do agree that cameras shouldn't be confused with codecs. Given identical source material (as in Mikes tests) MPEG2 may outperform DVCProHD - and the 50Mbs variants upwards are 4:2:2 themselves - but different cameras mean different source materials.

Wow, this stuff can get complex when all of the variables are taken into account.

I think I will just listen for a while as I need to learn more about this space.

Topic changed from "A different Flash XDR topic"

to "Quality expected from the EX3 and Flash XDR"

-- please avoid ambiguous thread titles on this site.

Same here Tim!

Dear Steve and Tim,

We started our testing with a video image, created by a static image, using a locked down camera, in other words, no motion.

We did this for a number of reasons, but foremost was that this was a great way to show the very detailed differences between the quality of the codecs, using the exact same image as the source.

When viewing video at 24 frames per second, or 30 frames a second, small imperfections in the image are sometimes difficult to see.

We fully realize that posting full motion video is very desirable and we will be doing so in the near future.

Mike Schell and I have been discussing the best way to accomplish a meaningful comparison of full motion video.

We have, in house, a very nice Canon XL H1 and Canon XH G1. Both produce high quality uncompressed images through the HD-SDI port.

My suggestion to Mike, for the first round of full motion video testing, was to get a very nice floral arrangement, and then pan around while zoomed in on the intricate parts of the flowers.

Our challenges include the fact that we want to do this in a studio setting (Convergent-Design's office) since we are recording uncompressed HD-SDI to a Macintosh, using Final Cut Pro. We can go outside, if we get a longer HD-SDI cable. One option, once we get a longer cable, would be to record cars passing by. I believe that we could accentuate the movement by recording at high zoom ratios.

In any case, for these early tests, we want to show the differences between the various codecs such as HDV, DVCPro HD and uncompressed. Also, we feel that it is useful for us to show the differences between Long GOP and I-Frame only codecs.

We felt that MPEG2, Long GOP at 25 MB maximum bit rate would benefit from a higher bit rate encoding.

I did some HDV torture testing when I first purchased my XL H1. I intentionally recorded images that I thought it could not handle and found it could handle at lot more than I ever expected. But, I could find a few cases where it did have some problems.

Personally, while I believe HDV is almost a miracle in how well it actually works, it can benefit from a higher bit rate. So we used HDV (MPEG2 4:2:0 at 25 Mb) as are baseline for all comparisons.

Our results show that this is true, (but note that we also changed from 4:2:0 to 4:2:2 for our tests at higher bit rates.)

Switching from 4:2:0 to 4:2:2 is a very nice side benefit from using the HD-SDI port on most (if not all) cameras. Since we can record 4:2:2 using the Flash XDR so easily, we are doing so in our testing.

So, we welcome your suggestions as how we should proceed with our testing of full motion video.

At this point in our testing, we want to use one uncompressed HD-SDI image, as the basis for our codec/bit rate testing.

In the future, we will be comparing the outputs from various cameras, using their HD-SDI outputs, recording directly into our Flash XDR using various recording options.

Hello Dan,

I think motion and Color fidelity are at the top of my list to see in a comparison video.

I have read about some problems with 720p60 and artifacts in the JVC forum, so maybe some testing at that framerate would show some weakness in the HDV spec.

Overall as a potential consumer, I need to justify the increased image quality against almost the cost of another camera ($5,000) in purchasing the Flash XDR.

The amount of the noticeable image quality increase vs the price of memory for the camera will make my decision.

But I have to be honest, as camera memory prices fall, the case for the second camera becomes stronger :(

Mainly because DVD is still the main delivery method and a lot of the HD goodness may be lost in translation...

I agree, I am not trying to dispute.

I was mearly trying to point out that cameras and codecs are different.

Thomas' reply sounds as if the EX-1/3 are better than the DVCPRO codec.

The HPX series records DVCPRO HD 4:2:2 at 100 Mbps

As opposed to the EX1/3 which records Mpeg-2 4:2:0 at 35 Mbps

I wouldn't think a 4:2:0 color space would beat a 4:2:2 color space at any resonable bitrate.

On a side note:

Mike I went to the site and looked at the images.

I must say, I am looking forward to seeing some video before drawing any conclusions. Stills show some things, but video is a different animal.

Plus without seeing it on a production monitor or nice T.V., it is tough to make a call over the internet.

Hi Tim and Everyone-
These are very interesting discussions! I think everyone has now concluded that it's extremely difficult to compare different cameras with different sensors with different compressions CODECs. DVCProHD I-Frame (1280x 1080) 4:2:2 may or may not be better than MPEG2 Long-GOP (1920x1080) 4:2:0. On paper, it's a difficult call and may depend on the actual content.

But, I think we can continue to compare these various CODECs by using a single camera source (Canon XL-H1 or Sony EX1, for example) and capturing into these various CODECs. We can dial in the 35 Mbps EX1 CODEC in Flash XDR for these tests.

We can also do some comparisons of the front-end of some of these cameras by capturing the uncompressed HD-SDI output. This should allow us to separate the camera front-end from the native CODEC.

I am excited to report that the new Transcend 16GB 300X card is now available for around $240. This should support the 160 Mbps I-Frame only mode (about 13 minutes storage per card or 52 minutes for 4 cards). The 160 Mbps I-Frame only mode should yield some incredible results for 1080p24, since this is a full-raster (1920x1080) 4:2:2 8-bit capture with an effective compression of 5:1. This should be visually lossless and of course, no motion artifacts from Long-GOP.

I would still contend that for most applications, the 100Mbps Long-GOP (1920x1080) 4:2:2 is the "sweet spot". This rate is supported by the lower cost 32GB cards ($150) and should be very close to visually lossless. The only downside of this format occurs during very high-motion or fast changing scenes.

But Flash XDR gives you the option. Use 100Mbps Long-GOP for most of your shoots, then switch to 160Mbps for green-screen and high-action.

We will be posting video shot with both of these formats in the near future.

Also, as side note, I got a recent e-mail reminding me that Flash XDR can also be used to accelerate the encoding of Blu-Ray material. You should be a able to feed HD-SDI (from your NLE) to XDR and encode at 18-20 Mbps 4:2:0 in real-time. Then read the compressed footage (from the CF cards) back into your Blu-Ray authoring program. This should save you a ton of time in rendering the content.

So, we welcome your suggestions as how we should proceed with our testing of full motion video.
Firstly, I believe there are "reference" uncompressed video samples used by bodies such as the EBU, specifically to test various codec aspects. Some specifically test motion performance, some fine detail, others colour detail etc. I don't know if they are available to third parties, if not, your approach of generating a reference via HD-SDI, and deriving all the other versions from that uncompressed recording seems a sound approach.

In another thread, the technique is discussed of using Photoshop in difference mode, and using the filmstrip option extends making true comparisons from stills to moving images.

An example would be a clip of video, say 50 frames long, and lets say it consists of 15 frames of stationary image, 20 frames of panning over fine detail, and a further 15 frames of stationary image. An HDV (say) version is derived from the uncompressed original. Both versions are brought in to Photoshop as filmstrips, and separated into separate Y,U, and V. A given frame from the HDV version should be subtracted from the equivalent uncompressed version, and mid grey added.

If all the results were plain grey frames, the compression should be considered perfect. In practice, imperfections and the results of subsampling will show up as deviations from uniform grey.

If now frames from the first, second and third section are compared, and the deviations are found to be far greater for the second compared to the other two, it may be taken as a sign of how much the codec is "breaking down" with motion.

Dear David,

Thank you for your excellent suggestions!

Since HDV-II, Canon's Flavor of 25 Mb MPEG2 Long GOP recording, uses 15 frame "Group of Pictures", which is about 1/2 a second, we should have each segment be at least 1 to 2 seconds long so that we have a complete "Group of Pictures" for each segment.

David, your suggestions seem to be very sound. Our goal is to have the best possible comparisons.

When we shoot the motion video footage, we will be posting the footage on our website so that others can perform their own tests, in addition to the tests that we will be performing. Of course, the real problem is posting uncompressed footage due to the size of the files.

Mike,
Not sure I agree that it's difficutl to compare different cameras. Put a top-notch lens on a Varicam, the standard (I assume pretty decent) lens on the EX3, with Flash XDR and shoot a couple of different, challenging scenes. Put 'em on a decent sized monitor and decide if they look pretty close or poles apart.
What I would like is some fairly clear evidence that the combination is up there with Varicam/Sony 750 and would cut with them well. These 2 cams are pretty much the standard and the benchmark for HD TV in the UK, and most producers would flinch a little at using a 1/2" "semi-pro" camera (I know that definition is getting a bit blurred these days) unless one could show them some evidence that it'll do what they need it to.
Steve

Our goal is to have the best possible comparisons.

When we shoot the motion video footage, we will be posting the footage on our website so that others can perform their own tests, in addition to the tests that we will be performing. Of course, the real problem is posting uncompressed footage due to the size of the files.
It does occur to me that one of the biggest tests of interframe codecs (at least on the evidence of bitrate limited broadcast TV!) is quick mixes or fades.

Since you are interested in codec comparisons (as opposed to actual cameras) then bearing in mind what you say above, is there any merit in not using actual video for the test, but rather mixing between high quality 1920x1080 still images? Using that as a source of the uncompressed reference? As well as precision and repeatability, that may have an advantage in that whilst the scene is changing rapidly frame to frame, (and is hence a good test of long-GOP compression), nothing is actually moving in the frame, so it may be easier to draw conclusions via the Photoshop difference method and filmstrips.

It would also make it very easy to compare the differences between (say) 8, 16, and 30 frame mixes.

Posting the original images, for people to download and derive their own reference (as per your instructions), would also vastly reduce download file sizes compared to motion video.

Dear David,

Your suggestions are very interesting.

Yes, it would make the task easier and the downloads quicker since we could control the images better. Of course,we would have to obtain or create the still images.

But, I wonder if everyone would feel that this is a valid test? Would they think it is "kind of artifical" and then they would still want to see actual full motion video as created by a known camera.

Mike,
Not sure I agree that it's difficutl to compare different cameras. Put a top-notch lens on a Varicam, the standard (I assume pretty decent) lens on the EX3, with Flash XDR and shoot a couple of different, challenging scenes. Put 'em on a decent sized monitor and decide if they look pretty close or poles apart.
What I would like is some fairly clear evidence that the combination is up there with Varicam/Sony 750 and would cut with them well. These 2 cams are pretty much the standard and the benchmark for HD TV in the UK, and most producers would flinch a little at using a 1/2" "semi-pro" camera (I know that definition is getting a bit blurred these days) unless one could show them some evidence that it'll do what they need it to.
Steve

Hi Steve-
I only meant that it's difficult to compare various cameras from the point of view of determining whether the CODEC or the sensor controls the ultimate quality. Since the HD-SDI output bypasses the built-in recorder, we can easily compare the front-end of all these cameras.

I have seen resolution comparison charts of the EX1 vs cameras in the $25K price range. The EX1 clearly outperforms cameras with 2/3inch sensors on this resolution chart. I think the use of full-raster 1920x1080 sensor vs 1440x1080 (or 1280x1080) makes all the difference.

Since the MPEG2 CODEC on Flash XDR is also full-raster, we can keep all this quality through the compression process. DVCProHD CODEC uses a 1280x1080 raster, which results in a considerable loss in resolution.

I hope to post comparision video in the next week or so.

But, I wonder if everyone would feel that this is a valid test? Would they think it is "kind of artifical" and then they would still want to see actual full motion video as created by a known camera.
I certainly wouldn't propose it as the only test, and yes, I agree most people would want to be aware of real moving video performance. Normally, it's cameras that are being compared and hence front ends etc as well as codecs, so such an approach isn't possible.

But too often, I find it difficult to draw accurate conclusions from some tests along these lines - the motion can all too often cause such blurring etc that it's difficult to tell what degradation is due to any codec failure, and what is due to characteristics of the image itself.

Mixing between two still images shouldn't have the same problem, especially if the difference approach is used to compare the compressed and uncompressed versions - whilst still being able to tax inter-frame performance. "Kind of artificial"? Well, maybe, but sometimes "artificial" means are necessary to quantify video performance in a meaningful way. Hence the variety of test signals from analogue days - pulse and bar, colour bars etc.

But as you imply, all this is academic if the majority opinion is that the test is meaningless. Anyone care to give their views?

Dear David,

For discussion purposes:

(Please forgive me if all of this is completely obvious.)

Let's say we have 15 distinct images, each completely different from the 14 others. Then if we use a Long GOP Codec to encode them, then this would be an extreme test of the Long GOP Codec. If we use an I-Frame Only codec, it would make no difference that all were different.

In typical motion video, there are some similarities from one frame to another. Of course, when the scene changes, there can be a drastic change in which there are no similarities. But, typically these drastic changes usually do not occur more than once or twice in a group of 15 images.

So, for typical motion video, a Long GOP codec takes advantage of these similarities. The areas that do not have to be encoded, since they are similar to the previous frame, allow more bits to be used to encode the image, thus providing a high quality encoding.

In your proposed test, will each image be completely different?

Hi Dan, David-
Just wanted to mention that we are working on a program to stripe 160Mbps I-Frame only video across two of the low-cost ($150) 32GB Transcend cards and then re-assemble in post. Assuming this can be accomplished, I think we'll have an very cost effective solution for either 100 Mbps Long-GOP or 160 Mbps I-Frame only. Four of the 32GB cards would support 105 minutes of 160 Mbps record time w/o any card swaps.

So depending on the situation, you can either go with Long-GOP or I-Frame mode. Even if our tests do show good motion performance with Long-GOP (all indications are positive), you will still need to make a judgment call for any given shoot. We would simply like to provide you with the capability and let you make the choice.

In your proposed test, will each image be completely different?
You raise some good points!

Yes, I agree that a "test" where 15 totally different images are presented is hardly a fair test of a long GOP codec - though it may be interesting to see the results!

As far as my suggestion of using a mix, then I'd say that whilst each frame will be different from the ones before and after, it would be wrong to say it would be completely different. Imagine a 8x8 section of the image, starting as (say) left half white, right half black and mixing over 15 frames to a uniform red. One frame into the mix, the left side will still be quite high luminance, the right quite low luminance, and the whole still quite low chrominance - in other words, still with quite a lot of similarities to the previous frame.

Obviously, the slower the mix, the more similar adjacent frames will be, hence my suggestion to do the test at several different speeds. Effectively, varying the speed of the mix simulates varying the amount of motion in an action sequence, but does it in a controllable and repeatable fashion.

Whilst a camera can't mix from one image to another within itself, I understand the XDR is marketed as a recorder for applications other than purely single camera recording, so encoding mixes is far from academic. Even in single camera use, there's the situation where still camera flashes may be going off and causing rapid frame by frame changes, even more so than a uniform mix.

While others argue about the technical merits of this camera versus that camera and this codec versus that codec, I think Mike has hit the nail on the head - it is much more interesting to see some typical footage. People moving about, colourful images of a beach scene perhaps, some test chroma key footage etc. Photoshop difference maps are just so one company can claim their codec is better than anothers and I've seen plenty of instances where they mean nothing with real world images. Believe your eyes my friends - they are the ultimate QC...

From all accounts the EX3 (like the EX1) will be stunning HD full raster quality - if you want to whack a 1/2" broadcast HD lens on the front it's likely to get even better. Combine that with the FlashXDR and you have a technically sound recording device with cheaper alternative media.

The other stuff is just a distraction (an interesting distraction) - come on Mike - give us some real world footage to look at in HD. I'd prefer to see it off the EX1, but even the XL-H1 will do.

Dear David,

I agree completely.

Multiple Flashes going off in multiple frames are a torture test for a codec.

Mike's news is very nice to hear. What has been obvious in our discussion is that one codec has an advantage over another under specific conditions. So, the choice of which codec to use is provided by the Flash XDR.

I would like to perform your recommended tests as soon as we can fit it into our schedule.

Dear John,

Tests with a Sony EX1 are in our plans. I do not know the exact date, but it should not be a long wait. As Mike says: "Stay Tuned".

Dear Mike,

Supporting 160 Mb I-Frame only, in the very cost effective Transcend 32 GB cards is a very nice feature!

Comparing cameras and codecs is it really hard to believe that a EX3 and Flash XDR combo could out perform a varicam recording to tape? The EX3 combo should be noticeable better. The varicam is a 720p camera recording to a 40Mbps dvcprohd codec. Are you serious! The varicam is a great camera but it is old technology. The EX3 and XDR is future technology. It is the nature of the business. New stuff comes out old stuff gets replaced with cheaper options. Now a varicam recording to XDR is a different story. :)

I agree completely.

Multiple Flashes going off in multiple frames are a torture test for a codec.
Which brings a thought to mind. I've read quite a few posts where samples of EX footage have been examined and commented on with regard to rolling shutter effects and camera flashes. Yet in none of these posts has any comment been made about codec breakup with this footage (AFAIK). Which leads me to think that even the basic 35Mbs codec is far more robust than some may think. So..... going up to 50 or 100 Mbs should indeed be very promising from a quality point of view?
Photoshop difference maps are just so one company can claim their codec is better than anothers and I've seen plenty of instances where they mean nothing with real world images. Believe your eyes my friends - they are the ultimate QC....
Sorry John, I disagree. Except at the low end (which is not what we're talking about here) I'd argue that all the decent codecs look pretty good with real world images for straightforward record/playback. Full raster codecs are obviously going to look sharper than subsampled codecs - if the source material is up to it - but any differences the eyes will see at this level are largely due to camera front ends.

Where codec differences start to become more important is when decoding/recoding happens, and that can happen several times cascaded in a broadcast chain. Some codecs can also stand up far better during the post production process, during FX, grading etc work. And difference maps not just show up, but actually help quantify differences between images, differences that are just not discernable in straight forward viewing. But differences which do become important cascading down the production/broadcast chain.

http://www.photoshopessentials.com/essentials/jpeg-compression/ explains the basic principles of difference mapping for still images (with examples) if anyone is interested. The process can be taken further by separating the images into Y,U,V and comparing these individually.

A company doesn't need difference maps to CLAIM codec A is superior to codec B, but do need them (or something equivalent) to PROVE the claim if challenged. And if another company has a vested interest in codec B, and does seriously dispute the claim, that proof may be very valuable - far from "just a distraction ".

Nikol, not saying I disagree with you. I just to be able to convince producers that it's the case.
Steve

Sorry John, I disagree. Except at the low end (which is not what we're talking about here) I'd argue that all the decent codecs look pretty good with real world images for straightforward record/playback. Full raster codecs are obviously going to look sharper than subsampled codecs - if the source material is up to it - but any differences the eyes will see at this level are largely due to camera front ends.

David - I'm sorry if my previous post seemed a little blase. It wasn't my intention. I just see no point in comparing MPEG2 @ x data rate with DVCProHD @ x data rate, because the Flash XDR doesn't record to DVCProHD. It will however blow HDV and and 35Mb/s MPEG2 away on cheaper recording media, and because of it's full raster nature will be preferred over DVCProHD. I don't need tests to confirm that. But I would like to see some vision first :)

http://www.photoshopessentials.com/essentials/jpeg-compression/ explains the basic principles of difference mapping for still images (with examples) if anyone is interested. The process can be taken further by separating the images into Y,U,V and comparing these individually.

A company doesn't need difference maps to CLAIM codec A is superior to codec B, but do need them (or something equivalent) to PROVE the claim if challenged. And if another company has a vested interest in codec B, and does seriously dispute the claim, that proof may be very valuable - far from "just a distraction ".

I understand the way difference maps work, but applying a technique best used on still images is less than ideal for any intraframe codec. That's why I don't think it's relevant, because an interframe codec like DnxHD may "appear" to outperform an intraframe codec like MPEG2 for any particular frame (except for an iframe) because the MPEG2 codec has to recreate the image from earlier and later frames, in reality when you watch the motion footage those differences are irrelevant to the human perception system. That's why I say trust your eyes... not Photoshop.

And some things like repeated camera flashes will "break" an MPEG2 or AVCHD codec because that is not what the codec was designed to do - it's designed to compress motion as well as data by using predictive algorithms, and by it's very nature you cannot predict a camera flash. Of course the quality of images produced under a camera flash may depend more on the front end of the camera than the recording method.


Where codec differences start to become more important is when decoding/recoding happens, and that can happen several times cascaded in a broadcast chain. Some codecs can also stand up far better during the post production process, during FX, grading etc work. And difference maps not just show up, but actually help quantify differences between images, differences that are just not discernable in straight forward viewing. But differences which do become important cascading down the production/broadcast chain.

No argument here. Multi generation work is where Photoshop difference mattes are valuable. But I don't think you'd argue that MPEG2 or AVCHD were "good" for multi-generation work. Ideally if you anticipate doing a lot of post manipulation you will not use Mpeg 2 to work in - it is an acquisition and delivery format (and originally it was designed purely as a delivery format, it's just that manufacturers saw the value of compressing the data more at the acquisition end to improve quality at any given bit rate). You would use ProRes, DnxHD, Cineform or a totally uncompressed system like tiff sequences etc.

Much more important (to me anyway) is the real world stuff - can I pull a beautiful chroma key off vision recorded on the FlashXDR? How far can I colour correct and push things like exposure before it starts to break up at any given data rate? Can it capture fast moving action effectively without macroblocking? These are things I can only evaluate with real world footage from a decent front end.

I just see no point in comparing MPEG2 @ x data rate with DVCProHD @ x data rate, because the Flash XDR doesn't record to DVCProHD. It will however blow HDV and and 35Mb/s MPEG2 away on cheaper recording media, and because of it's full raster nature will be preferred over DVCProHD. I don't need tests to confirm that.
I suspect the reason that Convergent Design included DVCProHD in their initial tests was to act as a form of "control" sample. It's an established professional codec, and makes a useful benchmark against which to compare MPEG2 at different bitrates - it would be interesting to see how HDCAM also compared.

Of the codecs you mention, I would also bet on 100Mbs MPEG2 to outperform the others, no argument, but feel that PROVING it would be valuable for the reasons mentioned before. To that end, difference maps are the best way I can think of of making valid measurements. I would strongly suspect that DVCProHD would show substantially more differences in still images (partly due to subsampling), but the level to remain constant during any mix. I would also expect the difference levels for the MPEG2 samples to increase during the mix relative to the still images, but if the levels were still lower than for DVCProHD at the equivalent point, this would prove fairly unequivocally that it was a superior codec for quality, even in motion scenes.
I understand the way difference maps work, but applying a technique best used on still images is less than ideal for any intraframe codec. That's why I don't think it's relevant, .........., in reality when you watch the motion footage those differences are irrelevant to the human perception system. That's why I say trust your eyes... not Photoshop.
I have much agreement there. I've seen certain images put forward from very rapid motion to show the effects of interframe compression, and yes, they show something, but totally lost admidst the blur of motion footage in normal viewing.

But in a way it's why I say don't just trust your eyes. Test signals for analogue video were designed to show up imperfections that just weren't discernable on normal pictures, but could become a problem in a cascade. Hence of use for lining up one link in a broadcast chain in isolation to the rest, and it's a similar principle here. Small imperfections don't neccessarily matter in real life, but they do allow quantification and comparison.
And some things like repeated camera flashes will "break" an MPEG2 or AVCHD codec because that is not what the codec was designed to do - ....... by it's very nature you cannot predict a camera flash.
Yes, but unfortunately camera flashes going off, strobes etc is a fact of real life and a codec does need to be able to cope, at least if it's claiming an upmarket status. But even 35Mbs seems to do quite a good job in practice, and presumably 50 and 100Mbs will be better still. Flashes and mixes in this case may well not break any codec - but they will stretch and test it.
Much more important (to me anyway) is the real world stuff - can I pull a beautiful chroma key off vision recorded on the FlashXDR? How far can I colour correct and push things like exposure before it starts to break up at any given data rate? Can it capture fast moving action effectively without macroblocking? These are things I can only evaluate with real world footage from a decent front end.
I'm sure the conclusion will be "it's very, very good". Trouble is it may be so good that no difference is seen between it and rival codecs, or even uncompressed. Difference maps are more likely to show up subtle differences which may prove more of an issue on other images, or when codecs are cascaded.

Exposure and colour correction are more likely to be influenced by bit depth than data rate, and if there's anything about this that leaves me thinking "if only" is that it's still an 8 bit codec.

Dear David,

All MPEG2 is 8 bit.

However, with the upcoming Flash XDR firmware upgrade, which is scheduled to be available by the end of this year, the unit will be able to record 10-bit uncompressed.

This firmware upgrade is an additional cost feature.

At the Canon Booth at NAB, there was a nice Convergent-Design poster on the wall.

The poster had a very nice gradient which I expected to show color banding in the image.

I aimed the Canon XL H1s at the poster and the image on the Apple display did not show any banding that I could detect. I was very impressed.

While others argue about the technical merits of this camera versus that camera and this codec versus that codec, I think Mike has hit the nail on the head - it is much more interesting to see some typical footage. People moving about, colourful images of a beach scene perhaps, some test chroma key footage etc. Photoshop difference maps are just so one company can claim their codec is better than anothers and I've seen plenty of instances where they mean nothing with real world images. Believe your eyes my friends - they are the ultimate QC...

From all accounts the EX3 (like the EX1) will be stunning HD full raster quality - if you want to whack a 1/2" broadcast HD lens on the front it's likely to get even better. Combine that with the FlashXDR and you have a technically sound recording device with cheaper alternative media.

The other stuff is just a distraction (an interesting distraction) - come on Mike - give us some real world footage to look at in HD. I'd prefer to see it off the EX1, but even the XL-H1 will do.

Hi John-
Could not agree more! As Dan Keaton mentioned, we have some full-motion video recording in the works at 50 / 100 Mbps Long-GOP and 160 Mbps I-Frame.

We just received the new Transcend 16GB 300X Compact Flash card ($240). It is rated at 45 MBytes/sec (360 Mbps) read and write. Based on our tests of the 32GB 133X card (also from Transcend), I am betting this card will easily meet these numbers. (The Transcend 32 GB 133X card is about 150X write and 300X read speeds, a real over-achiever). We will use this new card for initial tests of the 160 Mbps I-Frame.

We have now confirmed read and write operations to all four Compact Flash cards. We feel very confident about offereing the uncompressed feature. Initially, we will limit the overall bandwidth (across all four cards) to 130 MBytes/sec, which will support 1080p24/25 and 720p24/25/30 at 10-bit and all other formats at 8-bit (all 4:2:2). Four of these 16GB Transcend cards will provide about 8 minutes of record / playback time.

I have another couple of days checking out the analog audio and then we will return to more tests using the Canon XL-H1 and the Sony EX1. We do have some blue/green screen material (captured uncompressed from the EX1) for testing keys. We'll also post this material for everyone to test.