5DMkII 1080p Video Imatest MTF50 Resolution Testing Results
 

The test sample is my own 5DMkII body with V1.07 firmware, wearing a 50mm F1.4 prime lens, shooting an ISO12233 equivalent target in the center of the frame. Shooting the target from a tripod was 6.5 feet from the film plane. The camera settings were locked to the sweet spot of the lens, F/5.6, 1/50, ISO 400.

This is the same setup I have used to test my other video cameras that I have reported on at DVinfo. You can review those results as well, just search on my name and keyword "imatest".

Imatest is image testing software widely used for evaluating lenses and systems. Although the numbers are not comparable to other labs having their own established methods and procedures, they are comparable to my own as I have conducted the tests identically for the following video cameras systems:

Canon HV10
Canon XH-A1
Sony PMW-EX1
Canon 5DMkII

The test I performed was only for center resolution, MTF50, modulation transfer function, which reports the resolution at the point where contrast drops to 50%. The test normalizes the result to a standard 2-pixel sharpening radius, so that cameras with different levels of in-camera sharpening can be compared.

The 5DMkII has generated a lot of excitement for users, including me. Notable is the low light performance and potential for shallow depth of field opening many creative possibilities. Such videos have been amply demonstrated.

Unfortunately, it seems likely that more of that is owing to the talent of those users as the camera itself turned in rather pedestrian, underwhelming numbers. The 50mm F/1.4 prime is capable of out resolving the 21mp sensor at the center by a substantial margin, especially stopped down to F5.6, yet the limiting factor seems to be the Digic 4 processing which shoulders the task of downconverting the full frame sensor's native resolution to 1920x1080 in realtime.

In the end, the 1080/30p MTF50 results have the 5DMkII lagging behind all of the other HD/HDV video cams tested, the corrected horizontal resolution measuring just 623 lines, while the vertical a much more respectable 756 lines. For comparison, the Sony PMW-EX1 managed 931 horizontal lines, and an eye popping 1109 vertical lines.

5DMkII Horizontal

5DMkII Vertical

Do you think that your figures for the 5DII actually mean anything in real life? As an owner of both XH-A1s, an HV30 & 5DIIs I know which I think shoots the more detailed video.

How do you explain the difference between what your test measures & even the most casual recording on the 5DII by users without any talent? Perhaps some kind of optical illusion that fools us into thinking that the picture quality is better than it really is?

Not to argue with how you conducted your test, but how does the HV10 even measure up to this? resolution-wise? A Day in Varanasi on Vimeo

I've got one and I know it doesn't... So I guess you can say I'm confused...

Perhaps the 5d's sharpness with less lines is better than a less sharp image with more lines?

I have to agree with the above. Tests are fine some but for me, I'm not interested. I just like the look of the footage.

Avey

Can someone please sum up the results for me in layman's english?

Much thanks.

It doesn't surprise me that the true resolution of the 5D2 is somewhat lower that these other video cameras. But is the result really due to the Digic 4 downconverting? In the end, if you are working with imatest, you are working with a captured still image. It's also possible that the h.264 encoding done by the 5d2 causes the loss in resolution. Maybe that's what you meant by "Digic 4 processing".

An interesting experiment would be to check both 5D2 raw and JPEG still images (not the video captured to a still) at different resolutions and see what you get with imatest.

However the camera does it, image wise, it appears to me to outshoot my HV20 and my FX1. Obviously unscientific. I don't know the tester, but something doesn't make sense here. I am getting the impression the the tester thinks the problem is with the processing of the image.

In any event, the resulting image is what it is all about, and resolving power is not what this camera, in the video world, is about. The fullframe sensor provides a new depth of field perspective for the low budget film maker, and thats what it is all about. The image that astounds many but apparently fail the OP's test, is a big plus too!

The resolution limits are fairly obvious in real world pictures that contain horizontal or vertically spaced lines near the extinction point, moire patterns appear. They can be observed from the live view lcd window of the 5DMkII.

The following pictures illustrate this. The first is a small 100% crop of an area of detail. The second picture is a .jpg screen grab from the native .mov file captured by the Zoombrowser utility. The third picture is the same as the first, except that it has been resized from the native 5616x3744 by the Canon Digital Photo Professional utility, to 1920x1080 to match the dimension of the movie screen capture. This image also contains the EXIF shooting information. What it compares, is the effectiveness of the digic 4 resampling (or h.264) to resampling of the same raw by DPP.

All three images were shot with the 50mm prime, aperture locked at F/5.6, shutter 1/160th, ISO 100.

Area of detail, 100% crop
Screen grab of .mov
Still photo resized to 1920x1080 by DPP, with EXIF info

Your browser window needs to display the above images at 100% magnification for proper viewing.

But Tom - You are comparing a still frame from the 5D2's video against a down-sized image from the 5D2's original 21MP image. It's like comparing the resolution of a still from HD camera to downrezed 35mm film material. The result is would be obvious. To better make your point is to compare the 5D2's movie frame to a movie frame of the same scene shot the HV10.

Okay. I understand. The pictured scene is from the parking lot at work. I'll bring the HV10 with me tomorrow, and the EX1, and redo the shots.

Hi Tom:

As an engineer, it is easy to quantify cameras based upon numbers, better numbers equal a higher quality image. But as an artist, all of this measurement goes out the window. I shoot with the HPX170 mostly because I own it. I own it because I really like the way it records images I have created. I have two clients who own the EX1 and when I shoot for them, I use their EX1s. The EX1, by measurement alone, easily outperforms the HPX170 with much higher resolution, there is no denying that Sony EX1 is a much more detailed, sharper and higher resolution camera.

But to my artistic eye, the Panasonic HPX170 looks better. It uses a better codec, has more color space sampling and the color, contrast, gamma curve and matrix are more filmic than the EX1. To me, the EX1 has that interlaced Discovery HD look even when shooting in progressive modes. It looks great, but it does not convey the same feel that the footage from the 170 does.

I wonder if we are experiencing the same with the 5D MKII? Most people, in an A/B blind test with the other cameras you mention, would find the picture on the 5D MKII better, more detailed and more filmic. Yet according to your measurements, the 5D MKII has fairly average resolution measurements. I think the take away is that we perceive overall picture quality much more based off of an emotional recognition of how an image looks than by sheer resolution and detail.

Interesting test and thanks for doing it.

Dan

Interestingly, I experienced the moire effect in a recent shoot. It was occuring on tile roofs in back ground, with my zoom at wide angle (28mm), and f stop around 5.6. Everything was in pretty sharp focus, and the moire pattern showed up on the tile roofs in background. In other shots, with F 3.5 set, and zoomed in a bit closer to the subject, the background loses focuses, and moire disappears.

I absolutely agree, and also with what Dan said. The moire is not going to be apparent in a lot of the intended desirable situations, tighter shots, shallow depth of field etc.

The 5DMkii 1080/30p video grab frame from below for example, there would never be a problem with.

Best friend

favorite quote, "When a dog is a man's best friend, THAT dog has a problem."

There might be one other variable thrown in, when comparing various video cameras: the quality of the decoder. The Canon HV10 the Canon XH-A1 are HDV (1440 x 1080, 1.33 PAR) and use MPEG-2 compression, so they can be compared. The Sony PMW-EX1 also uses MPEG-2, but at a higher bitrate and at a full 1920 x 1080. The Canon 5DMkII uses h.264 (MPEG-4), so that throws another variable into the equation.

But that said, depending on the testing workflow, a given computer will have a specific decoder for that type of compression. The test might come out differently on different computers. It's not as big an issue (since decoders are pretty standardized), as some others in this type of testing, but it still needs to be considered.

Interesting. Thank you for spending the time to perform the test. I plan to do my own as soon as I get a round tuit.

The aliasing artifacts of a zone plate test indicate that the 5D2 is only reading one out of every three lines and applying some noise reduction in an attempt to suppress the moiré slightly.

Nigel, your post struck me as rude and belittling.

A number can only measure one aspect or dimension of image quality. Higher numbers equal higher quality images only with respect to the specific aspect of quality measured by that number. Problems occur when people assign greater scope to a number than it measures.

I find that the most common reason to throw measurements out the window is because they are misunderstood.

For example, someone might throw out Tom's test because the 5D2 has more "clarity" and "stronger colors" than the XH-A1. The mistake is thinking that his test was intended to measure "clarity" when it wasn't. It could be much higher MTF at lower frequencies, less lens flare, or the higher color accuracy of Bayer filter arrays over prism-based systems, or another factor.

Another artist might throw out Tom's resolution test because their eyes tell them that the 5D2 has more "pop" than the XH-A1. The "pop" can just be thinner depth of field.

Again, no single number is going to quantify all aspects of image quality. Just one aspect. To quantify the many important dimensions of image quality, including depth of field, color accuracy, saturation, contrast of all frequencies, etc. would require a lot of numbers, not one; and even then will probably miss a few factors.

Perhaps all of the factors that makes the HPX170 look better to you could be quantified with numbers that are already common in imaging systems, such as Delta-E for color accuracy, etc. Or perhaps a measurement has not been invented yet that can quantify it, though there are a lot of them out there.

It seems to me that "most people" like heavy noise reduction and strong aliasing artifacts, so they would prefer the 5D2 over the XH-A1 for those reasons. But they also like oversaturated, bright images, low dynamic range, and poor color accuracy: the XH-A1 matches those descriptions better.

That depends on how you setup the test. The depth of field, color accuracy, dynamic range, low light, etc. are more filmic than the XH-A1, but the noise reduction, aliasing artifacts, and the auto shutter speed are not very filmic at all.

Moiré is only the worst, most obnoxious, and universally disliked manifestation of aliasing. But there are many more and varied aliasing artifacts that affect even shallow DOF shots, including what may be described as jaggies, stair-stepping, unwanted sparkling, "snap to grid", wavy lines, fringing, popping, strobing, noise, or false detail.

Of course, those are precisely the "artifacts" that so many viewers love and enjoy. Most describe them with positive terms such as ”crunchiness”, ”sharpness”, etc. Other photographers perceive the artifacts as highly unnatural, distracting, and a telltale ”digital” look.

I'm surprised that so many people here are in love with the image quality of the 5D2 because I dislike so much about it. I only put up with the image quality problems (shutter speed, aliasing, and noise reduction) to get the thin DOF and low light capability.

Just because the resolution of captured still images from the 5d2 may not be as good as some other video cameras, doesn't necessarily mean that the 5d2 video doesn't appear as sharp to the viewer. Unless you're looking at uncompressed frames (and none of the tests listed in this thread, do that), all video straight out of each device is compressed in some fashion. Even the SI video cams use Cineform, which is nearly lossless, but still a wavelet compression. Both the bit rate and compression algorithm affect the way the viewer perceives the video, just as much as actual resolution, color balance, DOF and other factors. The 5d2 captured still images may not look as good as some other video cameras, but the net result may still look better as a video.

<<I'm surprised that so many people here are in love with the image quality of the 5D2 because I dislike so much about it. I only put up with the image quality problems (shutter speed, aliasing, and noise reduction) to get the thin DOF and low light capability.>>

Maybe I'm just lacking in visual and artistic acuity but I've been writing and producing professionally for years with some of the best film and video gear, and footage from the 5D MKII looks gorgeous in my work.

I'm shooting field work and narrative, Nikons on crane dolly and tripod and Canon glass with IS handheld and aside from Canon's unfortunate implementation it looks better than my best work with any other rig including cobos with the P&S system.

All the footage I've seen utilizes the thin DOF, low light, or lenses that aren't available on similarly priced video cameras. Perhaps those are the reasons you think it's gorgeous? Because in other situations (deep DOF, ample light, and normal lenses), I find most aspects of the image quality very poor compared to the XH-A1. Until the image quality issues are fixed, I'll have to keep both cameras.

Daniel, how are you handling your footage in post, out of curiosity?

Originally Posted by Nigel Barker
Do you think that your figures for the 5DII actually mean anything in real life? How do you explain the difference between what your test measures & even the most casual recording on the 5DII by users without any talent? Perhaps some kind of optical illusion that fools us into thinking that the picture quality is better than it really is?
It wasn't meant to be rude. I am genuinely bemused as to how these test results are so out of line with what everyone else's subjective opinion of the stellar picture quality of the 5DII. Movies of any kind are in themselves an optical illusion that fools the eye into thinking we are seeing real moving images.

You appear to be out of step with everyone else who thinks that the picture quality of the 5DII is amazing. Are we even talking about the same metrics here? Universally when I have shown people footage shot on the 5DII there has a been jaw-dropping 'Wow!' expressed. I have never received that response with any DV or HDV footage that I have demonstrated.

Maybe it's my fault in some way but in common with several other people posting on this forum I have moved to the 5DII from the XH-A1 just because the image quality to me looks so much better.

Here is a random clip recorded hand held on a 5DII with deep DOF & ample light that surpasses any XH-A1 footage that I have shot. What is poor about image quality that you see here but I don't? Please bear in mind that from this link that the Full HD version of this clip it has been compressed from the original 40+Mbps & 250MB file size to 7Mbps & 44MB with corresponding lower bit rates for the smaller video sizes.
Barkers Videos- powered by SmugMug

Later today I will upload the original file that came off the camera so that you can critique that.

Cheers

Nigel

Also I am in jaw-drop-mode after seeing footage from the MKII -- but technically it might be due to other reasons than what Tom tested -- i.e., full frame DOF, or something, I dunno. Still, it would be nice to see a linear A/B test comparing all those cameras side by side.

-- peer

It's coming, round II.

I wish you luck (although the testing effort is very interesting). You should keep in mind that even the way that MPEG determines the quality of video, is by subjective evaluation. They use standardized viewing conditions and ask the audience what they think of the quality of the video.

I have a hard time fathoming why people are even arguing this matter. I know why I decided to go with the 5DII. It gives me the opportunity to shoot incredible looking HD footage with some amazing results in regards to DOF for under $4,000. The low light capabilities of this cam are amazing as well.

I don't believe that you can get this type of result with any other cam/adapter setup in this price range. Not to mention it completely cuts the need to set up tons of equipment before a shoot ie. setting up a rail system that has a 35mm adapter on it and then adjusting backfocus, meanwhile I have already pulled my cam out of my bag and am shooting before any lights are even put up. Try trecking the xh-a1 up a 14er with all the equipment necessary to obtain the shots that can be achieved by the 5D with nothing more than a lens that mounts right to the cam.

If I were to try to put together a system with the H1, or any other cam in this price range, that shoots footage comperable to the 5DII in terms of DOF, I would be spending close to 6K. To me the decision to buy the cam was a no-brainer and I am completely pleased with the results even with the lack of full manual control. Once you get to know the camera a bit, it is not that hard to get it to do what you want and create gorgeous shots, even with Canon lenses.

If Canon did add full manual control through a firmware upgrade, I won't hold my breath, then I would honestly consider this camera to be my dream camera.

My .02

Ryan, the purpose is informational. The reason for doing the Imatest is because the moire in bright, sharply focused situations seems severe enough to adversely impact resolution, or originates from a lower than expected starting point.

I think we're all in agreement we've seen some really spectacular footage. The goal is to expand the uses to include workable video in the broad sense as well.

I just now realized that you are in Denver. Hello there neighbor!

If the tests are in fact to figure out ways to overcome some of the cams shortcomings then by all means test away! I personally find the tests interesting, but I think that some people might take them as trying to prove the cam to be an inferior camera for video. I was just trying to point out that even with the shortcomings of the cam, I am still incredibly pleased with it's performance based on size and cost. That's all.

I hand it to my editor and say "handle this". :) He's a more talented and experienced editor than I am. In December I helped him get started using CoreAVC and MPEG Streamclip to transcode to uncompressed proxies. Then I think the new Quicktime version came out and we stopped using CoreAVC. Then we switched to rewrapping as MP4 files and then transcoding to Cineform.

Again, the thin DOF, low light, and lenses are what give the 5D2 images that "wow!". To me, that is true in spite of the aliasing and other problems. Without those advantages, the problems make me prefer the image of cameras like the XH-A1.

For one, I see aliasing artifacts. They appear anywhere in your footage that there is very fine detail. You probably don't see them as artifacts, but as "sharpness", "zing", "resolution", or "crunch". If you want to know what 5D2 video would look like without aliasing artifacts, just try shooting a raw file and resample it to 1920x1080 with a quality lanczos algorithm. You might find that it is a "creamy", "soft", or "dreamy" look compared to an image with aliasing artifacts. To you that might be a negative, but it is the look that I prefer.

If you want to read more about aliasing, I suggest the following:

http://www.pbase.com/jps_photo/image...3/original.jpg
Aliasing and Moire patterns
Sigma SD9 Digital Camera Review: Test Results & Conclusion

Most people like aliasing artifacts, but I don't. I would rather have an ounce of real detail than a pound of false detail. The 5D2 in still mode is correctly anti-aliased. It's only in video mode where Canon is forced to forego anti-aliasing because they have to skip two out of three lines when reading the sensor. False detail is the natural and unfortunate result.

Preference for aliasing is like many other aspects of an image. Take sharpening for example. Some people prefer just a slight USM. Others prefer to sharpen until ringing/halo artifacts become very strong. All of our local news stations, for example, go way overboard on the sharpening, which is on top of the aliasing and interlace artifacts. Many syndicated shows, too, have a ton of aliasing and oversharpening artifacts.

Saturation is another one. Some people love vibrant blues, hyperrealistic greens, and Velvia reds. Others prefer accurate colors. Desired dynamic range, too, varies by taste. Some like ultra-high contrast with blown whites and clipped blacks. Others prefer a more low-contrast image to keep more detail in the highlights and shadows. It's a matter of taste.

The 5D2 has the worst aliasing I've seen in any video camera in at least the last 5 years, but the thin DOF, low light, and lenses are the reasons I keep using it anyway.

i agree with you Daniel. I just shot a documentary intercutting footage between an XH-A1 and 5D. This was shot in and around a forest.

We shot at night in a cabin lit by candlelight, and the 5D was amazing. Likewise for close up shots of flowers/plants and insects in which I could use macro lenses and get incredible depth of field control.

But for deep depth of field shots of landscapes in full daylight, the XHA1 footage was clearly more detailed and did not contain the nasty artifacts you mention (I too have issues with moire and other image problems the 5D exhibits).

The 5D is a wonderful tool but it has a number of shortcomings which mean in most situations (for me anyway) it needs to be paired with a more traditional camcorder with manual control and fewer foibles.

Thank you for performing the test Tom. Personally I don't find the figures surprising, and do appreciate your time. It's interesting to know.

Pics are up
 

It was threatening to rain so I worked fast, no special attention given to any of the cams, except careful focus.

Notes:

These pics are about 500KB ea. You have to view them in your browser at 100% so you don't alias them yourself. The 5DMkII pic I converted from PC-RGB to Studio RGB with Cineform. All snapshots were video grabs from Vegas except the setup shot and area of detail, both by the 5DMkII.

5DMkII - 50mm prime, shot at F/5.6
HV10 - shot at F/4.0
EX1 - shot at F/4.0

The setup
Area of detail and focus(from yesterday)
The 5DMkII
The HV10
The EX1

Thanks Tom.

Wish I had an EX1 to complement my 5D. These shots clearly bear out your numbers. Different tools for different purposes.

Josh

I think, that color latitude from 5d is much wider, there are more color generally, compared to Ex1 or any other HDV camera. I'd put Canon H1 with x16 lens second to it, A1 goes next, EX1 follows up, IMHO>>
I've shot with Ex1 many times already, my assistant owns it, we've tested different cameras side by side, shapness from 5d is just pleasing for a human eye shown on an HD panel,
EX1 produces "dry", flat picture, (subjective to our eyes off course), no matter how much we tried to get a good color from it..
(that's why they went with Canon cameras for a blockbuster Crank, I guess :) using Ex1 only for slow mo shooting 720 60p, I guess)

Unruly moire was the first negative I noted when I shot with the 5D in video mode: it was on shingles, brick facades, anything with patterned detail that was even partially in focus. Now my eyes immediately go to it, like the other night when I shot folks in a concert audience -- a very wide shot -- and the lady on the far left was wearing a shirt with horizontal stripes. The colored moire as the camera moved in relation to her was not a pretty sight.

On the other hand, the shots of the band and closeups of the crowd - lit only by the reflection from the stage lights - were incredible. It will have to do until I get my RED Scarlet later in the year.

Tom, thanks for posting those images. My wife & I spent a long time laying in bed looking at them very carefully (it is Saturday morning after all:-). They all look different & I guess that different 'looks' appeal to different people. The 5DII looks sharper but has crushed blacks & moire patterns but it has an overall 'glassy' shine & richer looking colours that made us think that the image is better. The EX1 may have a the more natural colour but the 5DII is 'punchier'.

Photos & videos are all an optical illusion anyway as they are only representations of real life. Whether the more detailed image fools the brain that it is more lifelike or whether it's the shiny sharpened image that the brain interprets as the more realistic is clearly open to debate & may simple be a matter of preference.

One possibility that should be considered is that as the image has been extracted from a video & processed differently for each different recording technology that we are not really comparing like with like. It is also a rather artificial test for a video camera as we are not looking at the video as these photos are still captures that nobody ordinarily would see.

Interesting nonetheless. I shall have to try a similar shoot out between a 5DII, a XH-A1 & a HV30 but processing the video in FCP.

You bring up valid point about punchiness/color, that I think was mistake on my part for not addressing with the shots. Some time ago I had made adjustment of the HV10 to reduce it's contrast and saturation in relation to EX1. The HV10 only has coarse adjustments for this, but the default settings take on something of a "neon" appearance otherwise.

The 5DMkII was using the "Standard" picture style for the pics. I believe it would more closely align with the EX1/HV10 if I had selected the "Neutral" style. To be fair, the EX1 can be enhanced for a more vivid look either in-cam or in-post.

There is also a fair amount (not excessive) of sharpening that accompanies the 5d2 standard picture style, that you can see as a darkened halo outline around the hood/engine bonnet of the pictured vehicle.

I first noticed this when I ran the Imatest with sharpness at 3, it was reporting sharpness as 21% over the standard for a 2-pixel sharpening radius. I reran the test with sharpness at 0, and now it reported sharpness as 15% under, and that's what I posted for the Imatest screen cap in the first post of the topic. But it should be noted, Imatest returned the same MTF50 corrected numbers either way. In any case, 3 may be too much but 0 is not enough.

Daniel Browning mentioned this earlier as well, and unless there is a fix for it, the achille's heel.

Daniel stated that the 5DMkII samples every 3rd line. I've been thinking about that ever since. More than just a processing limitation, it could be physical limitation to manage heating of the cmos sensor. While the Imatest reports the resolution numbers as low on the horizontal, the vertical is actually above the average.

I think it's important to remember that perceived sharpness/resolution is relative to the content of the image - i.e. could you tell the difference in sharpness between any of these cameras when filming a flatly lit white wall? I think several aspects of the 5D's default image rendition contribute to greater perceived detail than is clearly present in objective measurements like these.

First off it shoots a fairly high-contrast image, even with contrast turned down in the image settings, and higher contrast tends to emphasize edges better than low contrast. Under similar conditions I also find the 5D to almost always produce less noise than my XHA1, which results in a very 'smooth' looking image - I believe this is also a function of the h.264 encoding which tends to smooth out noise in low-detail areas of the image more than MPEG2 does. This again increases the perceived contrast between sharp lines and the surrounding areas of low detail - so detailed areas and hard edges look 'sharper' than their measured resolution would suggest. This effect is further enhanced in shots which take advantage of the ability to achieve shallow depth of field - soft focus elements in the image make the focused areas appear sharper than they may really be.

All if which only goes to reiterate what so many people have said so far - the camera shoots an image which is more appealing than the other cameras for a lot of people. If that's you, it doesn't matter how the camera performs in these tests.

The solution is a softening filter. My tests show that a Tiffen Soft/fx1 works well on an 85mm lens. I plan to get an fx2 for my 50mm lens and an fx3 for the 24mm...

The filter you choose is a compromise. A less aggressive filter will not remove all of the moire. A more aggressive filter will soften your edges. Unfortunately, there are no optical brickwall filters, so don't expect perfect resolution and anti-aliasing.

This is also my thinking -- a filter would be the best solution of keeping the balance between a sharp image and getting rid of the moire.

-- peer