Assessment of the 5D2 by the BBC (Alan Roberts)
 

Alan Roberts assessed the performance of the 5D2 (and many other cameras) for the BBC. Their official web site is not updated yet, but I have been allowed to host the white paper here:

BBC R&D White Paper WHP034 - Alan Roberts

A zone plate was used to measure the aliasing of the 5D2:

http://thebrownings.name/WHP034/imag...zone-plate.png

Here is what the zone plate is supposed to look like:

http://thebrownings.name/WHP034/imag...zone-plate.png

The conclusion is "pictures with aliasing at the levels seen here are not acceptable as HDTV."

This one sux.

No, the 5D2 does not "sux". I can understand that the artifacts and image quality problems on the 5D2 are disgusting to you, but kindly remember that there are plenty of people who don't even notice those things.

For every one person like you, who hates aliasing and other artifacts, there are a thousand who don't care, and are happy to have aliasing and compression artifacts if it means gaining thin DOF.

That's why I'm happy to continue using my 5D2 for clients that don't care about things like resolution, aliasing artifacts, compression artifacts, etc. The last wedding I shot was completely 5D2. We brought two XH-A1, but never ended up even touching them.

Don't let this White Paper discourage you from buying the 5D2. There are many projects where the 5D2's advantages (thin DOF, low light, lenses, size) outweigh its terrible image quality problems.

Thank you for speaking up Daniel.

Thanks Daniel, and for others may be interested in the Panasonic HPX301 details from your site too http://thebrownings.name/WHP034/pdf/...nic_HPX301.pdf
Steve

It also says the test was done using a preproduction model, any chance we might get an update?

I've shot a sine zone plate with a production 5D2 (Dec 08) myself and I got the same result as Alan. (Graeme Nattress analyzed the zone plate, too.) I don't think anything changed from his preproduction model to the production model.

This is very interesting and useful information, particularly if you read some of the other camera assessments.

I hope Canon sees this, as the tester seems to suggest some of the problems have fairly simple solutions.

The small Canon (video) cameras actually come out looking pretty good next to some pro cameras costing 10 times as much.

-Mike

This is another case of "know your tools."

Yes, the 5D2 aliases - especially when you have high frequencies in the vertical dimension.

Doing a documentary on fine lace? Choose a different camera. Showing images of people's faces with the background out of focus? Choose the 5D2.

The "Beeb" is notorious for testing cameras and giving a pass/fail grade. This makes it easier for their programming people to accept/deny content without having to manually check the technical quality. Given that the 5D2 doesn't do 25p, it's no surprise that they would want to reject it.

If shooting for the Beeb, choose a different camera. If shooting a 5D2 for yourself, try to avoid fine patterns of high contrast. And stabilize the thing to avoid rolling shutter artifacts. And don't stare at the sun...

BTW, if you can film an image of Princess Diana's ghost, the neither the Beeb nor any other news channel on the planet would care if it was shot on your kid's cell phone...

I have a lot of respect for Alan and he assisted me with my choice of the HPX301 as he sent me the BBC test results.

I am sure the canon is a superb stills camera but for broadcast video content we have certain rules, judging by the amount of bad pictures and artefacts that are present on numerous tv channels here in the UK it is more important than ever that standards for HD are kept up.

Logically, the above statement does not preclude pictures without aliasing. How many times have you seen that particular test chart on your favourite TV programme?

It would be a shame (for anyone) to throw the baby out with the bath water.

Having said that, I haven't read the white paper and perhaps there are other issues (aside from 30p).

If somebody should slip the BBC some good work shot on the 5D2, would they spot it?

If so, stick to ITV - the future [cringe]!

Very interesting White Paper. Yes I agree if you are shooting for the BBC ( Which I do about 3-4 days a week) then the 5d2 is not the camera for most programming. But I have just completed a 90 min feature film on one which is being graded at the Mill & they have already seen the rushes & think they look superb ! That is neither here nor there though & it does not get away from the point that the 5d2 creates great images.
It's funny though as the BBC will accept footage shot on a PD170 or a Z1 & to the best of my belief the 5d2 is better quality than both of those cameras.
I believe that the 5d2 definatley has it's place in film & documentary making and as somebody has already mentioned if you gave them a breaking news story shot on the camera I'd eat my hat if they didn't accept it!

I agree with Jon, know your tools.

Personally, I have had success removing aliasing by using Caprock Anti-Moire filters. They reduce the aliasing/moire without making any noticeable compromise on resolution. I can't completely get rid of the problem, but I can make it disappear in the rare occasions where it may be noticeable.

I do think it's ridiculous to judge a cameras capabilities on this particular test, which does not clearly reflect the cameras performance in practice. As filmmakers we don't spend our time shooting resolution charts, how can this alone possibly represent our tools?

The BBC will accept PD 170 & Z1 for SD programmes and described as SD in their HD programme guidelines. The video feature seems have been fitted to the 5d2 for shooting news for the web by news stills photographers. Regardless of this, on an important news story any image is acceptable, including mobile phone cameras, so I wouldn't use that as claiming anything.

People have to do their own tests and decide if the camera is appropriate for the film they're making. For some people it will be fine, while for others the flaws will be unacceptable.

I believe the Z1 is an HDV camera that can shoot dvcam does this mean that the BBC doesn't accept it as HD even in it's HDV mode? Is that what you are saying?

That's right - the rules apply unless the rules don't apply.

Canon's a bunch of engineers. They know these problems, and because of these issues probably underestimated the camera's impact. Another example of the big things being much more important than the details. The BBC will accept 35mm adapters on 1/3 cameras, which is rather pitiful compared to the 5DII.

We know Canon will come out with a much cleaner version of this technology. Thank goodness for Scarlet. On their own the Japanese companies would stretch this out for years.

In many ways it is better, but in some ways it is worse. For example, if you compare a 10-year-old computer with a small monitor and anti-aliased fonts, then compare it with a brand new computer with a large monitor and anti-aliased fonts, the new one will look much better. But what if the new one does not have anti-aliased fonts? Some people don't notice at all. But others are driven absolutely crazy by it.

Same thing with the PD170 and 5D2. The former at least has a natural anti-aliased look, even if it is low-res, low contrast, etc. The latter cannot achieve a natural-looking image if anything is in focus because it's riddled with aliasing artifacts. But again, most people don't mind (or even see) the aliasing artifacts. Some are driven nuts by it.

Furthermore, in the case of the BBC and this White Paper, the main problem cited is not the aliasing artifacts themselves. It's the compression artifacts *caused* by the aliasing. It can be helped greatly by having only a tiny sliver of the image in focus, but if you have any deep DOF shots (e.g. wide angle), the compression artifacts go crazy. They are bad enough already on the out-of-camera 5D2 files, but they get much worse when compressed again for broadcast.

Aliasing isn't really new with the 5D2. Before, people were adding aliasing artifacts in post (mostly on accident, but some of them did it on purpose because they like the "crunchy" look), but now it comes aliased out of the box.

You may not notice the aliasing artifacts in your footage, but some do. There has been no 5D2 clip ever made that is both sharp and alias-free. It's either too soft (out of focus, diffraction, no small details, etc.) or it has aliasing.

The chart just allows more precise measurement of the artifacts. They do show up in real footage too.

Aliasing artifacts are explained in this metaphor. In real life, when you pour two liters of water into a one liter container, water spills out and makes a mess. But camera design is different: when you pour two liters of water into a one liter container, the water folds back on itself and corrupts the entire container. The amount of water is the level detail (spatial frequency), and the volume of the container is the number of megapixels in the camera. Aliasing is the corruption. Anti-aliasing filters reduce detail down to a level that can fit within the pixel resolution.

Aliases are a certain kind of image artifact; they can be described as jaggies, stair-stepping, unwanted sparkling, "snap to grid", wavy lines, bands, patterns, fringing, popping, strobing, noise, or false detail. Some photographers desire aliasing artifacts and describe them with positive terms such as ”crunchiness”, ”sharpness”, etc. Other photographers perceive the artifacts as an unnatural, unwelcome ”digital” look. The only aliasing artifact that is universally disliked by all photographers is moiré.

This image shows aliasing artifacts. Again, some people see no difference below. Others see "fake, digital, yuckiness" vs "natural and smooth":

http://www.pbase.com/jps_photo/image...3/original.jpg

This site has a great explanation of aliasing:

http://www.wfu.edu/~matthews/misc/DigPhotog/alias/

Aliasing is also described in this SD9 review:

http://www.imaging-resource.com/PRODS/SD9/SD9A12.HTM

To me this is just very useful information about how best to use this camera. I never had any illusion that it would not have some serious compromises.

Some remaining questions (to me):

1. Not good for use with heavy compression, but knowing exactly how much this effects something like DVD compression, which is much less than HDTV compression, would be useful. There might be optimizations possible.

2. How useful the technique mentioned in connection with some of the other cameras (softening the image slightly before it hits the sensor to lessen aliasing artifacts) might be with this camera. Even if it cannot make it suitable for heavy compression HDTV use, it may still be very useful for other uses.

3. The largest part of the problem seems to be from using de-bayering optimized for still use with moving images. I wonder if some sort of external software might be devised to partially 're-de-bayer' the images, if the original transform is known, and known camera settings are used. Plus, there is a programmer already mucking about in the cameras internal code, I wonder...

It looks like this cameras best technical use is actually the very thing people were most interested in - shots like closeups with very soft backgrounds. Not so bad.

-Mike

I am also wondering what things like pre-editing conversions might do to help or hinder-- things like Cineform's codec where the there changes to correct some luminance issues.

The keepers of the gates don't like your key.
 

Doesn't the BBC paper say the camera they tested is 29.97?
It's 30 true fps as far as I can tell.

Regardless... the 5Dmk2 is a really wonderful tool. I've shot with it twice now and can not believe what it's capable of for the price. And knowing that the EF L lenses are going to be usable on Scarlet (or RED should Scarlett never surface) and the 7D and whatever comes out next from Canon, I couldn't be happier.

Engineers can throw all the negative charts, graphs, papers, and blogposts at me that they like.

Ex: attached is a 5D capture from a video shoot from this past Sunday. I find it a bit more visually satisfying than the BBC's chart. If the aliasing artifacts bother you, I apologize.

Yes, quite a few broadcasters only allow a percentage (15%) of each HD programme's content to be shot with SD formats. Although, broadcasters will make exceptions if the subject matter makes sense, for example shooting with a Z1s on "Deadliest Catch", during which a large number of cameras get written off on each series.

The SD formats are listed on P4, which includes HDV.
http://www.bbc.co.uk/commissioning/p...d_delivery.pdf

Two years ago I found a previous paper that Alan Roberts wrote about the Sony A1 very helpful eg it gave useful guidance about the optimum exposure settings for this camera.

The moire test seems fair comment. Hardly anything I have shot with the 5DmkII so far has been free of aliasing and sometimes eg storefront shutters at night, it has been quite wild.

This camera is the first of a new generation. Within a year it will hopefully be made obsolete by new film cameras based on this technology.

Let's hope that Alan's paper helps to prod the engineers at Canon, Sony, Nikon etc into addressing the shortcomings of the technology.

I do think Alan's assessment is fair. I've encountered really bad aliasing on wide angle and highly detailed shots from the 5DmkII. Have a look at the buildings about a third of the way into this video One Week in Urumqi - Uighur ethnic unrest in western China on Vimeo

I am one of the news photographers that Canon originally aimed the 5DmkII at, indeed I was one of those calling for video on DSLR cameras for years. I think for news even the BBC will accept 5DmkII footage, its all about being there at the right place, right time. As the DSLR video thing has taken off the criteria changed and now many others want to use the camera for other uses. Canon didn't see this coming.

I do think it is only a matter of time before these issues are addressed, but for now we have to live with it.

Dan

I noticed the beautiful footage but had to look for the aliasing.

As far as Canon figuring out what to do with this huge interest in dslr video, I doubt they will make good choices. It crosses division lines in a large, conservative company. Canon does much better on the counter attack than leading the charge.

There's a chance, however, that they will get closer to matching red technically with the next 1 series. That's a technical challenge, not a product design and positioning decision.

Anyway, someone will exploit this new area.

Agreed. Given that the Optical LPF in a DSLR is tuned for photos, and there are way more photosites than we need for video, it's just a matter of adding enough processing to do good 2D digital filtering. No new inventions or miracles needed.

Scarlet is 3K wide and is intended for 2K or 1080 output. It might have as much as 2.4K resolution.

The 5D2 is 5.6K wide. With better digital filtering, it should be capable of virtually perfect 1080p.

Given that silicon gets faster and cheaper all the time, it shouldn't take long.

Rolling shutter is the tougher nut to crack on large sensor cameras. To cut the skew by a factor of five, the chip has to get five times faster. The more sensible solution, IMHO, is to add another layer of transistors that will allow CMOS with a global shutter. That will take a few more years, unless RED has something up their sleeve that we don't know about yet...

I'll take FF35 with aliasing over claustrophobic 1/2" or 1/3" any day...

Why are the guidelines for broadcast video so strict? Most broadcast footage looks like crap anyways. People don't care about aliasing, especially on 20" SD televisions. Seems like a bunch of pompous BS...

Interesting, I looked at their website and saw four variants: 4.0, 5.6, 8.0 and 11.0.
Which one do you recommend for the 5DMKII?

I think it's due to the degradation that takes place during transmission, meaning that any issues that are there in the original footage will be amplified when it's compressed and sent down the satelite.
I know lots of people that say that their HDV footage looks fabulous when camera plugged straight into HD TV set, but once broadcast it looks rubbish!
Steve

I agree- I clicked on the link to look for objectionable aliasing, but ended up blown away by the beauty of the cinematography. Well done.

This is really missing the point about aliasing. It's danger is not that it looks objectionable at first sight - it may be barely visible - but it could show up very badly after further compression, such as on broadcast transmission.

The original test report explains why:
Steve Philips and Daniel have got it right - footage may look great when played straight to TV, or even through an entire NLE, but show unacceptable degradation because of aliasing when transmitted. A second camera may look identical on straight viewing, but lack of aliasing mean a far better image makes it to the viewer.

The only way to predict this in advance is by the use of test charts, and I agree with the BBC report - the 5D2 shows an extremely high aliasing level.

There's no doubt such as the BBC may use 5D2 footage if it was all that was available of a unique event, and to hell with the quality. But this test is more interested in evaluating cameras for planned and commissioned work, and there is no sense in adopting a camera whose use would involve having to increase channel bitrates to cope with camera aliasing.

I think is something many people forget about, digital material can also degrade as it goes down a few generations due to compression. Also, the different codecs used during the chain can create problems. TV programmes can go through a complex post production process and I don't think Canon were planning for this when making this camera for the news guys.

This is a first generation camera and I suspect there may be two approaches to the HD DSLR: using a codec which requires less processing power on basically a stills camera or a shooting stills on a camera which has the processing power to handle a codec for motion pictures without artefacts. RED seems to be going for the latter, but it appears to be more expensive and larger than a stills camera design.

A smaller sensor than FF35 for HD work does make a lot of sense. There are focusing problems with the larger sensor size and a S35 sized sensor allows you to also use cine lenses. I know some people have a shallow depth of field fetish, but you don't need a FF35 DOF to create great images. I was watching "I am Cuba" the other day and it's more cinematic than most of these shallow DOF films - they used wide angle lenses with a large DOF in many shots.

I expect the next generation of these hybrid cameras will be better.

I noticed major aliasing artifacts on a nationally broadcast football game yesterday. These were evident when they switched to the little POV cameras that fly around above the field. Artifacts on the white lines on the field were most evident in wide angle shots but went away when they zoomed in. I also noticed the same thing when the sports show this morning showed a high school football game shot with a cell phone camera. My skepticism about this being an issue is now gone. I still think that there is a place for HDV and cameras like the 5D II in broadcast TV. I've never noticed any problems, for example, with those Discovery Channel shows that are shot almost exclusively on HDV cameras.

Pat

FWIW, that was only true with film. In digital, you can stop down a large sensor, increase the gain, and you will get the same image as a small sensor. (Same DOF, brightness, noise, diffraction, etc.).

Hi Martin,

We need something a little stronger. The lower the number, the higher strength of the filter. There is a 2.0 and a 1.0 that are not listed on the web page. I use the 2.0 myself, but testing the 1.0 too. I am in contact with someone who is using the 1.0 and I will try and see if they can comment on its performance.

To reduce the moire / aliasing we must filter out high spacial frequencies. The trick is not to blur everything all at once, just areas causing problems.

It is very rare that I use the filters. To be honest, this problem has not bothered me enough to invest a lot time finding a solution.

Greater sensitivity has been one of the big advantages for the larger sensors (assuming a similar number of pixels), but in practical terms the lenses do start to become much larger. This applies more for zoom lenses than prime lenses, even 35mm film cameras become bulky once they're mounted onto the camera.

I suspect it would also depend on the sensors being compared if they would give the same image in all respects.

Its important to remember that the artifacts will be very much less in a year or two. If you don't do photography it's easy to forget that these are still cameras that would cost about the same without the video feature. There's apparently little added cost foMr video. Most buyers have no interest in video.
Canon, Sony, Nikon or someone else will release this type of device without these flaws. Like red it will be a video camera first. It would be great if Nikon did that. They don't have to have discussions with the managers of the traditional video lines. I wonder how Canon's video division feels about the 5DII.

Thanks Daniel, this was the first time I heard of such filters.
I also had no real problems with aliasing so far but its always good to be prepared.

Ah - but look at the test reports from the same source for the Z1 - http://thebrownings.name/WHP034/pdf/...VRZ1E-FX1E.pdf .

The Z1 is not up to the standard of true broadcast cameras, and does show an amount of aliasing - but nowhere near as much as the 5D2. What that means is that what emerges at the viewers screen after compression and transmission won't be as good as from a higher priced camera - but at least it won't be too different from what is seen during editing etc. And that's the real problem with the 5D2 for broadcast.

It's quite likely that at the TV station some 5D2 pictures may look significantly better than some from an HDV camera like a Z1. But also very likely that after compression/transmission the 5D2 pictures will degrade more than the Z1 pictures and end up looking significantly worse. Wecome to the wonderful world of aliasing and digital compression! This is why cameras can't be sensibly assessed for broadcast pictures simply by looking at pictures alone.

For many cameras, pre-filtering may improve aliasing a lot (at the expense of sharpness), but the 5D2 results show quite a lot of nasty things going on. The multiple centres to the alias rings and their coloured nature indicate many causes, not simply those associated with detail too sharp for the chip. As the report also says:
In other words, soft filtration will be only of limited use in getting rid of aliasing for video with this camera. Many of the problems are caused not by the normal causes, but by the processing used to downsample the chip resolution to HDTV resolutions.

One day there may be a camera that will produce good stills and high quality video, but it's not here yet. And don't be misled by what the pictures look like straight from the camera, it's how they will stand up to further processing that is key. A user may think they are not troubled by aliasing - then get a phone call saying "that material you supplied us with yesterday...... yeah, looked good, didn't it......but, the funniest thing, I saw it at home and it looked really bad compared to what that other guy shot on his HDV camera......."

Diffusion filter should be built in
 

One thing I notice, reading through these various reports, is the pretty consistent recommendation that a diffusion filter that creates the optimal circle of confusion for that system be used.

While not so useful for the 5D, it seems to me that when these sensors are used in newer systems intended more for video use, it would be wise to include such a filter - selectable on or off for cases of soft lenses, or for still use - right in the stock system, much as ND filters are commonly included now to solve THAT consistent problem.

The manufacturer should be best positioned to pick the best option for this filter.

-MD