EMMY nominated camera tests is a new one, but part 3 covers rolling shutter and skin tones.

The Great Camera Shootout 2011: SCCE ~ Episode Three | Zacuto USA (http://www.zacuto.com/the-great-camera-shootout-2011/episode-three?utm_source=camerablast&utm_medium=sara%40zacuto.com&utm_campaign=blast)

The next test - shooting the same challenging scene with various cameras - sounds interesting. They won't reveal the cameras until the end.

Regarding rolling shutter, I wish they would have measured it numerically with a strobe test and then shown a real world comparison. The spinning contraptions weren't numeric or real.

I love these. They're very entertaining, informative, and fun. I wish they tried a little harder on scientific method and factual accuracy, but hey, it's free.

Is there a numerical standard for rolling shutter? I suspect how important artifact is depends on what you're shooting and you make a decision on that basis. So, if you're shooting a fast crabbing shot through railings, the DSLRs aren't the best camera to use, unless you wish to use the skew as a visual effect.

The spinning device is real if you're filming an aircraft propeller, but also does test things to destruction and so you do get a sense of the resistance to the effect that each camera has.

I note they say "Some updates to certain cameras such as “S Log” on the Sony F3 and “CineStyle” for the Canon cameras were not available at the time of these tests.", then later

"After the screening, Raine went on to emphasize that the real difference between the cameras in this comparison was how much information they record and whether or not the cheaper cameras could hold up to substantial color correction in post."
Surely, for such as the F3, whether or not S-log is used or not will make a MASSIVE difference to such as post colour correction. Doesn't that make the tests as they stand a bit of a waste of time?

No GH2, sad!

After the screening, Raine went on to emphasize that the real difference between the cameras in this comparison was how much information they record and whether or not the cheaper cameras could hold up to substantial color correction in post.


I suspect that would add to the complexity of a mass test, but it is important if you're going to do much heavy duty grading in post. DP Nick Paton did a test that might be something to consider as a possibility when he compared 7D and the Alexa.

Alexa vs 7d latitude tests on Vimeo

Of course, this is the last of three Zacuto "shoot out" videos, so you'd need to make any judgements a camera on all of these, plus other sources and your own tests.

I suspect that would add to the complexity of a mass test, but it is important if you're going to do much heavy duty grading in post.
Yes, it will create practical difficulties (not least that I believe the option didn't come out until later!) but whilst S-log may not make much difference to some aspects of performance, to things such as post grading the use of native codec versus S-log is likely to make a vast difference.

In spite of the practicalities, then what is the use of any test that may say the differences between two cameras are small (the limiting factor is recording format, not camera head) - but when used in S-log mode one of them becomes vastly better?

One of the main things that really struck me was the rolling shutter on RED.... whay worse than I thought it would have been.

In spite of the practicalities, then what is the use of any test that may say the differences between two cameras are small (the limiting factor is recording format, not camera head) - but when used in S-log mode one of them becomes vastly better?

Yes, in a comprehensive testing this should be covered, although there are a lot of apples and oranges being compared in these particular camera tests anyway. By next year, if Canon or RED have brought out a S35 camera that is in the F3 budget range, this would be an increasingly important deciding factor.

The differences between the RAW version and other codec versions of the same cameras could another factor.

With the number increasing numbers of specialist "motion" large sensor cameras coming out next year, I'm not sure how the DSLRs will quite fit into another shoot out, especially if there is no real improvement in these cameras or new models.

Pretty stupid that in the rolling shutter test, all the DSLR's were zoomed in closer. Offcourse they will have more rolling shutter when closer to the object because then the object moves faster. Daa.

The bad rolling shutter on RED didn't really come as a surprise, there are many movies and music videos that are shot on RED that have quite a lot of rolling shutter. F3 has about the same as FS100/AF101.

1d mark IV has it pretty well under control, I guess Vincent Laforet wasn't lying when he said it did have significantly less rolling shutter than the 5d.

Still, I kinda wonder why in such a controlled test, the rolling shutter artifacts were tested with completely different FOV's on the DSLRs than on the regular cams. The results are not completely comparable.

Only the last two were significantly tighter than the earlier cameras, but it's a temporal issue rather an image size artifact. The rolling shutter artifact shown on the disc is far larger compared to the other cameras than the slightly increased image size of the disc on the last two DSLSs. The disc is rotating in a flat plane more or less parallel to the sensor plane, so this image size difference wouldn't significantly affect how the spokes are picked up by the sensor.

Is there a numerical standard for rolling shutter?

You can measure the delay from the top of the picture to the bottom of the picture. For the 5D2, it is roughly 25ms. 24fps is 41.7ms per frame. At 24fps, that's 60% of the frame time. At 30p, it's 75% of the frame time. Having those numbers for each camera would let us compare them.

The way I tested the 5D2 is with a manual flash unit set to the fastest available speed. Film it. Find a frame that captured the flash from start to finish. Find a second set of frames where the flash started at the bottom of one frame and ended at the top of the following frame. Count the lines on each frame. From this information, one can calculate the rolling shutter latency as well as the actual shutter speed. Cool huh?

If I find a camera that has 10% less rolling shutter than the 5D2 (22ms), I don't really care. It's not that significant a difference. There will still be lots of jello. If it has half the rolling shutter (12ms), I'll be impressed. I will still want to control camera motion, but I'll know that the artifact will be significantly less noticeable. If the rolling shutter is 10% of that of the 5D2 (2ms), I'll be confident that it's virtually gone and I can shoot handheld on a boat in a storm.

I guess it can be debated over giving numbers or just showing a visual test. There'll be people asking what the numbers look like if you just give a number, while others just like numbers.

I agree that we want both the numbers and real-world pictures. Unfortunately, the shootout test fell into the trough in between - it was geeky without being numeric or real.

For real images, I would have opened up the ASC handbook and designed motor controlled pan and tilt tests that fall just within the ASC limits. I believe that for 24p, you want 7 seconds for an edge to edge pan. For 16x9, that is the same speed as roughly 4 seconds for a top to bottom tilt. Set up a nice scene that tracks people walking or a forklift elevating that has lines in the background, shoot it at 24p with a 180 degree shutter and you have a nice real-world test. Shoot it machine controlled as well as handheld and it would be pretty complete.

With the combination of those scenes and the measured numbers, we can get an idea about how much rolling shutter we can tolerate.

The nice thing about the numbers is that they tell you exactly how much lean or compression/expansion you'll get for a given scene. A lot of specs (MTF, THD, frequency response numbers for 20-20kHz) have numbers that don't nearly tell the whole story. With rolling shutter, the number is simple and definitive. It's a single dimension, point spec: the time delay from the top to the bottom of the image.

They did a great job with a very difficult subject. On episode 1 the Canon DSLR's did a pretty good job. Episode 2, they slipped a little, Episode 3, they fell a lot! It matches my results pretty closely in real life with a 7D (1D and 5D are similar), the resolution just isn't there for truly fine work. skins do tend to look a little pasty and 4,2,0 color is definitely limiting on CC in post.

I'm waiting for Canon to send us their newest fare in the next few months and hopefully get a sub $10K camera that will close the gap between - or better - the Alexa's and Reds. Things are changing fast and I feel good that the next wave will be sufficient for my long term needs. To be honest, there isn't a burning need to get much better than the 7D for computer viewing except moire and aliasing. Increasingly many find the standard of how something looks on a large computer screen over broadband since so much is viewed on that medium. How many really see the large screen as realistic goal for their work? Perhaps a sub $10K cameras in the next 24 months will surpass some of the much more expensive lines of today, hopefully with faster frame rate options, good controls, and external storage options if internal processors can't handle 4,2,2 or better!

Personally I want to thank the team which organized these tests. Regardless of any downsides, it was a tremendous effort to bring us some standarized data on all these models, and as such I applaud it. It does offer an interesting comparison, and I am glad they did it. And that they released it for free to the community.

Thanks for your efforts ladies and gentlemen!

^ DITTO!!!

Well said, Bart.

I can't believe how many are complaining about these expensive and FREE tests. My *only* issue is with the timing because I wish that they had waited a couple months for the S-Log update to be released. From my testing, recording S-Log to a PIX240 or Cinedeck Extreme makes a HUGE difference and it is worth every dollar. For under $20k, one can get a 13.5 stop, 10bit 422 camera recording to Pro Res and DNxHD.

I totally disagree with the thought that a DSLR is 'good enough' for the web and there is no need to use a better camera because you cannot see the difference over the internet. The higher quality the source image is, the higher quality the compressed image will be.