Is the sensor being fully utilized? at DVinfo.net

Go Back   DV Info Net > Canon EOS / MXF / AVCHD / HDV / DV Camera Systems > Canon EOS Crop Sensor for HD

Canon EOS Crop Sensor for HD
APS-C sensor cameras including the 80D, 70D, 7D Mk. II, 7D, EOS M and Rebel models for HD video recording.


Reply
 
Thread Tools Search this Thread
Old September 14th, 2009, 08:52 PM   #1
Major Player
 
Join Date: Sep 2007
Location: Tokyo
Posts: 230
Is the sensor being fully utilized?

Does anyone have any concrete information about how the camera is using an 18 megapixel sensor to record a 2 megapixel video stream? Does it record a full resolution (whole sensor) image which the processor resizes down during compression, or is only part of the sensor used for video recording?

We read that the camera has a bigger sensor than most video cameras, but I wonder how much of the sensor is actually being used for the video?
Brian Parker is offline   Reply With Quote
Old September 14th, 2009, 11:15 PM   #2
Inner Circle
 
Join Date: May 2006
Location: Camas, WA, USA
Posts: 5,513
On the 5D, the whole width is used. It's cropped top and bottom to make a 16:9 image. Horizontally, the image is filtered to balance aliasing and resolution. Vertically, the photosites are "binned" (added together), which causes aliasing.

Earlier reports claimed that one of every third line was used, but I recall an interview with a Canon engineer where he said that they were binned. That makes sense, as the noise will be lower with binning than with skipping two of three rows.

From what I've seen, the 7D almost certainly takes the same approach for 1080p, though it's not confirmed. I have no idea what it does to achieve 720p, though I assume that it still uses the whole sensor width.
__________________
Jon Fairhurst
Jon Fairhurst is offline   Reply With Quote
Old September 15th, 2009, 09:32 AM   #3
Barry Wan Kenobi
 
Join Date: Jul 2003
Location: North Carolina
Posts: 3,863
Quote:
Originally Posted by Jon Fairhurst View Post
Earlier reports claimed that one of every third line was used, but I recall an interview with a Canon engineer where he said that they were binned. That makes sense, as the noise will be lower with binning than with skipping two of three rows.
Can you please point to any authoritative information that discusses how these cameras work? I haven't found any official statements from anyone.

Binning makes no sense on a Bayer sensor, no sense at all. Now, if it was an RGB Stripe sensor, vertical binning would make perfect sense, but on Bayer it just plain wouldn't work.
Barry Green is offline   Reply With Quote
Old September 15th, 2009, 09:39 AM   #4
Major Player
 
Join Date: Mar 2006
Location: Chicago
Posts: 706
The AA filter strength is determined by the small pixels used in still photos. A stronger AA would help video, but hurt still.
So does red just throw out pixels? That would explain reds worse noise but less aliasing output. When the 5DII was released, Red said something about their superior ability to read/reset the chip. I have difficulty reconciling that comment to Canon reading every pixel every frame.
Also, both Red and the 7D overcrank at reduced resolution (double speed means half the pixels). Is this done by skipping lines?
Don Miller is offline   Reply With Quote
Old September 15th, 2009, 09:46 AM   #5
Barry Wan Kenobi
 
Join Date: Jul 2003
Location: North Carolina
Posts: 3,863
Quote:
Originally Posted by Brian Parker View Post
Does anyone have any concrete information about how the camera is using an 18 megapixel sensor to record a 2 megapixel video stream? Does it record a full resolution (whole sensor) image which the processor resizes down during compression, or is only part of the sensor used for video recording?
The speculation is that it skips pixels. The theory being that it can't read all 18 megapixels at 60 frames per second, that's an enormous task -- and it certainly isn't likely to read all 18 megapixels and perform demosaic'ing and scaling down to 2mpix (1080p mode) or 1mpix (720p mode) in realtime. Therefore the speculation is that it skips pixels horizontally and vertically to reduce the amount of chip that it has to read.

It's not "windowing" the sensor. It's not using the central 1920x1080 out of the chip or anything like that. It uses the full size of the chip, but perhaps not all the pixels on the chip.

Such an approach would lead to significant aliasing and moire issues, and indeed those have been observed in the video, so it seems like a likely explanation for how the chips are working. I just haven't found any official confirmation that that is indeed how they work. Alan Roberts' BBC assessment showed the effects, but he didn't speculate on whether it was row-skipping, he just concluded that it had a poor downscaling algorithm. You can see the results here: http://www.dvinfo.net/forum/canon-eo...n-roberts.html

Unquestionably the 5D delivers a gorgeous image, with some aliasing and moire. The 7D will probably perform comparably. The BBC chart looks scary, but unless you're shooting for the BBC you probably don't need to stress about it too much; instead spend your time looking at actual footage coming from the cameras and see if they meet your expectations and requirements.
Barry Green is offline   Reply With Quote
Old September 15th, 2009, 09:46 AM   #6
Major Player
 
Join Date: Mar 2006
Location: Chicago
Posts: 706
Quote:
Originally Posted by Barry Green View Post
Can you please point to any authoritative information that discusses how these cameras work? I haven't found any official statements from anyone.

Binning makes no sense on a Bayer sensor, no sense at all. Now, if it was an RGB Stripe sensor, vertical binning would make perfect sense, but on Bayer it just plain wouldn't work.
It wouldn't be a straight bayer algorithm. But certainly you can derive a pixel value based on many surrounding RGB values.

I recall discussions about binning, but no direct quote. Binning certainly could make"jello" worse.

I don't know how canon video works.
Don Miller is offline   Reply With Quote
Old September 15th, 2009, 09:52 AM   #7
Major Player
 
Join Date: Mar 2006
Location: Chicago
Posts: 706
Quote:
Originally Posted by Barry Green View Post
............

It's not "windowing" the sensor. It's not using the central 1920x1080 out of the chip or anything like that. It uses the full size of the chip, but perhaps not all the pixels on the chip.
Which Red does, yes? Crops for a given resolution?

It is possible to bin pixels on chip, as Foveon does.
Don Miller is offline   Reply With Quote
Old September 15th, 2009, 09:52 AM   #8
Barry Wan Kenobi
 
Join Date: Jul 2003
Location: North Carolina
Posts: 3,863
Quote:
Originally Posted by Don Miller View Post
The AA filter strength is determined by the small pixels used in still photos. A stronger AA would help video, but hurt still.
Maybe, maybe not. If it was using the entire sensor, yes. But if it's actually skipping pixels, then a stronger AA filter might just result in lower overall resolution. An AA filter is basically a "blur" filter that masks out the highest frequencies and tiniest details, but if the video frame is made using pixel-skipping, then it's dependent on those same tiniest details being reassembled into a smaller-sized full frame, so a stronger AA filter might just result in ruining the overall sharpness of the video...

Quote:
So does red just throw out pixels?
Well, no, not in the same terms that Canon is suspected of doing; Red reads the entire sensor for its full-frame 4K imagery. Their sensor is purpose-designed for video performance, and is designed to be read at full resolution at up to 30fps.

Quote:
Also, both Red and the 7D overcrank at reduced resolution (double speed means half the pixels). Is this done by skipping lines?
When going into faster frame rates, Red requires smaller frame sizes, because they window into the sensor. You can get faster frame rates in 3K mode, and even faster frame rates in 2K mode, because the 3K and 2K modes don't read the entire chip. However, they don't use the reported pixel-skipping technique either; they just read a smaller "window" off of the chip. The field of view shrinks in 3K mode, and shrinks further in 2K mode, as it's reading a smaller window out of the chip.

Windowing lets them retain the Bayer pattern of the chip and deliver proper color accuracy, etc.

Binning, would be a perhaps ideal way to accomplish lower resolution and higher frame rates, but it can't be done on a single Bayer sensor. Again, these chips are all monochrome; there aren't color sensors. Now, if you had three chips, you could bin them and get faster frame rates, less aliasing, more sensitivity, and less noise... but with a single chip you can't bin red and blue and green all together into one pixel, because the chips don't record color information, they only record brightness information.
Barry Green is offline   Reply With Quote
Old September 15th, 2009, 09:54 AM   #9
Barry Wan Kenobi
 
Join Date: Jul 2003
Location: North Carolina
Posts: 3,863
Quote:
Originally Posted by Don Miller View Post
It wouldn't be a straight bayer algorithm. But certainly you can derive a pixel value based on many surrounding RGB values.
Only if you've read all those values. Binning doesn't read them individually, it throws the green in with the red and the blue into a big soup, losing the color value for each pixel. Hence why binning would make no sense on a Bayer sensor.

The speculation about Canon using pixel-skipping, and specifically skipping to every 3rd pixel, is that doing so retains the R/G/B/G nature of the Bayer pattern.
Barry Green is offline   Reply With Quote
Old September 15th, 2009, 09:55 AM   #10
Barry Wan Kenobi
 
Join Date: Jul 2003
Location: North Carolina
Posts: 3,863
Quote:
Originally Posted by Don Miller View Post
Which Red does, yes? Crops for a given resolution?
Yep.

Quote:
It is possible to bin pixels on chip, as Foveon does.
Yes, but Foveon is, in essence, a three-chip system. It's just three chips stacked in one place, instead of split by a prism. Foveon is not a Bayer pattern. The restriction on binning is specifically because of the Bayer pattern.

RGB Stripe would also be inappropriate for binning horizontally, but could be done vertically.
Barry Green is offline   Reply With Quote
Old September 15th, 2009, 11:49 AM   #11
Major Player
 
Join Date: Feb 2008
Location: Voorheesville, NY
Posts: 433
This has been posted here before:

Canon : Canon Technology | The Minds Behind the Magic

and it is long out of date, since it talks only about the 1G consumer camcorder, the HV-10. But if you go a few pages into the interview, it gives an example of how an EOS DSLR captures 8 fps @ 8 Mpixels. When you combine that information with the rest of skimpy info that Canon has online regarding it's CMOS sensors, you can get an idea as to the viability of the two "theories" espoused in this thread, as to how the 5d2 and the 7d turn large numbers of Mpixels into 1920x1080 (every third row used vs. binning, etc.).

Canon is not too likely to give out its trade secrets, so this is about all we "consumers" are going to find out, at least for now. What other people have tried is searching for the public information regarding the patents of the various Canon R&D employees, such as:

IMAGE PICKUP APPARATUS AND METHOD FOR CONTROLLING THE SAME - Patent

So far, though, we're still stuck with educated guesses as to how the two Canon HDSLRs actually.
Jay Bloomfield is offline   Reply With Quote
Old September 15th, 2009, 12:00 PM   #12
Inner Circle
 
Join Date: May 2006
Location: Camas, WA, USA
Posts: 5,513
If I can find the source that claimed binning, I'll post it. I saw it about two months ago, and was somewhat surprised.

One clue that leads me to believe that we are seeing "creative binning" is the strong color errors on black and white zone plates. If we were to take every third line and filter it properly, color errors would be minimal. Show every third line, and we would see monochrome aliasing. Instead we see strange color artifacts. Maybe this is due to poor filtering in the horizontal direction, but I can't help but think that there is something more going on.

Straight binning would mean adding values together. It's possible that only common color photosites are added, and that they are appropriately scaled - in the case that you add two sites together, divide the result by two with resistors on the sensor itself. Certainly, skipping two of three rows would be faster and simpler.

Anyway, back to the original poster's question, It's pretty clear that (at least, nearly) the entire width of the sensor is being read, and the vertical image is cropped to provide a 16x9 image. The exact reading structure is not known for sure.
__________________
Jon Fairhurst
Jon Fairhurst is offline   Reply With Quote
Old September 19th, 2009, 12:48 PM   #13
Major Player
 
Join Date: Jan 2007
Location: Portland, OR
Posts: 949
Quote:
Originally Posted by Jon Fairhurst View Post
Earlier reports claimed that one of every third line was used, but I recall an interview with a Canon engineer where he said that they were binned. That makes sense, as the noise will be lower with binning than with skipping two of three rows.
But the noise isn't lower. In my (unscientific) analysis, the noise looks about the same as taking a JPEG and downsampling it with a nearest neighbor algorithm. It is much noisier than taking a JPEG and downsampling with sinc or any good algorithm.

It's worse in photon shot noise dominated image tones (midtones at low ISO and highlights at high ISO) as well as read noise dominated image toines (midtones at high ISO and extreme shadows at low ISO). If it was binning, the photon-shot-noise-dominated tones would have the same noise and the read noise would be lower than just resampling.

Binning a Bayer CFA sensor would likely result in certain artifacts, but not nearly as bad as what the Canon video has. There are some known techniques for reducing the artifacts in real time, Phase One is probably using them in the binning mode they added through a firmware update a few months ago.

Canon's sRAW2 is more in line what I would like to see with Canon video: the noise and aliasing are worse than a downsampled still, but not nearly as bad as the video currently is. But sRAW only needs to do 4 FPS, not 30.
Daniel Browning is offline   Reply With Quote
Old September 20th, 2009, 12:45 PM   #14
Inner Circle
 
Join Date: May 2006
Location: Camas, WA, USA
Posts: 5,513
Here's a credible source: Tim Smith, Product Manager of Digital Imaging at Canon.

Digital Cinema Society - New Streaming
(Go partway down the page. There are two QT files of Tim Smith. It's from July 23rd.)

In part 1, about half way through, Tim Smith strongly recommends UDMA cards. He also confirms that they don't pixel skip. They bin them in groups of six to pull down to 1080. He also says Keeping the camera level helps.

He talks about the need to re-clock the chip to get to 24p, and says its unlikely, but not impossible that there could be a firmware update. Then again, he would have bet money that they wouldn't release a manual control firmware update.

Tramm got Canon's attention.

Canon is thinking about cine-style lenses. Some of the development group silos are falling down.

Part 2 shows a Ghost Whisperer webisode, shot back in the auto-only era. He talks about his experience on that shoot. He mentions the 50L, 85L and 16-35L.

Tim recommends leaving HTP on. (I don't, unless the dynamic range of the scene is extreme.)

He's seen the 5D with a PL adapter. He hasn't yet seen a film transfer. When projected large, he expects that you will see lens flaws with cheaper glass.

The best news is that Tim Smith "gets" the needs of videographers. It's not like he's a photo guy who is just now being exposed to video and film.

Anyway, the videos are worth watching for anybody interested in the development of HDSLRs.
__________________
Jon Fairhurst
Jon Fairhurst is offline   Reply With Quote
Old September 20th, 2009, 01:32 PM   #15
Inner Circle
 
Join Date: Nov 2005
Location: Elk Grove CA
Posts: 6,838
Jon:

Very interesting video... First acknowledgment by Canon that I have seen that Tramm's work has caught their attention.

I am not sure how technically competent he is, as he seems to believe file size can be increased by a hack. My understanding is that file size (at least in HiDef) is limited by the 4 gig maximum size. I think the reference to the guy in the guy is Seattle not having hacked that is kind o f interesting, as they seem to be mixing your location, somebody in Europe and Tramm as an all in one deal.... maybe he is referring to linked files when size limit is over reached.

He is pretty clear that 24p won't come out of chip without clock change, but then again, something that technical isn't his forte. Most interesting is the acknowledgment that Canon never saw this camera gettting anywhere near film making...
__________________
Chris J. Barcellos
Chris Barcellos is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Canon EOS / MXF / AVCHD / HDV / DV Camera Systems > Canon EOS Crop Sensor for HD

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 08:26 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network