Magic Lantern unlocks Canon 5D to shoot 14 bit raw via liveview

[David Heath]

Originally Posted by Ted Ramasola

Its not an April fools ;). I'm careful in stating "raw in liveview" and not "raw video" as its still in the early stages.
However, the developers say it passed the biggest hurdle. They currently can do 10fps of compressed raw. They say "raw video" meaning at at least 24fps is potentially possible.

My understanding is that "RAW" is normally used to mean the digitised data straight from the sensor, so in the case of the 5DMk3 I'd expect 5616 x 3744 (the sensor resolution) words of 14 bits? (More likely to be 5616x3159 - the 16:9 crop)

But references to "in liveview mode" and 720p resolution seem to indicate that's not what is on offer here?

"Liveview" is normally obtained by ignoring or grouping a percentage of the total photosites - typically as many as 75%. ("Pixel skipping", "line skipping" or "pixel binning" are various methods used.) It's the only way the sensor can be read at rates fast enough for video.

I suspect what's being done is obtaining the liveview stream pre the processing which gives normal video, obtaining the 14 bit samples, but not from the whole number of photosites of the chip. This may give some benefits in that there will be more controllability in post - but resolution, aliasing etc will be the same as in normal mode. It will give benefits over the normal recorded signal - in terms of post manipulation - though probably less than many may be hoping for. The question is whether the improvement is worth the far greater data rate that needs recording....?

The real issue is that more photosites is not necessarily a good thing, certainly for video. It's far better to have the optimum number and make use of them all, than too many and have to ignore a percentage of them. If the 5D has 20MP and uses 25% of them in video mode, it can never be as good as using a 5MP chip and fully using all of them.

Which is why chips and cameras are designed to have optimal performance for stills OR video. It's unrealistic to expect a camera primarily designed to take high quality still images to have comparable performance to one in the same price class designed specifically for video.

[Alister Chapman]

For the raw hack you can choose either a line/pixel skipped output that uses the full sensor but pixel skips to achieve different output sizes or a crop mode which does not pixel skip but only uses a much smaller part of the sensor.

[David Heath]

Originally Posted by Alister Chapman

For the raw hack you can choose either a line/pixel skipped output that uses the full sensor but pixel skips to achieve different output sizes or a crop mode which does not pixel skip but only uses a much smaller part of the sensor.

......Which makes perfect engineering sense!

The problem comes that if pixel skipping, the sense of the Bayer pattern is lost, and if the gaps (unread photosites) in the pattern are large, then expect high aliasing/moire levels, or significant softening if this is controlled via a low pass filter. This seems to be held up by a quick look at the resolution charts posted.

Do you have figures for how big the area is in crop mode, Alister? It's reasonable to expect a readout of about 5MP (a quarter of the total), which I'd reckon to be about 2808x1580, and here could be a true Bayer pattern. That should deBayer down well to full 1080 resolution, with little aliasing etc - but in terms of lens angle/crop/dof will mean it's only operating as a 4/3 (approx.) size sensor camera. Which rather defeats the point of having a ff 35mm camera in the first place....? :-)

I don't deny this is a clever technical achievement, but don't anybody get too carried away. It may have benefits over the normal video modes in some respects (at the cost of far larger file sizes, and much faster - hence expensive - memory) but don't expect it will ever mean you'll be getting F5 class results at bargain prices.

If you want optimum video quality, buy a video camera, period. And marvel that it's possible to get decent video from a stills camera at all........

[Noa Put]

I read somewhere that currently it's only possible to record for about 40 seconds in raw before the camera stops, is that right and if so, has it to do with a fat32 file size limit that makes the camera stop when it reaches a certain filesize?

It all looks beautiful but I also have seen some footage with errors in it like vertical lines and other artifacts but I guess that over time this should be eliminated as well. Currently it doesn't seem like anything usable in a paid production.

[Alister Chapman]

You can choose various H and V sizes, they are all multiples of 4, so all bayer compatible. Some of the frame sizes currently enabled are:

Full Frame or near full frame with line/pixel skipping
3592 ×1320 - Reported as very glitchy and dropping frames, not surprising as it is over 150MB/s
1920 x 1280 (3:2) Approx 70 MB/s
1920 x 1080 (16:9) Approx 60 MB/s
1280 x 720 (16:9) Approx 30 MB/s
1280 x 1280 (1:1)


1:1 crop modes -

3592 x 1320 Very glitchy as above.
2880 x 1320 Approx 110 MB/s
2560 x 1080 Approx 90 MB/s
1920 x 1080

The raw format is 14bit linear uncompressed RGGB raw and you then use a command line utility called raw2dng to convert the raw data to Cinema DNG.

X1000 CF cards are said to be able to hit about 140MB/s, so being realistic for video that means a useable speed of half that so I would suggest the real speed limit is 75MB/s, so you might be able to do 1920 x 1280 or 1920 x 1080 reliably but your really going to want 2880 x 1320 to get full resolution and that needs around 100MB/s so it's seriously pushing the technology even if it was actually designed to do it.

But the pixel/line skipping will kill any real advantages full frame, you will still suffer all the 5D artefacts that have always been there.

Crop mode should eliminate the artefacts, but the BM pocket camera or BMCC make more sense if you want a smaller sensor. A 64GB card will give 10 mins, 128GB card will give about 20 mins of record time, but with FAT you'll only be able to shoot for 40 seconds unless they can figure out a way to do split files. At $700 for 128GB or $450 for 32GB per card that's more expensive per minute than AXS, SSD or Red media.

[Ted Ramasola]

Here is Macgregor's video he shot with raw. I helped him out getting some of the files needed to do the module install and file extract.

He currently has a slow card so he only shot with 1920x720.

We've seen lotsa trees and rocks, this is docu style showing a craftsman doing his trade.

Timple - 5Dmk3 RAW video test - on Vimeo

[Peer Landa]

Originally Posted by Alister Chapman

At $700 for 128GB or $450 for 32GB per card that's more expensive per minute than AXS, SSD or Red media.

Actually, a 128GB RED-mag cost $1,250.-

-- peer

[David Heath]

Originally Posted by Alister Chapman

X1000 CF cards are said to be able to hit about 140MB/s, so being realistic for video that means a useable speed of half that so I would suggest the real speed limit is 75MB/s, so you might be able to do 1920 x 1280 or 1920 x 1080 reliably but your really going to want 2880 x 1320 to get full resolution and that needs around 100MB/s so it's seriously pushing the technology even if it was actually designed to do it.

Thanks for the information. As far as speed goes, then it's likely there will be an element of in camera buffering, so even a slow card would be able to record a high bitrate stream - just not for very long!!! :-) Faster the card, the longer it will take for the camera buffer to fill, hence why a faster card will record for longer before cutting out. (And why a card will manage to record a bitrate much faster than it's rated for - if not for very long.)

But the pixel/line skipping will kill any real advantages full frame, you will still suffer all the 5D artefacts that have always been there.

Fundamentally agreed. This "liveview RAW" mode may give more post manipulation possibilities, and avoid the basic compression - but since the drawbacks will be still be more significant, is it worth it?

Crop mode should eliminate the artefacts, but the BM pocket camera or BMCC make more sense if you want a smaller sensor.

Again, agreed. It's clever, and may have niche applications, but fundamentally if you want a camera to shoot general video, something like a BMCC makes far more sense, far better value for money.

[Cliff Totten]

From the few videos I have seen, the ML software is doing some truly amazing things. The RAW capture is impressive but the video sharpness they are getting from a sensor like that just knocks me out. Aside from bypassing the internal codec, how are they doing it? Have they changed the sensor scanning read out order? It doesn’t appear to have “typical” line skipping artifacts that we always see. Are they pixel binning? (is that even possible?)

Bigger industry questions:

How will this affect Canon’s upper model sales? (C 100/300)

How will this affect BlackMagic Design? (BMD is no longer the only player in the sub $4000 RAW recording anymore. (I’m sure there are many, MANY Canon folks rethinking their BMC pre-orders)

How will this affect Sony, Canon and Panny in the long term? RAW recording in $1,000 to $4,000 cameras is going to be extremely popular moving forward. (once you go RAW, you never go back)

How will this affect future designs and will it pressure the “big three” to build RAW into new lower tier models? (Until recently, RAW has been reserved for top end customers only)

It seems to me that BlackMagic and Magic Lantern have created a watershed moment in the industry. We might look back on all this years from now and say they were the “rule-breakers” that forced a big industry marketing model change. (Compression-free, RAW recording for the “little guys”)

[Lawrence Bansbach]

Originally Posted by Cliff Totten

. . . once you go RAW, you never go back . . .

I can see not wanting to have to deal with the compromises of compressed codecs, but most pros will use the format best suited to the situation, and that may not always be raw.

[Tim Polster]

It has to come sometime. When you shoot a RAW still with the 5D you can see the detail the sensor can make. Greater than most will ever need at this point in time for video. Since HD video is mostly progressive in this space it is logical that it is just down to transferring the bits quick enough. And having the marketing will to break new ground.

Not to say Canon is evil here because they have to represent a professional working product, not a few seconds here and there with a special uber CF card. Make me think of the famous Steve Jobs quote about needing to innovate or your competitors will do it for you.

It will happen, but is it time now? We will see over the next few weeks.

About RAW. Image quality or more precisely dynamic range is a matter of the RAW processing software as much as the captured data. This update is in the early stages but to make it work one needs a way to process the video like Adobe Lightroom or Red's free program. The image quality improvements outside of detail will come from the ability to process the footage.

When I shoot landscape photos I have learned over the years to completely expose for the highlights. This results in a very dark terrain at times. But from experience I know I can lift the mids & shadows to equal the highlight exposure without much penalty because the image started in RAW format. Only after this lift does the image come to life and appear to have great dynamic range. The sensor captured the information, it just needs to accessed. This two stage process is not the usual video processing approach but will be adapted with the right software. I can see Adobe adding a module to Lightroom for this very purpose.

This is clearly early days but I think we only go up from here.

[David Heath]

Originally Posted by Tim Polster

It has to come sometime. When you shoot a RAW still with the 5D you can see the detail the sensor can make. Greater than most will ever need at this point in time for video.

And this is where the confusion is. What this hack gives you is *NOT* the same as RAW stills at video rate. With a RAW still you have about 20 million 14 bit words (one for each photosite). It's impossible to read that many 30 times every second, let alone 50 or 60. Read Alisters description earlier, and you see that what we're talking about is omitting most of the 20 million.

Since HD video is mostly progressive in this space it is logical that it is just down to transferring the bits quick enough. And having the marketing will to break new ground.

Don't underestimate the task of "just" transferring the bits fast enough. If you really were to be reading the full sensor at 60fps, at 14 bit depth, we're talking for 16:9 of 5616x3160x60x14 bits per second. That's about 15BILLIONbits every second. Hence the need to only read a fraction of the total each frame. It's vital to draw the difference between "true" RAW and "liveview RAW", and that's not coming through clearly.

Not to say Canon is evil here because they have to represent a professional working product, not a few seconds here and there with a special uber CF card. Make me think of the famous Steve Jobs quote about needing to innovate or your competitors will do it for you.

Well, it would have to be a very special "uber CF card" to record the datarate above, if your assumption about the entire sensor being read every frame was correct! It's nothing to do with "Canon being evil", full sensor read out at video rate would likely melt the chip very quickly, let alone the bit rate being unrealistic.

Although Canon do provide the facility to read out continuous frames at full resolution - but just at 3.9fps! And expect that to be a technology limit. Either read full sensor resolution at up to 3.9fps - or a pixel skipped or cropped mode at video rate. Either about 20 MP at 3.9fps - or 2MP at 30fps. Same data in each case, yes?

It will happen, but is it time now? We will see over the next few weeks.

If you're referring to recording true full resolution RAW images from a DSLR, at normal video frame rates, then don't hold your breath. That's misunderstanding what this is about. "RAW liveview" is a different matter. It may well be an improvement over basic DSLR video, it won't rival a true digital cinema camera, even something at the lower end like a C100

[Tim Polster]

Thanks for the numbers David. I did not know the facts behind the data. This is important info to get out there as the term RAW is such a buzzword right now. Looks like "RAW" will turn into a term just like "HD" which changes based upon how the person or entity wants it to be.

It sounds like the 5D will be using way less information than true RAW output, but if it improves the images compared to the H.264 images then it is a step forward. If anything, the detail improvement is most welcomed as it is such a shame to take sharp stills but have soft video from the same camera.

I am sure this will be a hot topic for quite some time into the future.

[David Heath]

Originally Posted by Tim Polster

This is important info to get out there as the term RAW is such a buzzword right now.

Very true! :-)

Looks like "RAW" will turn into a term just like "HD" which changes based upon how the person or entity wants it to be.

And again, very true. There's a certain inevitability there, as a highend, low noise camera will benefit far more from a greater bitdepth than a noisier one. But I'm sure we'll see the marketing people wanting "16 bit RAW!!!" as a sales point, even if it makes no advantage in the case of a cheaper camera. Or is that too cynical?

To be fair, the title here is "shoot 14 bit raw via liveview". Unfortunately, people are seeing "RAW" and not realising the significance of the liveview part.

It sounds like the 5D will be using way less information than true RAW output, but if it improves the images compared to the H.264 images then it is a step forward. If anything, the detail improvement is most welcomed as it is such a shame to take sharp stills but have soft video from the same camera.

Yeeess.... but the question starts to become "is it worth it.....?" You can end up spending so much on a DSLR that maybe the same money would just be better spent on something like a C100. And as I understand it, the impetus behind using DSLRs was driven by dof and a large sensor. The best results here seem to be using the technique in crop mode (not pixel skipping the whole sensor area) - so you're effectively losing the whole point.

To put some numbers behind that, then in ff mode, it's 36mmx24mm and 5616 x 3744 photosites. Going into 1920x1080 crop mode, that means the used area becomes about 12.3x6.9mm - so smaller than super16 film, let alone the Black Magic and 4/3" cameras. Heck, it's only about a quarter the area of a s35 sensor!

And even in 1920x1080 crop mode, that won't be true 1080 resolution. The deBayering will knock it down to something like 80%, about 850 lines for luminance. Better than a native DSLR - but dof etc no longer anything like a ff chip!

Go to ff mode and the inevitable pixel skipping will mean it will fundamentally be unlikely to be any sharper than native DSLR. (Though the RAW nature will give far greater control over things like detail enhancement. There will be an improvement - but it won't be anything like as much as a lot of people are hoping.)

[Jon Fairhurst]

Shooting 2880 x 1320 in crop mode looks like it would have few downsides. Yes, it's not full frame, but it will lack aliasing and be well oversampled and sharp. The Tokina 11-16 gives you a 22-36 mm equivalent view, using the strong part of the glass.

And for full frame shooting, there is the Mosaic Engineering anti-aliasing filter.

It wouldn't make sense for a lot of projects, but when shooting green screen effects work (which doesn't only need clean keys but also requires subtle grading) and for anything with smooth gradients and the need for strong grading, this gives the DSLR shooter new powers.