View Full Version : Magi Lantern and RAW workflow doubts


Rafael Lopes
May 16th, 2013, 02:15 AM
Haya fellas,

So, with all this talk about how Magic Lantern is enabling RAW recording on the MarkIII (and possibly other Canon cameras) I have some questions regarding the entire workflow, but specially post:

1 - Iīve been shooting a web series and each episode (around 5 min each) ends up with around 100 minutes of footage. How much disk space are we talking about here?

2 - What is the more standard editing workflow for raw? Do people use an intermediate codec like cineform or just go with an ultra powerful computer?

2.1 - Is most people go with an ultra powerful computer, what is the system requirement smoothly edit RAW? Iīm talking CPU, Cores, RAM, Disk space and speed, GPU, etc. I have limited space at my place and I was wondering if it is remotely possible to edit RAW on a powerful laptop or if itīs just crazy.

3 - This is more of a development question and being so soon in the development process it might not have an answer now. Is it possible to record Full HD 24p RAW into fast SD cards? I keep hearing everybody talking about CF x1000...which basically would mean that you have to go with either the MarkII or MarkIII (I think these are the only Canon video DSLRs that have CF ports).

Fergus Kennedy
May 29th, 2013, 09:21 AM
I've been trying out ML for the Mark III over the last week or so.

1.Depends on what res and frame rate you shoot at but at 1080p 24fps, you would be looking in the region of 500GB of storage or even more. A 64gb card gives you about 10 mins I think.

2. Workflow is a bit laborious- after you've got the firmware working on your camera (often takes a while to get right). You shoot using appropriate settings in ML.
Download the .RAW files to your computer. Use Raw2dng to convert them to a string of .dng still images. Import dngs into lightroom or similar- process them, export to jpegs or tiffs, use another program such as quicktime to convert them to a video file which you can import into your editing software. Takes a while for each clip. Although you can record audio in ML, I think it would probably be easier at the moment to record with an external recorder. Either way you have to sync audio in post.

3. Not possible at the moment to record full HD 24p to an SD card. None of the SDs are fast enough. You can record lower resolutions I believe.

Chris Barcellos
May 31st, 2013, 11:12 PM
I have been shooting with my 5D Mark II. I use a Komputerbay CF 1000x 64 gb, which others said was working well. It was the least expensive card I could find. Latest version also has continuous shooting past the 4 gig limitation.

Largest frame I am able to record is 1880 by 840. Somewhere around 2.35:1. Latest version also has wave sound recording capability that decreases band with for video.

For me, post is fairly easy in my PC processing. You take the resulting files, drop them on a file called rawtodng.exe. It takes the single file, and turns it in to individual dngs. It is fairly quick.

From there, I open GoPro's Cineform Studio premium, and process the files into an avi for easy editing. Cineform claims there converted files from DNGs preserve the original dynamic range and color correction capabilities, and I have no reason to doubt that at this point.

Fergus Kennedy
June 1st, 2013, 07:01 AM
Just to update on workflow- there's also now 'rawanizer' a free download which will batch process the .raw files to DNGs and even convert them to prores or cineform video files all in one. I couldn't get the last step to work on my PC, and in any case I prefer using lightroom for grading as that's what I'm used to from my stills photography.

I also found I could cut out the quicktime step I mentioned above and just import the jpegs (from lightroom) as an image sequence into the NLE.

I guess the preferred workflow will vary from person to person.

Shawn Clary
June 4th, 2013, 01:38 PM
I shot a full 64 GB test movie; it was over 12 minutes long. 5d3, 1000x Komputerbay card (amazon, $114). 1920x1080 23.976 FPS with a separate audio file.

Personally I wouldn't waste the time for a web series and raw. 100 minutes each episode would be a nightmare. Shoot in the regular H264 mode and grade it well in post (apply some nice sharpening) and you'll be fine.

Robert Benda
June 4th, 2013, 02:01 PM
May I ask, apart from how absolutely cool this is...

What would you use the RAW shooting for?

John Carroll
June 5th, 2013, 07:35 AM
Perhaps if you are planning to do a lot of special effects work/color grading. Or if you are shooting crazy dynamic range shots like sunsets, silouttes or bright windows in the background where you need to be able to pull detail out of areas that will normally be blown out or black... Otherwise this is a lot of effort for normal everyday type shooting.

Great to have the capability, but if you have a shot that tricky/important, I would go with Alexa or something along those lines. It's only money!

Chris Barcellos
June 5th, 2013, 11:13 AM
The whole idea of the DSLR revolution is to beat the system. That was what was behind Magic Lantern when Trammel Hudson first started breaking the code three years ago. The DSLR and Magic Lantern is made for those who want to push the boundries of their equipment, and not fall into the trap of paying manufacturers for to climb the technological mountain. If you aren't into that process, and you have the money (meaning a real budget), you need to buy Alexa or Red, Sony or Canon, etc, etc. high priced rigs.

Because my idea of film making is doing my own thing, and not being told what to shoot,l I personally don't have that kind of cash but still want to participate in film making to the best level I can. I am not independently wealthy, nor do I expect that I will make a living from my film making activity. I suspect that about 98% of avid shooters out there are in the same boat. That's why I worked with Magic Lantern from the beginning, and that is why I have also participated in Black Magic's grand experiment.

Black Magic Raw is going to be tamed and will become a very useful tool to those of us who just want to make films, and I salute the developers as they continue. Somedays I have questioned the need for some of the add ons made to Magic Lantern, but this Magic Lantern raw is a great update and once ironed out, will be great.

Chris Barcellos
June 5th, 2013, 11:29 AM
May I ask, apart from how absolutely cool this is...

What would you use the RAW shooting for?

If you have never worked with raw, you will understand once you do. One way to discover the difference is to shoot raw photos on you camera, and then shoot a .jpeg level of the same scene. Then take both files into Photoshop, and try to adjust various levels, saturation and sharpening, Try to bring up detail in highlights or shadows by making these adjustments. You should find that the raw photo has much more detail hidden in the shadows and highlights than the .jpeg.

Raw film shooting is exactly the same. It provides detail over a wider range of highlights and detail and easier color correction capabilities. It also provides much better resolution and apparent sharpness without need of adding sharpening. In short, everything about a raw image is just better because you have more digital information to manipulate..

John Carroll
June 5th, 2013, 11:49 AM
It's cool to do your own thing and test the boundaries of your equipment and all that, but if you make your living with your camera (and editing), time is a pretty important factor. I don't know many people who get to just do what they want to do... if you don't have to make a boss/client happy with your work, consider yourself extremely lucky! Most of us don't have that luxury... probably about 98% of us. :)

RAW is just another tool, that is great if you need it, but could quickly kill a budget for shoots that don't require the additional latitude, especially for one-man-bands like me. In a controlled environment, you should be able to get outstanding images without having to use RAW at all.

I'm not "dissing" RAW at all... I'm just saying there's a time and place for it to be truly useful. I can think of plenty of situations where RAW would be very beneficial, but I can also think of plenty of situations where it will add time/complexity to a shoot, and the client would never be able to tell the difference in the finished image... unless you show them a side by side comparison, maybe not even then.

The folks at Magic Lantern do outstanding work! Not dissing them either!

Jeroen Wolf
June 7th, 2013, 05:20 AM
It's cool to do your own thing and test the boundaries of your equipment and all that, but if you make your living with your camera (and editing), time is a pretty important factor. I don't know many people who get to just do what they want to do... if you don't have to make a boss/client happy with your work, consider yourself extremely lucky! Most of us don't have that luxury... probably about 98% of us. :)

RAW is just another tool, that is great if you need it, but could quickly kill a budget for shoots that don't require the additional latitude, especially for one-man-bands like me. In a controlled environment, you should be able to get outstanding images without having to use RAW at all.

I'm not "dissing" RAW at all... I'm just saying there's a time and place for it to be truly useful. I can think of plenty of situations where RAW would be very beneficial, but I can also think of plenty of situations where it will add time/complexity to a shoot, and the client would never be able to tell the difference in the finished image... unless you show them a side by side comparison, maybe not even then.

The folks at Magic Lantern do outstanding work! Not dissing them either!

To label RAW as 'just another tool' is unappreciative of its production value: the h264 files coming natively out of the Mark 3 are mushy, soft and can't deal with high contrast. RAW turns this around completely. It's not just about coloring but also about resolution and that's no 'luxury'.
Additionally, especially in uncontrolled environments, the ability to color correct RAW images can save your b*tt...
As far as the time issue is concerned: unless you are floating in work, the extra time it takes to work with RAW is manageable. I guess there are people who have to turn in their finished films the same or next day but I always plan plenty of editing time for my corporate and other commercial work.
I am a one man band as well and just because of that, you are more likely to make errors that you want to be able to correct. RAW gives you that option. Now THAT makes my clients happy...

Before long we'll get some affordable 1TB CF cards and a stable postproduction workflow and we'll all be shooting RAW...

Rafael Lopes
June 7th, 2013, 06:12 AM
After a lot of reading I really donīt think I will benefit at the moment by shooting raw. On the contraty I would have to delay production and invest in a bunch o new equipment to deal with RAW files.

Besides I was looking in the Magic Lantern FAQ and I could not find a stable build for the Canon 6D.

Michael Kraus
June 7th, 2013, 08:50 AM
Besides I was looking in the Magic Lantern FAQ and I could not find a stable build for the Canon 6D.

Yeah it might be a while before Magic Lantern is officially out for the 6D...perhaps especially now that everyone is focusing on the RAW recording functionality. In the meantime there are nightly builds for it that work just fine. They just haven't been tested to MLs standards just yet. I couldn't resist so I went ahead and gave it a try....works great. I'll definitely not be taking it with me on a paid shoot though.

John Carroll
June 7th, 2013, 01:19 PM
Yeah, I agree... pretty cool stuff for sure. I can't help feel like I am stepping on some folk's feelings here by saying this is "not quite ready for prime time", but that's not my intention... just my opinion. No denying the power of the RAW workflow. If you really need that though, makes sense to go with a camera that was built for it. As you all know, it's not just a matter of installing ML on your card, there are plenty of ripple effects into time/workflow/equipment that really should be considered. Does your client ask you to use this workflow? Do they care? If you are giving away that added capability to your client now, you'll never be able to charge for it later... but that is really a different conversation.

If you just like it and want to work that way, cool. No need to defend yourself. If it helps you out as a safety net in case you make a mistake during the shoot, that's great too! Although, I doubt this makes your client any happier. The ability of you to bail yourself out of your own mistake really only keeps your client from finding out you messed up, right?

RAW is great and we'll all probably shooting it eventually anyway... someday. I'm just saying that for me, right now, for most of the shoots I produce, I don't need it. Actually, I can't even remember wishing I had it. I color grade every project that goes through my pipeline. But I've never pushed an image so far through the ringer that the image begins to break apart. I transcode to ProRes so maybe that helps... perhaps if I was a little crazier with color grading it might be a different story... or if I had to fix an image that I completely f'd up in camera.

Plenty of folks have been working for years without RAW and producing incredible artistic imagery. The ability to shoot with less discipline or knowledge with the intent of "fixing in post" seems like a slippery slope to me.

Chris Barcellos
June 7th, 2013, 05:02 PM
Actually, raw is a protection for you clients. Clients sometimes ask for one thing, and actually need something else. You can light and shoot as client wants and you can shoot ETTR and deliver what client ask for in dailies by correcting raw footage, and then rendering. Ultimately you still have the raw if client has a change in idea, and the raw can most likely be corrected to the new vision. This is a tool that expands your capabilities, if you are willing to learn it and make the effort.

John Carroll
June 7th, 2013, 08:06 PM
Could happen I suppose, even if it doesn't seem very likely. Again... it's just never happened to me before. That's not to say it has never happened to anyone else before.

Jon Fairhurst
June 10th, 2013, 11:36 AM
RAW is an amazing tool, but definitely has its limitations. We considered it for a greenscreen project, but we want to capture improvisation by leaving the camera rolling. RAW is too data hungry for that.

One place to consider it is for establishing shots. Say your doing a romantic piece. Shoot the sunrise over the garden in RAW, if not RAW timelapse. Make this introduction perfectly beautiful to draw the audience in. Maybe stick with RAW as the character walks down the street and into the house. Now go to h.264 for the interior dialog shots where you need multiple takes to capture the nuance of the performance. At this point, you need to rely on the story and acting to keep the audience interested, not just pretty pictures. Go back to RAW when the couple walks arm in arm into the sunset.

As always, it's not about what's "best". It's about what's best for the job at hand.

John Carroll
June 10th, 2013, 01:19 PM
Nice example Jon... that makes a lot more sense than shooting everything that way. Play to it's strenghts!

Evan Donn
June 11th, 2013, 12:35 AM
The ability to shoot with less discipline or knowledge with the intent of "fixing in post" seems like a slippery slope to me.

Sure, but if you don't have the discipline or knowledge you're probably not going to get it right in camera either. If you know what you're doing the post work you do with your footage isn't a 'fix' but rather a key part of the entire imaging workflow - and having the greater range of data available for that step allows it to be more effective. For instance - not having to ND a window on location, or set up more lights inside to compensate, because you know you have enough dynamic range to capture both interior and exterior at once and can adjust them to suit in post, isn't a 'fix' for an error - it's using the entire range of tools available at your disposal to work most efficiently. And that doesn't mean you'll never ND a window again - it just means you've got a wider set of options to choose from based on each particular project's needs. You're right that raw isn't a holy grail that makes everything better, but it's one more tool in the toolbox (and a powerful one, at that) which can help you get the best possible results.

John Carroll
June 11th, 2013, 08:04 AM
Good point... RAW is an outstanding way to go if you are shooting in an uncontrolled light situation. That has been my position all along.
Does every shot need to be recorded RAW? Probably not. Depends on your delivery. If your ultimate delivery is for web, you are really wasting your time shooting RAW in a controlled light situation. (Web-based work is 95% of what I do, so it doesn't make sense for me, there are some exceptions of course, but not many) If you are shooting a feature film or broadcast project, probably a different story.

But again, if you simply want to shoot everything RAW, knock yourselves out and have a great time. If nothing else it will be a great way to fine-tune your workflow for when you do really need it, and have a better understanding of what it will and won't help you with.

There won't be any regrets for shooting RAW when it comes to the image quality, but I would imagine you would grow tired of the additional work if you can achieve the same results with an easier path... especially for web clients. Everybody has a different situation, so try not to feel insulted if I don't agree. Give it time... either you'll look back on this little thread and think this guy was a dumb a**, or maybe he knew what he was talking about after all.

Chris Barcellos
June 11th, 2013, 09:45 AM
John Carroll:

"If your ultimate delivery is for web, you are really wasting your time shooting RAW in a controlled light situation. (Web-based work is 95% of what I do, so it doesn't make sense for me, there are some exceptions of course, but not many) If you are shooting a feature film or broadcast project, probably a different story."

If you can tell the difference of a web posted film shot with raw (and you can) then this doesn't make sense either. There is no doubt in my observation that raw gives a sharper image and better resolving image. It also provides an easier to manage color correction process, even in controlled light setting.

I think your point is that you don't need the benefits of raw for your business model and work flow process, and the extra work is not worth it. That is cool, and acceptable. If its not broken, why fix it.

John Carroll
June 11th, 2013, 11:48 AM
We are still talking about 1920x1080 right? Lots of movies will most likely have been shot at 4K and at that point you aren't really comparing apples to apples.
But you're right... very big difference in the two images.

Jon Fairhurst
June 11th, 2013, 11:51 AM
Of course, it can depend on what type of web video you're producing. If it's "three guys in an office", h.264 is probably just fine. If it's "The Chameleon fights Flame Boy on Planet Purple", definitely shoot RAW - even if your initial lighting is highly controlled.

BTW, anybody notice how over-processed the Hollywood style is these days? I recently saw Star Trek Into Darkness and The Great Gatsby and was amazed at how cartoon-like the looks were for both films. The faces are so extremely noise reduced that they look like plastic 3D renderings. Of course, that helps during the scenes when the characters *are* 3D renderings as there is less of a jolt between live action and CGI.

Anyway, for that modern, over-processed Hollywood look, RAW would be the only way to go.

John Carroll
June 11th, 2013, 12:03 PM
Now you're just being silly... everyone knows that Flame Boy hasn't been anywhere near Planet Purple in ages!!

Jon Fairhurst
June 11th, 2013, 03:56 PM
Ahh. That's just what The Chameleon wants you to believe!

Regarding the modern Hollywood look, I think 300 mist be the first popular film to push the live cartoon thing. The green of The Matrix and the sepia of Oh Brother Where Art Though were strongly graded, but it was really color only. Now, it seems that the trend for post processing is extremely heavy handed. People who dislike the orange/teal look must be driven nuts by the plastic man thing.

But if you want to follow the trend, you want to shoot RAW.

What I'd really like to see is a super-saturated HDR look on a new comic hero film. Gotta sell the 5D and rent an EPIC for that!

Chris Barcellos
June 11th, 2013, 04:58 PM
We are still talking about 1920x1080 right? Lots of movies will most likely have been shot at 4K and at that point you aren't really comparing apples to apples.
But you're right... very big difference in the two images.

In my case, shooting with the 5D2, we are talking about somewhat less than 1920 x 1080 and in my case with my Black Magic Cinema Camera we are talking 2400 x 1350. But there is more to it than pixel count. It is a matter of what information the sensor provides that is preserved. By compressing to the small avchd file size, necessarily, I imagine some information is discard and other information is truncated. These avchd cameras do a fantastic job getting to that point, but a lot of what is preserved in raw, is gone.

Evan Donn
June 11th, 2013, 05:08 PM
The over-processed skin thing really bugs me - I just had to do some heavy noise reduction on high-ISO 5DmkII footage and ended up adding (better looking) noise back in afterwards to minimize the plastic face look. I do feel that's one of the advantages of raw though - noise comes out a lot cleaner and still looks good so there's less need to over-process it in post, whereas it tends to get blocky and smear-y once it goes through h.264 and gives it more of a 'video' look.

But really with the cameras we've got now and in the near future raw isn't the only choice. We shot a ton of stuff this past weekend on a C100 with a ninja2 straight to prores and the quality is amazing, even with only 8-bit output. With something like the BM cameras where you can do 10-bit prores internally I think there's probably far fewer situations where you really would need to choose full raw. Unfortunately for the 5D series we're not likely to ever see onboard prores through a hack so the choice is only raw or heavily compressed.

Jon Fairhurst
June 11th, 2013, 05:41 PM
Great point about low compression and 10-bit as an alternative to RAW. While 10-bit 4:2:2 might not be as good in the extremes as RAW, it potentially uses a lot fewer bits. That means you can shoot longer and pay less for storage and backups.

Specifically, for an improv, greenscreen project we're putting together, we'd like a nice, small-to-medium sensor, 4:2:2, 10-bit, moderate compression solution. The only reason we won't go RAW is that we want to let the camera roll for improv. That said, the original BMC still looks like a contender as it has more pixels in width than a 1080p Bayer cam. 3-chip 4:2:2 cams start to blow the budget.

BTW, I don't mean to hijack the RAW discussion. The greenscreen camera thread is here:
http://www.dvinfo.net/forum/open-dv-discussion/516709-best-inexpensive-green-screen-camera.html

Still, RAW looks like a fantastic way to get beautiful, alias-free, long focal length shots out of the 5D2. Buying a fast card is on my to do list as I want to check it out...

Evan Donn
June 14th, 2013, 07:06 PM
Still, RAW looks like a fantastic way to get beautiful, alias-free, long focal length shots out of the 5D2

Are you sure about the 'alias-free' part? Several of the mkII tests I've seen look much worse due to the increased detail, whereas the mkIII is very clean.

Jon Fairhurst
June 14th, 2013, 08:49 PM
I haven't shot RAW yet with my 5D2, but I believe that the 1:1 crop is alias-free; hence, my "long focal length" statement. The full frame stuff will still alias. But I have access to a Mosaic AA filter. :)