7D footage storing solution?
 

Hi,

This might be something that I should probably solve by myself but I'm curious about how you guys do it. As you all know 7D files are huge and especially take up a lot of space on our hardrives when are converted to prores.

Do you guys keep the "raw" video files or you get rid of them and stick with the prores ones?

I keep them but consumes a lot of GB and I don't think I'll have enough space on my desk for so many hardrives!

Last question: back up's, which is the best solution? More hardrives?

I hope it didn't sound too stupid!

Thanks in advance,

Javier

Here's the way I am doing it now:

I bought a 1.5 terrabyte USB drive for storing the original out of the camera. That amount of space should last me for a year or more. I keep all the original footage there in folders labeled by project.

I convert to ProRes 422, using MPEG Streamclip, and send those ProRes files to one of my other drives, and use that footage for editing.

The ProRes drives will fill up fairly fast, but i can add more as needed. However, once a project is totally completed and I'm sure there will be no revisions, I could dump the ProRes files if necessary. In theory I could redo the conversion to ProRes from my original in the future if needed, and Final Cut should recognize the new ones and reconnect. I haven't tried that yet.

I think it's always a good idea to keep the original. There may be something better than ProRes one of these days.

Also, I don't consider a hard drive a safe storage solution, so I back up all the original files to DVD. That's a bit of a hassle since it takes up to 4 DVDs for every 16 gig card. I might get a Blu-Ray burner eventually for that purpose. Some people are using LTO tape drives for that. I realize that a DVD is not a true archival storage medium, but it should be as safe as videotape for what I consider long term backup.

Thanks Bill,

That's what I thought. It's a good idea keeping the originals in a different drive. But as you've said hardrives are not the best choice for back up (I've had horrible experiences with them) and DVDs are quite scary besides I've red they have an "expired date" and plus, imagine the waste of time that means burning such an amount of them per day.

It's funny, it looked like with cards (hd storage) everything was going to be faster than with tapes and look at us... anyway...

LTO tape drives looks the way to go but they're superexpensive though. Nothing is perfect!

So it seems it's all about giga's!

I think I'll have to buy 2 dedicated drives for the 7D footage and for back up's... I'll pray God!

I'm using the same backup workflow, only that I'll be getting a BluRay drive soon. 50GB on a disc sounds nice as a backup for originals, plus supposedly BD discs last much longer than DVD's

Mauricio

I use bare SATA disk drives in a dock to archive the raw files. A 1TB disk drive costs under $100 now. Make a copy of the video files onto two disks & then put each disk on a shelf in a different location. If the footage is very precious make a 3rd copy or a 4th, whatever.

I don't muck around with burning DVDs any more. It is time consuming & they are very unreliable for archiving. Burning BluRay disks would also take too long. I am not sure about the reliability of these disks but they are in any case more expensive than using bare SATA disks for storage,

True about the fear of BD discs, it is still an unknown fact.

I've had a couple of hard drives died on me, and my feeling is that they are also very fragile, and unless you have a copy of the copy of your backup (meaning lots of $$ if you have several terabytes of information), the issue of data loss is still there.

On the bright side, I just opened up a couple of very old CD's (not dvd's) with some DV footage and data is still there. I had stored them carefully in a dry place, so perhaps that's the key to preserving them.

I guess that in the end it comes down to practicality in matter of money and space.

This may be a bit outside your budget, but here's what I've done.

• Drobo Pro - This is the big brother to the Drobo system. It holds up to 8 drives, right now I have 4 2TB drives in it. I have the Drobo setup with dual redundency so up to 2 drives can fail without losing any data.
• Internal RAID0 - I use this drive for active editing. Original footage is transcoded to CineForm HD using Prospect HD (you could also transcode to ProRes).
• Once the project is complete or when I want to make a backup, I move (or copy in the case of backup) the project from the internal RAID0 to the Drobo Pro.
• I use JungleDisk Backup software to backup the data on the Drobo Pro to the "Cloud" using Amazon Simple Storage Service (Amazon S3). This provides me with an off-site backup in case of major disaster. My S3 bill is about $50 per month right now but I don't have to hassle with DVDs and offsite physcial storage. At some point I will probably change this to not backup the intermedary files. The down side to this is that if I have to restore from the cloud it will take some time to download all the files.

Well, I asked about your workflow but I didn't explain mine!

I'm using 2 Lacie Raid HD with 4 interchangeable 500GB internal disks (2TB each). I use one of them as a backup (silverkeeper soft does the work for me.)

Like the Dropo Pro, 2 drives can fail without losing data.

I like the idea of the internal hd for active editing and then back up the project. I think is a reasonable workflow and right now looks like the safest way without investing a fortune (it's a bit expensive though).

It is still not as safest as I'd like it to be but is a good starting point.

BD could be a solution for a second backup but is too time consuming.

I'm using RAID 0 but transitioning to RAID 5 for my online editing. I'm starting to use RAID 5 because it's potentially as fast as RAID 0 but if 1 drive fails I can 'hot swap' the bad drive out for a new one. I'm trying out the OWC Qx2 for with eSATA interface but jury's out on that because I had a problem with it, getting an exchange soon. I'm also probably going to convert my 4 internal Mac drives into a RAID 5 using a RAID card and some special SATA adapter cables.

To backup the online storage, I use Chronosync to back up various drives or directories to some Drobos overnight. Those are nice units but unfortunately too slow for much online editing. If my RAIDs go down they would work for editing in a pinch, but are like 1/6 the speed of the RAIDs. They are nice units and don't require matching drives and have good software.

I also back everything up to the 'cloud' in case there's catastrophe. Again, would be some time to restore, but at least it's not lost forever in that case.

All this seems pretty extreme, but there's nothing that's irreplaceable other than your work and the uniqueness of the recordings so don't skimp on backups. Hard drives can and do fail all the time.

I like the Drobo in a lot of regards and I have heard good things about Drobo for archiving. But what are some comparable products/solutions to that system?

i copy original footage from camera to 3 different drives. Two of those drives live in a box in the closet and hold the original .mov, a copy of the cineform DI's, and eventually all project info/files. Drives are cheap, so i have LOTS.

The third drive(s) are my project media drives, temporary, live in my workstation, and once the project is done all related material lives on the "back up" drives.

AND Additionally another hard drive goes to the client with a copy of media, project files, music, graphics, AE files, etc. they can do whatever they want with it.

Drobo for Editing?
 

I currently use an internal drive for editing, but I need to upgrade as I plan to start working with higher-quality footage (and the larger 7D ProRes files).

Should I get a bigger internal drive (i.e. 1 - 2TB) and then have a Drobo (or equivalent) to back things up to? Or is it practical to edit off a Drobo, set to a RAID 1 or 5, and then once completed archive all files to another drive for "shelf storage"? I don't have ahuge budget just yet, and want a system that can grow as my needs for space/speed increase. I also want the fastest-possible system for active editing, and frankly I don't know enough right now about RAIDs and HD-friendly storage, etc. to make a good decision on my own.

Drobo is kind of a unique type of solution. It is like RAID 5 or 6, in that one of the set of drives can go bad and it can keep functioning until you replace the drive. The difference between a Drobo and another RAID 5 system is that you can 'mix and match' drive sizes and types, which allows you to up the capacity of the Drobo over time as larger and larger SATA drives become available. With a standard RAID box, you have to have matched sets of drives, like 4 1TB drives, 4x1.5TB drives, and so on.

Unfortunately the least expensive Drobo is, in my opinion, too slow for a practical editing solution. It will work in a pinch, if you fast work drive goes bad, but it might be frustrating depending on the complexity of your edit.

Drobo has recently enhanced it's line of products, the next step up is the 'Drobo S' which has 5 bays, has faster processors, includes a eSATA interface, as well as FW800, and reports to be 50% to 100% faster than the standard Drobo. I was just at Macworld and talked with a Drobo rep about it. They admitted the standard Drobo was a bit to slow for editing video, but the Drobo S should do nicely. Drobo S, however, is about 2x the price of the standard Drobo, though currently there are a few rebates, coupons and deals floating around for them.

My personal solution, I have 2 standard Drobos, one at 2.7TB, one a 5.5TB: I'll be getting a Drobo S and take the drives in the 5.5TB Drobo and populate it with that. I still won't use the Drobos for editing, but I feel that if my fast RAID editing drives go down, I'll still have a usable Drobo S for editing in a pinch.

For my 'working drives' I'm using a combination of drives in my Mac Pro configured as RAID 0 (plan to eventually get a RAID 5 card and use all 4 as RAID 5), and I am using an external RAID 5 box, the Other World Computing Qx2 with 5.5TB of online storage. For any editing drives, SATA or eSATA is the best interface. Firewire 800 gets up to about 70MB/second speed, a single SATA or eSATA channel gets up to about 225MB/second, about 3 times as fast. A single SATA raw drive maxes out about 100MB/second, but in a RAID configuration, the speed goes up as you add more drives working together to push the data back and forth.

For real speed hounds you can get a SAS (serial attached SCSI) card and box, and have the equivalent of 4 channels of SATA-speed connections from your computer to the RAID box, theoretically resulting in up to 1TB / second speeds or more. This might be overkill, but I see myself going to this in the near future.

Hope this helps, feel free to ask any further questions about this or Drobo.

you could always use an HDV cam - place the 7d footage on a timeline in Vegas and render to HDV to Tape out (record to DV tape). DV tapes are very cheap but you will of course be limited to the HDV spec.

I keep the original Canon 7D files and I also keep the transcoded copies from Neoscene. I like too have the originals as in the future better software tools might come along and these files can be reprocessed through them.

Storage is cheap I have 8 external drives for on and off site storage. I also buy different brands of hard drives just in case of a design flaw in one make. Call me paranoid!