Video Storage/Archive/Backup Solution at

Go Back   DV Info Net > The Tools of DV and HD Production > Open DV Discussion

Open DV Discussion
For topics which don't fit into any of the other categories.

Thread Tools Search this Thread
Old May 28th, 2010, 11:43 AM   #1
Regular Crew
Join Date: Mar 2009
Location: Colorado Springs, CO
Posts: 29
Video Storage/Archive/Backup Solution

We need to setup a video storage/archive/backup solution for our growing video library and am wondering how others do this and if you have suggestions on possible solutions. We work with a non-profit so the budget is not bottomless but we need something in place that is going to work long-term.

Currently there are two of us who need to access the library. One on a PC and one on a Mac (and no, neither of us will change ;) ). The library is on one of the original Drobo's, connected to my PC with network shares (far from ideal) loaded with 4 2TB drives. We are backing up to three 2TB external hard drives. So a single backup and nothing offsite. Reusing equipment we have would be an advantage but not essential.

Workflow is we copy from the library to our local computers and edit there. Resulting files and projects are copied back to the drobo and the local copies deleted. I have a second 500GB hard drive in my PC and we are considering getting an external FW800 drive for my Mac friend to use.

After some research these are some of the ideas I have come across that may contribute to a final solution. Put a small gigabit switch between us, buy a DroboShare and connect the Drobo to the switch. Understandably this will be limited by the Drobo's USB connection. Alternatively upgrade to a Drobo FS which has gigabit ethernet built-in. I imagine there are many other solutions other than the Drobo but I like being able to use any size drive together.

For archiving old footage when the library fills up (which isn't far off) there are two possible solutions I have come across. Burn to blu-ray disks or get a hard drive dock and archive to hard drives. Either way we would be doing 2 copies so one would go off-site.

We would appreciate any advice and suggestions. How are you managing all this?

Many Thanks!
Darren Burns is offline   Reply With Quote
Old May 28th, 2010, 03:45 PM   #2
Regular Crew
Join Date: Mar 2007
Location: Denver, CO, USA
Posts: 85
+1 on the Drobo devices. They are easy to setup, expand and use. The customer support is also top notch.

For archive, I'm using Jungle Disk to backup to Amazon's S3 storage. This gives me an off site backup and can be scheduled to do its work in the wee hours of the morning when I'm not using the Internet connection. I backup active projects and archive completed projects to the "cloud". This gives me the peace of mind if the laptop is stolen or something happens at the home office.

This route is more expensive than archiving to individual hard drives, but Amazon distributes the data over several hard drives, so I'm protected if/when their hard drives fail.
Roger Wilson is offline   Reply With Quote
Old May 28th, 2010, 03:46 PM   #3
Join Date: Aug 2009
Location: Chicago, IL
Posts: 1,554
Ideally, you want to use iSCSI, which one of the Drobo versions has, but I don't know how robust it is. If you search my name and "iSCSI", you will find a lengthy thread where I help someone with their options and explain what iSCSI is and does. Here is a brief explanation: iSCSI is used for SANs, uses gigabit ethernet and is similar to fibre networks, ableit at a much lower cost and reduced speeds. The optimum iSCSI setup includes 2 gigabit ethernet ports on each computer, gigabit switch and 2-4 gigabit ports on the storage device. Using 2 gigabit ports on each computer allows you to "Bond" the 2 ports increasing speed/bandwidth. Furthermore, iSCSI appears as a local drive to each computer rather than network shares (it appears you already know the negatives to network shares). For certain programs, the 'local drive' is more reliable and provides faster access. With iSCSI, you won't need to download the video files anymore because you can edit directly from the storage device.

For good iSCSI devices, Thecus provides several affordable and high performance options.

I personally don't like the Drobo for a few reasons: 1) uses software Raid rather than hardware Raid 2) has FAR slower transfer and random access speeds due to mixing of different drives and/or software Raid 3) 100% propietary, so if the Drobo unit dies, you must get another Drobo if you want to access your data. Whereas with a hardware Raid unit, most parts can be replaced easily 4) due to software Raid, the rebuild times due to a dead drive will be far longer than hardware raid (eg, with 8TB of data, a hardware raid rebuild can take up to 2 days)

For a single person editing, the Drobo wouldnt be a bad idea. But for 2 people editing, the lack of its performance can cause issues, especially when both editors are accessing data simultaneously.

EDIT: I just read that the DroboPro's iSCSI can only be used with one target/computer.

For true long term archival, the only option is LTO tape. Hard drives and Blu Ray/DVD are not designed to last very long. A used hard drive sitting on a shelf for 6+ months starts to rot (there are technical terms for this).

I looked into Amazon's S3 service, but for large amounts of video, it would be more expensive than LTO on an annual basis.

Last edited by Steve Kalle; May 28th, 2010 at 04:19 PM.
Steve Kalle is offline   Reply With Quote
Old May 28th, 2010, 05:10 PM   #4
Inner Circle
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
Steve that's money right there. I am archiving to BluRay short term. Hopefully, a more legit option presents itself soon, or I'll be rotating BluRays annually. LTO is just not a viable option for me at this time.
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old June 7th, 2010, 02:00 AM   #5
Inner Circle
Join Date: May 2007
Location: Los Angeles, USA
Posts: 2,095
I have a Drobo.. and I still have. It's a great idea as long as everything works. You would think your data is safe since having two drives died simultaneously is very unlikely.

And one day you will find your Drobo stop behaving. You will be lucky if you are still within the first year warranty service. After that, if you don't pay for Drobo Care annually, things will be gone.. all gone.

Your Drobo grows bigger and bigger.. All your files are there now. It gets so big that you won't be able to back things up.

So now this is my solution for the growing data storage need
Trayless Removable Harddrive System | L.A. Color Blog

My Drobo is still in use. But I have to use additional harddrive to back up things in the Drobo. For the first year, I have 3 drobo disasters. Many wedding footage are lost.
LA Color Pros Blog
RODE Authorized Reseller . Comer LED Camera Lights . TakyBox HTML5 Menu Generator
Taky Cheung is offline   Reply With Quote
Old June 22nd, 2015, 09:54 AM   #6
Join Date: Sep 2013
Location: Bristol
Posts: 4
Re: Video Storage/Archive/Backup Solution

I've made some discoveries recently that have helped to address the issues of:
Storage of video files
Backing up projects
Archiving projects, and
Sharing projects across multiple computers

I made the video above to explain my findings in the hope it helps others. You can also find more details and links in the article on the New Vision Media Blog: Tips for Filmmakers | Video Storage, NAS & Bittorrent Sync

Hope this helps.
Jamie Wonnacott is offline   Reply With Quote
Old June 23rd, 2015, 09:43 PM   #7
Inner Circle
Join Date: Feb 2007
Location: Tucson AZ
Posts: 2,159
Re: Video Storage/Archive/Backup Solution

+1 on LTO. Generation 7 is within a few months of shipping with 6.4TB uncompressed per cartridge.

If you're going to store hard drives on the shelf you WILL encounter "bit rot" sooner or later. As long as hard drives are running in an array the system constantly runs checks on the data and repairs errors. Take the disk offline and nothing monitors or repairs it.

No user writable optical medium is truly archivable.
Jim Andrada is offline   Reply With Quote
Old June 25th, 2015, 09:35 PM   #8
Inner Circle
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,032
Re: Video Storage/Archive/Backup Solution

Spinrite is great for maintaining HDDs against bit rot / magnetic flipping.


Andrew Smith is offline   Reply With Quote
Old June 28th, 2015, 08:44 PM   #9
Inner Circle
Join Date: Feb 2007
Location: Tucson AZ
Posts: 2,159
Re: Video Storage/Archive/Backup Solution

Exactly - you have to mount the disk and power it up to run Spinrite - as long as the drive is just sitting on the shelf it is going to have problems that will be undetected.

If you want to store HDD's you should (must???) periodically power them up and run something like this to check them.

People are still using tapes that have been on the shelf for 40 years. Tape cartridges are designed to sit on the shelf for years. HDD's are not. Tape error correction encoding takes into consideration the ways in which a tape can be physically damaged - for example scratched longitudinally. So the data are scattered in such a way that the data and the associated ECC codes are never stored in the same straight line.
Jim Andrada is offline   Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

Omega Broadcast
(512) 251-7778
Austin, TX

(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

(800) 238-8480
Glendale, CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > The Tools of DV and HD Production > Open DV Discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search



All times are GMT -6. The time now is 09:52 AM.

DV Info Net -- Real Names, Real People, Real Info!
1998-2015 The Digital Video Information Network