Video archiving at DVinfo.net

Go Back   DV Info Net > The Tools of DV and HD Production > Open DV Discussion

Open DV Discussion
For topics which don't fit into any of the other categories.


Reply
 
Thread Tools Search this Thread
Old May 18th, 2010, 02:50 PM   #1
Major Player
 
Join Date: Mar 2003
Location: San Antonio
Posts: 355
Video archiving

My company's IT guys laughed at me when I asked for 5TB storage on our system.
They suggested off site backup.
I was not really aware of such a possibility. My quick Goggle search as me reeling trying to dechipher that language. I have DVCAM and P2 HD QT and Content files that I need to save.

What should I be looking for in such a service?
Any recommendations?
Gary Chavez is offline   Reply With Quote
Old May 18th, 2010, 03:22 PM   #2
Inner Circle
 
Join Date: Feb 2007
Location: Tucson AZ
Posts: 2,206
I think the IT guys' job is to worry about this kind of stuff for you. If they don't want to do it, then maybe your manager and their manager need to have a friendly talk. This all assumes of course that the video is work related.

Biggest hassle I think you're likely to run into is the amount of data you need to move over a network to an offsite service. Moving terabytes is really slow - much faster to send the service a couple of hard drives instead of transmitting the data. So maybe a criteria for choosing a service would relate to whether they can accept data this way.

Anyhow, this isn't something that non-IT professionals should have to figure out for themselves in a corporate setting. If the video is a digital asset of the company, I think the IT guys have to take responsibility for it. That would certainly be where I'd start from if I were in your situation
Jim Andrada is offline   Reply With Quote
Old May 18th, 2010, 03:27 PM   #3
Major Player
 
Join Date: Mar 2003
Location: San Antonio
Posts: 355
You are preaching to the choir. It's a battle we lost. Thx for the suggestions.
Gary Chavez is offline   Reply With Quote
Old May 18th, 2010, 04:17 PM   #4
Inner Circle
 
Join Date: Feb 2007
Location: Tucson AZ
Posts: 2,206
Wow - that's bad! They must not think there's much value to the asset!

OK then I guess if it were me I'd call some services and ask them for estimates based on amount of storage and expected volume of data transfer etc. I'd also check if they could handle receiving the data on disk vs transmission and if that would affect the cost. I'd be particularly alert to how they say they would ensure the accuracy of what winds up on their systems as errors in transmission/copying etc do occur. I'd also want to know how (specifically) they would propose to make sure that the data doesn't degrade while it's on their systems. (I know the Harvard University Library says they have a couple of hundred TB of digitized material and they keep two offsite copies and they scan all copies regularly looking for dropped/corrupted records and when (not if) they find them they recover the data from the other two copies.) Anyhow, I'd want the service to be really detailed and specific about how they would make sure there was no "bit rot" going on as the data sat there over time.

Then I'd take the estimates to management. I'd also try to get the IT group to at least review the estimates and look for obvious holes. I guess what I would want to be sure of is that their fingerprints are on whatever solution you wind up with so when the data disappears it isn't just you standing there all by yourself!

Other thing I'd look into (although again it should be the IT guys who do this or at least help you with it) would be doing it in-house with an LTO tape drive either standalone or in a small library. The latest version which is just hitting the market is Generation 5 and each cartridge holds 1.5TB uncompressed or approximately 3.75 TB using built in non-lossy compression. At most you'd need 4 or 5 cartridges. and data transfer time would be a lot faster than transmitting the data over the internet.

The new IBM G5 drives have a feature (which was demonstrated at NAB this year) that would let you drag and drop files to the drive without any specific backup software. Quantum has had a similar feature in their G4 drives I believe although their implementation is somewhat different/more kludgey than IBM's. I'd expect them to adopt the IBM methodology as it has been approved by the LTO Consortium and IBM has said that will make the design freely available.

If you used generation 4 drives which have been available for a couple of years now, the capacities would be from 800GB to 1.6TB again depending on the effectiveness of the built in compression algorithms. Tape cartridges run around $30 each for G4.

I think a couple of sets of cartridges would let you store one or two sets offsite and tape really has better shelf life characteristics than small HDD's.

Heck, you could even keep the cartridges in a couple of bank safe deposit boxes!

I'm glad I don't have your problem! Or your IT department.
Jim Andrada is offline   Reply With Quote
Old May 19th, 2010, 01:12 AM   #5
Inner Circle
 
Join Date: Oct 2001
Location: Honolulu, HI
Posts: 2,053
Quote:
Originally Posted by Gary Chavez View Post
My company's IT guys laughed at me when I asked for 5TB storage on our system.
If your IT people laughed at you, then the matter should be taken out of their hands and put into your area of responsibility. They obviously don't take your requirements seriously, are unwilling to give you due consideration and are ignorant of what you do.

That happened to a company I was working in. Our photo archives were under the jurisdiction of the photo department. When new management came in, it was turned over to the IT department. The IT techs didn't take appropriate care of the data and, after ignoring repeated warnings and as a result of carelessness, lost tens of thousands of historic photos.

It happened two more times. Finally, upper management gave the responsibility back to the photo department.

What were the warnings? #1: put all critical equipment on uninterruptable power supplies. #2: all critical daily operations and archive data need to go on mirrored raids. Neither was done. In fact, it was later discovered the data drive in the photo server was simply laid into the case. It wasn't secured at all.

Upper management eventually took my recommendations seriously. Better late than never. Hope that doesn't happen to you and all your work.
__________________
Dean Sensui
Exec Producer, Hawaii Goes Fishing
Dean Sensui is offline   Reply With Quote
Old May 19th, 2010, 07:42 AM   #6
Major Player
 
Join Date: Mar 2003
Location: San Antonio
Posts: 355
It must the nature of the IT personality to not value any needs outside their own.
I like the tape system and will explore that. The off site sounds risky. We didn't even discuss the disgruntled employee, the bank takeover, the data thief etc. Thx again for the help.
Gary Chavez is offline   Reply With Quote
Old May 19th, 2010, 01:35 PM   #7
Inner Circle
 
Join Date: Feb 2007
Location: Tucson AZ
Posts: 2,206
Gary

Bear in mind that I'm a bit biased in favor of LTO because I've been involved with its development and manufacturing for over 10 years. I really believe it is seriously good stuff.

By the way - all media is not created equal. Fuji Film, Maxell, IBM (rebranded but sometimes cheaper than the actual maker's stuff) are all excellent. I've seen some cartridges last for 20,000 to 30,000 load/unload cycles in the manufacturing line although the spec is "only" 5,000 cycles.

I've been out of the IT arena for more years than most people have been alive and Dean's story sounds so much like what I was used to in the "old days" - I would have hoped things would have improved in all these years. Technology improves, human nature doesn't, I guess.
Jim Andrada is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > The Tools of DV and HD Production > Open DV Discussion

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 12:54 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network