Archiving video. at DVinfo.net

Go Back   DV Info Net > Panasonic P2HD / AVCCAM / AVCHD / DV Camera Systems > Panasonic P2HD / DVCPRO HD Camcorders

Panasonic P2HD / DVCPRO HD Camcorders
All AG-HPX and AJ-PX Series camcorders and P2 / P2HD hardware.


Reply
 
Thread Tools Search this Thread
Old November 28th, 2006, 01:21 AM   #1
Regular Crew
 
Join Date: Apr 2006
Location: San Francisco/Paris
Posts: 121
Archiving video.

hello all,
I have a question about archiving our masters and MXF files from all the projects we shots using the P2 workflow.
I'm convinced DLT is the way to go, instead of buying more hard drive that take space and are more incline to failure.
Our IT guy, is convinced that hard drive are better because it's a cheaper solution.
he also think that we should compress the data ( what I mean by compressing is using ZIP or RAR or something like that)
Now, I've always heard this is a big no no in a post environment.
heard of stories where the data ended up corrupted when de-compressed.

My understanding of these compression format is very limited, this why I turn to all of you,
Am I crazy to think this is not a good idea?
loved to hear some real life experiences.
__________________
Sights Sounds Stories
www.tinboxtwo.com
www.reardenstudios.com
Eric Peltier is offline   Reply With Quote
Old November 28th, 2006, 02:14 AM   #2
Major Player
 
Join Date: Feb 2005
Location: Macau
Posts: 331
DLT is great. Just copy to it and use it when you need. Hard drives are okayish,but be sure to have two copies in two different HDD's. use them in regular intervals (once every month, do a virus scan on them or something that makes the drive mechanism spin), in order to prevent the drive motor to die. Store both mediums (DLT or HDD's) in a cool, dry and safe place (Humidity boxes).

With DLT, you'll feel a lot more safe than with HDD's, but its a lot more expensive than the HDD's. DLT is the industry standard, and used by the big corporations for their files backups.

Have you considered getting an Iomega REV drives? Those sound right. the company I work for has just invested on them. Haven't arrived yet, but sounds good on paper.

And don't forget Blu-ray. It should be a nice alternative for storage too.

Don't compress anything. Leave everything has they are. When you compress, you are changing something. If anything goes wrong in the compression and decompression process, its game over for your footage.
__________________
If you don't believe in your film, no one else will.
Sergio Perez is offline   Reply With Quote
Old November 28th, 2006, 03:31 AM   #3
Regular Crew
 
Join Date: Apr 2006
Location: San Francisco/Paris
Posts: 121
Quote:
Originally Posted by Sergio Perez
DLT is great. Just copy to it and use it when you need. Hard drives are okayish,but be sure to have two copies in two different HDD's. use them in regular intervals (once every month, do a virus scan on them or something that makes the drive mechanism spin), in order to prevent the drive motor to die. Store both mediums (DLT or HDD's) in a cool, dry and safe place (Humidity boxes).

With DLT, you'll feel a lot more safe than with HDD's, but its a lot more expensive than the HDD's. DLT is the industry standard, and used by the big corporations for their files backups.

Have you considered getting an Iomega REV drives? Those sound right. the company I work for has just invested on them. Haven't arrived yet, but sounds good on paper.
Thanks sergio
That confirms what I thought about compressing the data.
I did take a look at Rev pro media, sounds interesting but unfortunately the media can only hold 35gig max (as of today) and cost around $60 per disk.
DLT tape is 300gig and cost around $90.
I just did an interesting calculation to compare prices between hard drive and DLT backup.
Based on $1000 per TB which is what hard drive cost now.
and based on the price of the new Quantum drive $7000 + $90 per DLT tape
here's what I came up to.
Capacity(TB)---HD costs------DLT Cost
---1------------ $1,000----------$7,270
---2-------------$2,000----------$7,540
---3-------------$3,000----------$7,810
---4-------------$4,000----------$8,080
---5-------------$5,000----------$8,350
---6-------------$6,000----------$8,620
---7-------------$7,000----------$8,890
---8-------------$8,000----------$9,160
---9-------------$9,000----------$9,430
---10-----------$10,000----------$9,700
---11-----------$11,000----------$9,970
---13-----------$13,000---------$10,510
---14-----------$14,000---------$10,780
---15-----------$15,000---------$11,050
---16-----------$16,000---------$11,320
---17-----------$17,000---------$11,590
---18-----------$18,000---------$11,860
---19-----------$19,000---------$12,130
---20-----------$20,000---------$12,400
past 10TB DLT becomes much cheaper, for 20TB you pay $20k in hard drive vs $12,400 for DLT.
+ I'm not counting the time and energy (AC Power) it takes to run those hard drive.
We have at our facility around 7TB that needs to be backed up cause our Raids are getting full, I think DLT is definitely the winner.
thanks for your feddback
__________________
Sights Sounds Stories
www.tinboxtwo.com
www.reardenstudios.com
Eric Peltier is offline   Reply With Quote
Old November 28th, 2006, 03:32 AM   #4
Inner Circle
 
Join Date: Oct 2001
Location: Honolulu, HI
Posts: 2,053
I transfer my P2 footage to a mirrored RAID. It's based on Firmtek hardware. The drives are IBM/Hitachi SATAs on sleds which allow them to be hot-swappable. Ultimate capacity is unlimited except for one's budget.

In the event a drive dies, the second one continues to operate undisturbed. I actually did a test where I pulled a drive to simulate a drive failure while data was being written and the "good" drive continued to operate normally.

Plugging in the "dead" drive and reformatting it, then mirroring it, restored things back to normal in this test.

Writing data to a mirrored RAID is, essentially, making an instant backup. Storage cost is about $1.15 per minute or so. Including the cost of the drives, sleds and shipping.

Fast, reliable and economical.
__________________
Dean Sensui
Exec Producer, Hawaii Goes Fishing
Dean Sensui is offline   Reply With Quote
Old November 28th, 2006, 09:05 PM   #5
Regular Crew
 
Join Date: May 2006
Location: Anacortes, WA
Posts: 78
Those hard drive prices are a little high. A 300GB Maxtor DiamondMax w/16MB buffer goes for $119. That makes it aprox $440 per usable TB. So 20Tb is only $8,800.
Martin Iverson is offline   Reply With Quote
Old November 28th, 2006, 09:19 PM   #6
Regular Crew
 
Join Date: Apr 2006
Location: San Francisco/Paris
Posts: 121
Those price are based on Raid NOT just the hard drive.
Apple Raid are $12,999 for 7TB
LaCie Raid $1299 for 1 TB
Granite Digital $5955 for 6 TB

If you want secure backup Raid are the only way.
so no, those prices are correct.
e.
__________________
Sights Sounds Stories
www.tinboxtwo.com
www.reardenstudios.com
Eric Peltier is offline   Reply With Quote
Old November 29th, 2006, 11:36 AM   #7
Regular Crew
 
Join Date: May 2006
Location: Anacortes, WA
Posts: 78
RAID is inherently insecure and must itself have redundancy either built into the RAID protocol (ie RAID 50) or with a separate RAID system (ie. RAID 1). It seems illogical to use RAID for secure backup of master files.

It seems to me that the logical use of a hard drive for backup is to plug in a new unit, fill it with data (from a RAID), disconnect it, label it and put it in a secure storage space, the same as you would a master tape. Should that one brand new drive fail the data is much more likely to be recoverable than if it is spread across a RAID system.

This is just my opinion of corse Eric. I've been following and respecting your advise on this forum for quite a while and I offer this opinion in the spirit of contributing, not confrontation.

Last edited by Martin Iverson; November 29th, 2006 at 05:45 PM.
Martin Iverson is offline   Reply With Quote
Old November 29th, 2006, 11:53 AM   #8
Regular Crew
 
Join Date: Nov 2006
Location: Worcester. United Kingdom
Posts: 197
Quote:
Originally Posted by Dean Sensui
In the event a drive dies, the second one continues to operate undisturbed.
I really don't see this . . .

The whole idea of a raid is for speed - that's why they write to both disks at once. So how can this be viable - I'd love to know because raids don't write one file to one disk and another to the other disk - they spread the data between both at the same time - as I said, for speed.

And I agree with Martin - raids are for Editing rather than storage.

If I'm wrong, please educate me!

Andy
Andy Nickless is offline   Reply With Quote
Old November 29th, 2006, 12:29 PM   #9
Regular Crew
 
Join Date: Sep 2006
Location: Espa - Norway
Posts: 151
You're wrong! -;)

There are many levels and functions for RAID. What you describe is RAID 0, Then speed is the consideration. Than you have a total redundant RAID where data is written to two (sets of RAID 0 - sometimes) disks to have total backup. Then you sorta have everything inbetween. Some optimized for speed, others for security, some for both (Like RAID 5)

Gunleik
__________________
Red? Sure!
Gunleik Groven is offline   Reply With Quote
Old November 29th, 2006, 01:19 PM   #10
Inner Circle
 
Join Date: Oct 2001
Location: Honolulu, HI
Posts: 2,053
RAID 0 for speed.
RAID 1 for reliability.

The RAID I'm using for storage of master files is a level 1 RAID.

It writes identical data to two disks simultaneously.

This type of RAID is used for systems where reliability is a must. Such as banking data, air traffic control, etc. It's been around for quite sometime actually.

As mentioned before, I did a test where I pulled one of the drives while data was being written (to simulate a failed drive) and the data remained intact on the drive that was still running.

After the pulled drive was reformatted (to simulate installing a new drive), the data was re-mirrored and the system was back to normal.

With a RAID 1 you don't have to waste time writing to one set of drives, then to a second set because you're always writing to two drives simultaneously.
__________________
Dean Sensui
Exec Producer, Hawaii Goes Fishing
Dean Sensui is offline   Reply With Quote
Old November 29th, 2006, 01:49 PM   #11
Regular Crew
 
Join Date: Nov 2006
Location: Worcester. United Kingdom
Posts: 197
Quote:
Originally Posted by Dean Sensui
RAID 0 for speed.
RAID 1 for reliability. . .

. . . With a RAID 1 you don't have to waste time writing to one set of drives, then to a second set because you're always writing to two drives simultaneously.
Wow!
You learn something every day!

Thanks for that. I'll keep it in mind next time I need a new External HD.

Andy
Andy Nickless is offline   Reply With Quote
Old November 29th, 2006, 03:44 PM   #12
Regular Crew
 
Join Date: Apr 2006
Location: San Francisco/Paris
Posts: 121
Quote:
Originally Posted by Martin Iverson
RAID is inherently insecure and must itself have redundancy either built into the RAID protocol (ie RAID 50) or with a separate RAID system (ie. RAID 0). It seems illogical to use RAID for secure backup of master files.

It seems to me that the logical use of a hard drive for backup is to plug in a new unit, fill it with data (from a RAID), disconnect it, label it and put it in a secure storage space, the same as you would a master tape. Should that one brand new drive fail the data is much more likely to be recoverable than if it is spread across a RAID system.

This is just my opinion of corse Eric. I've been following and respecting your advise on this forum for quite a while and I offer this opinion in the spirit of contributing, not confrontation.
hey, no worries, this is exactly what this forum is about, I think something, you think different, sometimes I'm wrong, sometimes you are, in the end we both learn something.
e.
__________________
Sights Sounds Stories
www.tinboxtwo.com
www.reardenstudios.com
Eric Peltier is offline   Reply With Quote
Old November 29th, 2006, 06:09 PM   #13
Regular Crew
 
Join Date: May 2006
Location: Anacortes, WA
Posts: 78
You're right Eric and this time I have to correct myself. I edited my posting from so that it correctly states RAID 1 instead of RAID 0 for having built in redundancy. I knew that, just typed it wrong.

For those following along generally a RAID 1 will slow down your system as opposed to RAID 0 which will increase processing speed. RAID 1 is not recommended for video editing where processing speed is important. I run RAID 0 with 3 300GB hard drives and use a program called Carbon Clone Copy to copy the data on the RAID onto a single 750GB hard drive. The RAID gives me speed and the single hard drive gives me reliable back up.
Martin Iverson is offline   Reply With Quote
Old November 29th, 2006, 06:57 PM   #14
Inner Circle
 
Join Date: Oct 2001
Location: Honolulu, HI
Posts: 2,053
Here's the cost of a Firmtek solution:

Firmtek SeriTek/2eVEN4 External Bundle (External 4-Bay Enclosure & 4-Port PCI Host Adapter): $579

4 500-GB Hitachi/IBM SATA drives: $185 each x 4 = $740

Total startup cost for 2 terabytes of storage: $1,319.

Each additional terabyte requires a pair of Hitachi SATA drives and two Seritek "trays".
Drives: $185 x 2 = $370
Trays: $23 x 2 = $46.
Total : $416.

To get a total of 20 terabytes:
First two TB = $1,319
18 additional TB = $7,488
Total = $8,807

These drives can be configured in a variety of ways. Single drives, striped pair (RAID 0) or mirrored pair (RAID 1). And they can be set up in combinations as well. You can have one pair set up as RAID 0 while the other pair is RAID 1 in the same enclosure. Or you can go nuts and stripe all four drives as a single high-speed RAID.

Keep in mind that this cost analysis is for 20 TB of straight storage, not duplicated RAID 1 (mirrored drives). If you mirror the drives, the cost per usable terabyte doubles.

What a hard-drive system gives you, that DLTs won't, is random access. Locate that clip with a catalog system, plug in the drive pair, and grab it. No need to fast forward or shuttle a tape.

With any archive system, what's vitally important are backups. Even tapes fail. So those things have to be taken into consideration when designing a system for the long term.

And then there's upgrades. When something else takes the place of SATA or any other storage medium, how long will it take to migrate that data to a newer medium? For that matter, how many of us have migrated older tape formats to newer ones?

It's all scary stuff when you sit and really think about it.
__________________
Dean Sensui
Exec Producer, Hawaii Goes Fishing
Dean Sensui is offline   Reply With Quote
Old November 29th, 2006, 09:14 PM   #15
Major Player
 
Join Date: Feb 2005
Location: Macau
Posts: 331
Really scary, Eric. Bottom line, what we all really needed is a DVCPRO HD deck, but that's just too expensive. How about renting a deck, buying some tapes and copy your cards back to p2, then use the camera's FW port to copy to the deck? I really don't know how much a dvcpro HD tape goes for, but, concerning safety, having the footage on a playable tape were you can visually see the footage on a deck, always seemed the most reliable...

My ideal solution was to have a MXF backup and and a dvcpro HD TAPE backup- but that's insane and overbudget...
__________________
If you don't believe in your film, no one else will.
Sergio Perez is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Panasonic P2HD / AVCCAM / AVCHD / DV Camera Systems > Panasonic P2HD / DVCPRO HD Camcorders

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 12:46 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network