Archiving P2 Footage?

[Steve Rosen]

I think drives are the best too - they're cheap enough and small enough and fast enough that you can double back-up everything, even on the road - and you can buy them anywhere.

When I ingest footage into FCP I back up the Capture Scratch files to another drive as well.. It may be overly redundant, but I sleep at night.

[TingSern Wong]

Large enterprises - managing data centres, all backup their data on tapes. So do large video production shops. Nobody uses disks for that purpose. Disks are fine for working data. Once a project is finished - it gets archived ... and tapes are used for that purpose. The danger of removable disks for archival purpose is after 5 years of storage, the disk might not start up - motor problem.

As I mentioned previously, until the promised land of petabytes of petabytes of holographic memory storage becomes commercially available, tapes are still the best medium for long term storage of digital data.

There are affordable tape drives featuring automatic cartridge loaders as well. You can stack 4 to 8 blank tapes on the loader, and if you are doing a huge offload to tape, say, 4 disk RAID-0 drive (3TB) to 800GB tapes, the tape backup software and hardware will write on one tape, finishes, drops the first tape, automatically loads the second, write on the second tape, etc ... until the jobs finishes or it runs out of blank tapes (waits for you to load more tapes).

[Mikko Lopponen]

Well how about Raid 5? If a disk fails, no problem, just insert another one.

[TingSern Wong]

RAID5, RAID6, etc ... are not archiving solutions. They are meant to prevent a system collapse if you loose a disk. More important, you can't transport a RAID5 array to another place - unless that place has identical h/w setup for you to plug the RAID5 array in.

[Calvin Dean]

I've used tape back-up in the past. It was slow and inefficient. In order to grab one scene/clip, you had to grap an entire list. But that was 10 years ago. Perhaps it has improved. If today's tape back-up systems are fast and provide better managment, this would be a good option. Can anyone address these concerns?

[Barry Green]

Quantum makes DLT and LTO tape drives that address your concerns, yes. The drives are MXF-aware and have been specifically engineered to address the needs of the video community. The drives can seek out and restore one particular MXF file, or they can even extract just a sub-portion of a file and wrap that sub-portion into its own MXF wrapper.

The DLT version is capable of data transfer somewhere on the order of 40 megabytes per second; the LTO3 version does it in as fast as 80 megabytes per second. The tapes are huge (200 to 400 gigabytes, I think) and comparatively cheap (I think they're around $100).

[TingSern Wong]

Thanks, Barry. I was just about to mention Quantum SDLT tape drives. For more details - see here.

http://www.quantum.com/Products/Tape...00A/Index.aspx

[Calvin Dean]

I used DLT IV's ten years ago. As I mentioned in a prior post, they were slow. Head clogs presented other problems. Have newer DLT's improved on this?

Glad to know they're faster now and that a single file can be restored.

[TingSern Wong]

I understand that the Quantum is using SDLT now, not DLT. In my workplace, they are backing up midrange servers using DLT tape drives as well. Haven't heard any complaints about head clogs or DLT drives being slow.

I have no experience so far about this new Quantum SDLT-600a drive.

Generally, if you present a tape drive with lots of small amount of data to write (like, text files, small exe files, etc) - the tape drive will be slow. This is because the drive needs to stop, rewind the tape a bit, read the tape until last block, then starts writing the next block. However, with higher density tape drives, small blocks means there will be more gaps than data.

Dumping P2 or HD data to high density tape drives won't present that kind of problem - as the video data tends to be in large blocks anyway.

[Jim Long]

The last time that I was on the road (actually Europe), I resorted to using my laptop and writing the P2 card files onto DL discs. When I got back home, I thought about moving the files to a hard drive but they can fail but a disc doesn't. In a way, it's the closest thing to a P2 card---no moving parts.

[TingSern Wong]

I think we are confusing the issue here. Archiving data means offloading the data from an active storage media to something totally offline and kept offline for a long period of time.

Moving data from P2 card to a portable hard-disk, to a RAID array, etc ... to my definition, is not archiving. Moving the data to tape, DVDs, CDs, etc - this is archiving.

The medium of storage should preserve the data for at least 5 years - 10 years ... so I would rule out using hard-disks for that long term storage.

Most magnetic tapes for computer data can be read back even after 10 years of proper storage.

[Francesco Dal Bosco]

5...maybe 10 years...it's not a huge amount of time. We can see today
great copies of 80, 90 years old films from the silent era. Even very old 8mm home movies, from 1940's or '50s, if stored in a dry place, can show their shining grays or their wonderful colors today. It's seems strange to me that we can't have yet a solid, efficient, convenient data storage method in this super-technologic era. As a filmaker and HVX200 user I really would like to know more about this crucial subject matter. From some posts here I assume that hard disks are not a secure storage option. I'm also hearing about tape based storage methods but I would prefer to not come back to tape anymore. Assuming you can leave your data (your films) in a DVD or BD for,say, five years what should you do after this period? And what about the distributed DVD copies? How long they can survive?

[Dan Brockett]

Hi all:

Interesting discussion here. Has anyone heard of disc rot? I already have CDs from as few as five years ago, both home recorded and commercial that have oxidized and turned to rust. Unreadable. Anyone who is archiving P2 cards to DVDs and DL DVDs is in for a nasty surprise someday.

Hard drives seem even more fragile. Unless you have a long term cloning plan and actually clone them perhaps every year and keep a master and safety clone, you will lose data. Even if you do clone them every year to new, fresh drives, you still might lose the data. And of course, if an earthquake strikes and all of your drives are dashed into the ground from falling off of their shelves, ya got nothin'

The production company I was at for three years started archiving all P2 material to DVCProHD tape. Not easy, not cheap, but reliable as heck and compatible, for the time being, with lots of decks although as we all know, in five years, we will all quaintly look back on tape decks and say, "remember when we used TAPE? Weren't those the good old days?" ;-)

S-DLT seems to be the best option today but it too is a clunky and expensive one. Someone will come out with something soon (holographic RAM?) that will eventually be cheap, huge and reliable for a long term.

Dan

[Sulev Sepp]

Hi All,

Have You experience with Iomega Rev70 for P2 footage ( in future EX1 also) archiving? It is not cheaper than HDD per gb, but tape gives some advantage.

http://www.iomega-europe.com/home?p=4740

Transfer rate seems quite good, max 30MB/s

Maybe REV70 can be alternative until DVD DL and Blue-Ray/HDDVD do improvements in technical and in price side ?

Sulev

[Kaku Ito]

Originally Posted by Sulev Sepp

Hi All,

Have You experience with Iomega Rev70 for P2 footage ( in future EX1 also) archiving? It is not cheaper than HDD per gb, but tape gives some advantage.

http://www.iomega-europe.com/home?p=4740

Transfer rate seems quite good, max 30MB/s

Maybe REV70 can be alternative until DVD DL and Blue-Ray/HDDVD do improvements in technical and in price side ?

Sulev

I tested REV35 with firewire connection but the actual transferring rate wasn't that high. Plus with my Mac Pro, when I installed the iomega driver then the finder keeps crashing.

Also, answering to the other post, I didn't say bluray doesn't work but I said 50GB back up took more than 12 hours to make the backup. And it takes some time to get them back to your hard disk again.