Archiving Old Tape Formats
 

A buddy of mine raised an interesting question the other day and I'm not sure I have a good solution for his problem, I was wondering if someone here might.

The company he works for has a long standing relationship with a particular client who likes the idea of keeping every bit of footage shot for them around for future use. Problem is that a large portion of the early footage they have for this company was shot on Panasonic M2 tapes and the last remaining decks they have around that can play that format back are getting very old and are eating up space in the dub room for no apparent reason. They'd like to get all the footage off the old M2's and transfer them to something else for archival purposes but are stuck with making a decision of transferring 100's of tapes worth of footage to another tape format or placing all that footage on redundant drives.

What is a good way to go about doing something like this? If they go with drives, what codec would you recommend?

If they go with tapes or some other type of recording medium what would you recommend using that will be with us for quite some time?

We've been debating it the last few days and haven't come up with a good solution. Anyone know of anything?

BluRay data discs?(pricey per GB) LTO?(kinda old technology) Something I've never considered?

Took me a few moments to even find what this format was.

MII (videocassette format) - Wikipedia, the free encyclopedia

Given that it is standard definition, going with the DV codec could be a natural choice.
WD MyBook Essential 1 TB USB 2.0 external drives can be had for $125 USD.
If you use drives in pairs for redundancy, that's 72 hours of video storage for $250 USD.
And the space is far smaller than the boatload of tapes you have now.

I'm not sure how I feel about a long term archive "living" on drives, unless the usual archiving steps are followed ie. fire them up every xxx months/years and "exercise" them.

DV codec is certainly an option. I personally would probably look at a less lossy format, given that MII is a component format and should have quite reasonable quality, assuming that the issues that plague archived analog tapes haven't taken their toll.

Ahh, see now we're getting somewhere. I don't particularly like the idea of archiving on drives myself and this is why I have a hard time recommending this to my buddy. I also don't really think DV is the best codec route to go for the very reason you pointed out and that's why I was looking for other opinions. What codec would you use knowing what you do about MII?

As far as the problems of analog archive tapes, I think they are planning on cleaning the heads of the deck every few tapes to guard against too much buildup of the metal oxide or whatever magnetic substance that MII used since that stuff tends to separate from the plastic backing over the years.

If not drives, what would you go with? LTO tapes are an option but wouldn't they have some of the same problems storing on analog tape would have, namely the magnetic tape breaking down over time? Would BluRay be better because of this reason?

LTO also has the added issue of being a smaller tape base which MIGHT lead to even more rapid physical media breakdown than the original MII tapes. The advantage is that the INFORMATION is digital and less prone to degradation than the original analog media but that 1/4" tape size... long term storage? umm....

I would recommend BluRay-Rom (or whatever it's called) IF there was greater market acceptance than there is right now. I'd hate to see it be a fizzle.

As far as codec, I'd probably go with uncompressed OR a good high bitrate digital intermediate. Of course, the more common the codec (uncompressed) the greater the likelihood that it will be supported when you need to grab media.

I wish I could provide better advice but I'd definitely try to hold on to as MUCH resolution (spatial and temporal) and colour information as absolutely possible.

Two years ago, I would have suggested a transfer to DigiBeta.

Excellent point about uncompressed. The file sizes for 100 + tapes (many of which are the large 90 minute variety) would be staggering I'm afraid, but maybe that's just something theyll have to deal with.

What has been the accepted practice for this type of thing? Transferring to another tape format?

At my former 9 - 5, we had a bunch of old non-SP 3/4" U-Matic (in the 100's). What we would do with those was a transfer to DVD, mostly because the quality was SO terrible, as calls for access came up.

I'm not sure what I would do in similar circumstances. I've now walked away from 2 U-Matic archives and only need to face MY archives of Betacam, BetaSP, BetaSX, Hi-8, S-VHS, DV, DVCam and HDV now...

At the risk of sounding glib, any solution is a good one UNTIL it fails. Hard Drive, DLT, BD-R, digital video tape... All require SOMETHING to keep them viable into the future. Perhaps once Flash memory prices drop a bit more, THAT may be viable as well...

I wouldn't transfer them to another tape based format. That's just asking for trouble. Tape is on the way out. I also wouldn't transfer them to DV codec data files. I did a test the other day between the following SD codecs:

DV (180.7 mb)
ProRes (234.9 mb)
ProRes HQ (334.4 mb)
8-bit Uncompressed (840.8 mb)

It was amazing how poor quality the DV footage looked compared to the others. Most people had trouble telling the difference between ProRes, ProRes HQ and 8-bit Uncompressed, but EVERYONE could instantly tell when they were looking at the DV footage.

As MII is basically the same as Beta SP, I would recommend ProRes for the codec, but you'll have to decide how to archive the data. I've heard lots of people raving about the LaCie Rugged drives. Saw a guy speak about videography in Iraq and he stored everything on Rugged drives. Hasn't lost data yet and he's been using them for quite a while now in far more demanding conditions than we will ever use them.

The scariest thing I ever heard about archiving to hard drives is when a reputable source told me that storing a hard drive on the shelf is kind of like storing a car in the garage. Over many years, the car's internal engine components will start to rust from lack of use. He told me the same is true with a hard drive. He went on to say that his friends would get together once a month and have "hard drive spin up parties". (what a geek!)

Anyway, now you know everything I do about hard drive archiving. (but it's still what we're using. Western Digital My Studio portable FW800 drives. Buy them at the local Best Buy)

My only concern with ProRes is that it's tied so closely to Apple and FCP. If they were to change NLE's or ProRes support disappeared tomorrow then what? If I knew they'd be FCP for the next 10 years then ProRes sounds great. Maybe I'm over thinking it.

If they switched platforms (switching from Mac to PC) then that would be a problem. But ProRes is virtually guaranteed to stay in the Quicktime "bucket". When is the last time you can remember Apple ever "removing" a codec from Quicktime. I can't remember them ever doing it.

But yes, you're correct. If Apple was to declare bankruptcy, or get bought out by Microsoft (yikes!), then you'd have to transcode all that data. That would be a major pain. (let's hope that never happens)

I can see your point about DV. But you should probably chose DV Stream as it's a cross platform codec. I just can't get past the quality hit going to DV. MII actually looks pretty darn good. If I remember correctly, it was slightly better than Beta SP and they are both 4:2:2 color codecs.

I guess is the footage isn't that important, then the quality hit by going to DV wouldn't be so bad. But if it's not that important, why are you worried about archiving it? hehehehe :)

P.S. If you do the math, Blu-Ray is cheaper than DV tape.

It's not really my problem, I'm just trying to help out a former employer with his dilemma.

I'm not advocating using DV at all, the lowest I've recommend going is DV50 but even then I'd rather have something not in the DV family.

MII was one of those formats that was superior that didn't gain widespread acceptance. It was slightly better than Beta SP but for whatever reason SP won out. Happens.

Maybe I'll just advise him to go with ProRes and hope for the best. There isn't a good answer to this problem is there? I was hoping to post this on message boards and hear something I'd not considered but all this is doing is confirming my suspicions that 10bit uncompressed or ProRes is the way to go. I still feel a little funny about recommending LTO, but that seems to be the consensus on another forum too so maybe that's the answer since I don't trust hard drives even redundant ones.

I'd suggest DVCAM tape. It's relatively high quality (certainly equal to MII), would allow a component machine to machine dub (with TC), would probably be the most efficient both in cost and time and provides some future proofing. MII was a PIG and the reason that to this day I refuse to buy Panasonic. If you can get the MII decks to play (they barely worked on their best days anyway) then you will be lucky.

It's an interesting problem, and one not easily solved. One that Hollywood is facing as it brings it's catalog items forward from VERY old film. They've chosen Blu-Ray.

In my own work, I am bringing forward old material as well. About 100-150 hours of video. I also chose BluRay for a number of reasons.

1. The video/audio does NOT have to be stored in a playable format. This might seem trivial, but if the data has to be moved forward again, I won't need a special "player" to do it. Any computer with a converter can do it. This is one of the primary failings of tape-based formats.

2. Opticals are much less susceptible to atmospheric conditions. Heat, humidity, particles in the air, etc., can all affect tape and hard drives.

3. Magnetic sources like monitors do not affect optical media. Neither does X-Ray.

4. Water, even amounts that might flood a room, do not destroy opticals. Not so for tape, hard drive, or even flash.

5. With BluRay data storage, you can put the codec, the player, and a converter on EVERY disk. This greatly simplfies the method to move forward off the format. Essentially, you are making a self-contained transfer method right on every disk.


In terms of a codec, there are any number of good ones to use. Uncompressed is certainly an option, but probably totally unnecessary. Jpeg2000 is probably a reasonable option. You certainly want to pick something that is an SMPTE standard to ensure compatibility down the road. I tend to use Avid's DNxHD for my HD archiving, and will probably be using mpeg4, VC-1, or Jpeg2000 for SD. All compresses nicely and should allow you to put several hours of material on a single layer BluRay disk.

Just my thoughts on the matter.

DVCAM and DV are virtually identical quality. They both use the same codec. The only difference is that the DVCAM tape drive runs much faster than DV so there's less of a chance of drop out. MII is a much better quality codec than DV/DVCAM. MII is basically 4:2:2 8-bit uncompressed.