DVD vs Hard Drive
 

Which one would last longer for archiving videos? Store them on DVD or on a Hard drive?

stelios

Re: DVD vs Hard Drive
 

Let me turn the question around a bit

How long do you think you will store them?

If you're thinking a few months or a year or two, either will probably be OK.

If you're thinking 10 years, I think neither one is reliable.

Generally speaking, I'd go for a hard drive, but be sure to make two copies and also be sure to check the contents periodically. Monthly would be good, but probably nobody will do it. At least check the data once a year.

If you're looking for real long term archive, best thing around is LTO tape, but the drives are $500 - $1000. After that the tapes themselves are competitive with HDD.

Re: DVD vs Hard Drive
 

Imo, harddrive.

No form of storage lasts really long and it is a lot easier to copy 1 TB of data from one drive to a new one then it is to copy 100 or 200 DVD's ...

Re: DVD vs Hard Drive
 

In other words you should replace your hard drives every 2-3 years to a new one and copy your data to this new one?

stelios

Re: DVD vs Hard Drive
 

What I really recommend is a small raid array - some of them automatically scan the data continuously looking for errors which they can recover on the fly.

The problem with storing data on the shelf is that there is no ongoing error checking and recovery.

So one scheme that can work is to make two copies to HDD, then after 12 months or so copy the data to a third HDD and keep cycling around the three drives every year - this way by doing the copy, you have confirmed that all data is readable and recoverable.

Just letting drives sit around on a shelf for a few years is a good way to risk your data.

Re: DVD vs Hard Drive
 

I'd go hard drive too. Easier to check it every so often and back it up a second or third time.

Re: DVD vs Hard Drive
 

A good argument for harddrives, having your files "online" you have continuous access to monitor disk status, do background integrity check and recovery as well as fast migration to new media. Consider having to copy 1TB of DVDs to new DVDs compared to copying same size a HDD.

Some say that it's not good to have HDD lying on the shelf, that they should be used to stay operable. I think it's a myth that might once have been true in ancient times, but I doubt it with todays hard drives: The mechanics is in a hermetically closed environment, I don't see why it should not deteriorate faster by not being used.

After all these different media types, storage cost in free fall and data exploding, I don't think there is such a thing as a permanent archive, instead one should plan for continual migration.

BR, Erik

Re: DVD vs Hard Drive
 

For 20 years I worked in computer storage and I vehemently disagree with those who say hard drive.The truth is that all hard drives fail, full stop. Given enough time, they will. It could be in a year or it could be in 10. The question is, do you want to bet on when? If so, what happens if you're wrong.

Optical is a better solution for archiving. Pardon me for being completely blunt but if you don't know the difference between fault-tolerance, back-up and archiving you don't know enough to intelligently ask this question.

What you are doing with your production content is running a mini-data center. One with a huge amount of data. The best way to find an answer you can comfortably live with is to learn a bit about computer data management. A little research will teach you that different media are appropriate in different stages in data workflow.

If you don't feel like doing the research, go optical, but tape is better.

Re: DVD vs Hard Drive
 

Any mechanical device tends to work better with use. Any of the experts I've heard recently have stated that bearings losing lubrication over time is the problem with hard drives.

Relying on a cheap hard drive to store your data longer term mightn't be the way to test a myth.

Re: DVD vs Hard Drive
 

Even if you put all your eggs in a titanium basket, it's still one basket.

Even tapes can go bad, too. Everyone has seen a tape get eaten.

Always make multiple backups, whether it's tape, hard drive or optical.

And no matter what media you choose, be sure there's something that can read it in the future. Because even if the media is intact, it's no good if the system is so proprietary that no one has a working machine that can handle it.

Re: DVD vs Hard Drive
 

I transferred some old Works documents from my old computer, which could be opened there by Word 2003, onto my new computer and Word 2007 can't open them (even though it's supposed to).

The same thing happened with some old scripts done on Scriptware, that program couldn't open them on my new computer with the same program.

I had to use the old computer to create files of these documents that would work on the new machine.

Re: DVD vs Hard Drive
 

Ahh yes, the infamous "future proofing" problem. This is a really big deal for archives. Ether you have to periodically migrate from older to newer technology or you have to get the PC related industries to stop making major changes/improvements every year or two. So since that's not going to happen, you're forced into a periodic migration mode.

One of the really good things about LTO is that in order to be certified to use the logo, all drives have to be able to read tapes written on two generation earlier machines and to read and write tapes made on one generation earlier machines. Also, we normally have three generations of drives in current manufacture so you should be pretty well assured that you will be able to buy a new drive that will read your old tapes for at least ten to 15 years after you write them. And service/spare parts will be available for at least 7 to 10 years after final manufacture. So if you write a tape today, you can good confident that you'll be able to read it back 15 to 20 years or more later. The new drives may use different attachment protocols etc than the old one (SCSI vs Fibre Channel vs e-SATA vs SAS vs who knows what) but the data will be readable). A hard drive on a shelf on the other hand offers no guarantee that there will be a system to plug it into even if the data is still good. Heck, we're already running into this with laptops no longer supporting Firewire for camera attach - even if you have a good tape in a working camera, no guarantee you'll be able to attach it to anything you can afford.

Which then sort of begs the question of whether the software that's out there in 20 or 25 years will be able to process the data that you just read back! Will Cineform still be around in 25 years? I hope so. I guess the key is to archive in really standard formats/codecs.

Also - I think we talk about archiving without really defining the length of time we want to be sure the data will be retrievable. The time horizon is really important - if our definition of an archive is something that I can recover a year or two or three later, it's easy. If we're thinking about digital assets with 50 or 100 year or more life spans, it gets to be a different kettle of fish altogether.

Re: DVD vs Hard Drive
 

I think that a key part of the equation with digital is that you can - and possibly should - re-archive over time, including validation.

Hard drives aren't a bad strategy - if you include double storage and re-archive every four or five years. The nice thing is that sizes keep increasing. You might have 500GB drives one year and 4TB drives down the road. Each time you re-archive, it's possible that you consolidate into fewer and fewer - and less cost-per-byte - devices.

My archive horizon is forever... or at least my lifetime. The horizon of the specific media may be shorter. And that means that maintenance must be part of the plan.

If LTO were random access, I'd be more prone to consider it. Years ago, my wife and I backed up on to tape. It was expensive and time consuming. And thank goodness we never had to access any of it. Because it wasn't random access, we saw it as an emergency recovery system rather than a useful archive.

That was about 15 years ago. Hopefully, the tools have improved since then.

Re: DVD vs Hard Drive
 

This is really frightening. Everybody now is converting everything into digital form, books, pictures magazines e.t.c. Will my grand children be able to open these archived files? I am now busy taking pictures and videos of my grand children and store them in DVDs and hard drives. Will they be able to open and see these when they become adults? Thank god that I have the old pictures that my parents took when I was a baby. I can still see them with out using any electronic "gadgets" Thank God for the libraries. Books written by Greek, Egyptian and other historians and philosophers can be read today.

stelios

Re: DVD vs Hard Drive
 

Well, yes, things have changed a bit

Current LTO drives permit drag and drop to copy files to tape. Not quit random access, but I think max high speed search time is on the order of 20 to 30 minutes (depending on the drive.) I'll check the details.

I know what you mean about old fashioned PC tape - when I moved back to the US from Japan I sold my blazingly fast 386 system to a friend and backed everything up to tape to restore on my US system. What a disaster! I stuck with it for easily 10 to 12 days and finally declared a victory when I had restored about 75% of my files.

Re: DVD vs Hard Drive
 

Yes, Stelios, it is frightening.

And I think the answer is that no, they probably won't be able to see any of your digital archive - UNLESS - you keep it online somewhere in the "cloud" and pay some service to keep it safe and viewable in a browser.

Or print it to paper!!!

Oh yeah - how permanent is computer printout? Really!

Many many many years ago I was very serious about large format black and white photography - to the point that I took classes with people like Minor White and Ansel Adams. And I learned how to process for archival permanence. It was pretty simple - just a matter of chemistry, and getting special storage boxes that were truly acid free. a bit laborious, true, but something you owed to anyone who would purchase one of you prints. Now the idea of permanence is really twisted. A lot of institutions are struggling with the same question of preservation of digital archives as museums and libraries convert their collections to digital. But somehow it will never be the same as paper. I remember in college having a class at Harvard's rare book library - and sitting at a table looking at Isaac Newton's actual handwritten notebooks. Right there on the table in front of me - what a wonderful sensation of historic continuity. The plus of digital - everyone can look at Newton's notebooks without getting up from their computer. The negative - it's sort of a so what experience. Nothing but bits, nothing genuine - only the palest shadows of the palest shadows of the real thing.

Oh well, maybe nobody cares any more.

Re: DVD vs Hard Drive
 

My system is fairly simple:

I store all source and project files on one HDD at the time of project creation. I can re-render if needed in future. As I fill a drive, I buy an enclosure and label the drive. Cost, about $0.10/GB

I store all rendered files on a separate HDD as I render. If my source drive fails, I have the rendered project. As I fill a drive, I buy an external enclosure and label the drive. Cost, about $0.10/GB

I store my most valuable renders on a SD card and keep separate from my computer (at office) in case of fire or other disaster. My understanding is SD cards are reliable long term, but obviously this hasn't been conclusively proven since they haven't been around for decades yet. Cost, about $1/GB.

Re: DVD vs Hard Drive
 

I still think inexpensive mirrored hard drives are the way to go. If you periodically consolidate your data, the cost effectiveness just keeps getting better and better over time.

When I started archiving to hard drives, mirrored 100GB drives were around US$50-150. Now I can get mirrored 1TB drives for the same price - and I'm starting to see 2GB drive prices heading in the same direction. With a dual hot-swappable enclosure it's a no brainer.

Whenever I see large drives dropping in price, I buy a set and consolidate all the data, and use the older/smaller drives for current projects. Very cost effective and efficient.

Re: DVD vs Hard Drive
 

Hard Drives over DVDs. LTOs if the budget allows for it. Other options: Digibeta and HDCAM (SR), etc.

The question really isn't: How long will it last; rather: How long do you want it to last and what price are you willing to pay for it?

Re: DVD vs Hard Drive
 

Not "how long will it last", but "how long before I need to do another maintenance cycle."

Re: DVD vs Hard Drive
 

Well Jim well said. I suppose I am going to print those memorable moments for them to see....


stelios

Re: DVD vs Hard Drive
 

But 19th Century glass plate negatives could be around even longer.

Re: DVD vs Hard Drive
 

I believe that Mel Gibson's "Apocalypto" was shot on high-definition digital video, using the Panavision Genesis cameras. Does this mean that in 10-20 years it will "loose" it's high definition or unable to play? Interesting thought...

stelios

Re: DVD vs Hard Drive
 

And look how long some cave paintings have been around.... :-)

Re: DVD vs Hard Drive
 

Is anyone here archiving to Blu-ray? That's my plan right now.

Re: DVD vs Hard Drive
 

The studios commonly make 35mm black & white RGB separations for archiving feature films.

Re: DVD vs Hard Drive
 

Re Glass plates - I've actually used them. Not for photos, but for astronomical spectrograph analysis.

At school, we had quite a collection of astronomical photos, etc that were taken starting in the mid 1800's (I think the first camera - telescope photo would date to around 1840.)

Glass is great - rather permanent, and stable enough for detailed measurements.

But - it doesn't like being shaken or dropped. Our plate stacks were a 5 story building inside, but isolated from, a regular 3 (or 4??) story building. The stacks had separate foundations and were mounted on springs and shock absorbers, and it was sort of like getting into an elevator to enter the stacks - there was a space of an inch or so between the inner building and the outer building.

Anyhow, I think the point is that archiving ANYTHING is not a toss it on the shelf and forget it operation. It takes management, and effort to keep the assets in as close to perfect condition as possible. Air conditioning and filtration etc for physical assets, continuous checking and copying for digital assets. Probably the easiest thing for individuals would be to store your data in the "cloud" and let them keep it safe (at a price) - for as long as whatever cloud company you use stays in business.

Oh - as I remember reading, cave paintings don't like people breathing on them or they deteriorate.

Re: DVD vs Hard Drive
 

And even then, it's a maintenance problem. The most valuable caves are sealed. You can only walk on planks. Etc.

Re: DVD vs Hard Drive
 

well then I suppose the moral of the story is:
enjoy them today as tomorrow might not be around...

stelios

Re: DVD vs Hard Drive
 

All archival work requires maintenance, paintings etc are regularly checked my museums etc. Although, It appears that digital media needs more intensive maintenance than other media, which does add a cost that needs to be budgeted for.

Re: DVD vs Hard Drive
 

I have just had a Hard drive crash on me, it was only 18 months old. Fortunately I only had completed jobs on it, nothing which I could generate more income from, unless a customer wants a single DVD copy. I don't think any media is futureproof. When I first started out in digital imaging I backed everything up on MO disks, which would last for 100+ years, pity the hardware didn't last that long. Now I have disks but no drive to read them with.

At least with photographs you have a hard copy. I guess with film you also have a hard copy, but with digital video you just have data bits.

Re: DVD vs Hard Drive
 

I always make full backups of everything I work on, and assume the striped RAID is going to crash.

At the end of the day current projects get backed up to a separate drive that's taken offline after the copying process is done.

And when a project is completed it gets moved to a mirrored RAID and added to the catalog for future reference. That mirrored RAID is also taken offline and stored on a shelf.

I've worked this way for the past 8 years or more and it's proven safe.

Re: DVD vs Hard Drive
 

As a rule I also back up material, sometimes, as in the case i mentioned, I do a job and keep it for one or two years and then delete. I am talking about the editable master file. The final DVD files are usually kept ad-infinitum. Or at least a copy of the DVD production is.

I think storage will be the Achilles heel for digital video, until a fail safe long term storage system is introduced.

Re: DVD vs Hard Drive
 

At the moment I am backing up my video projects on an external WD Elements desktop HD 1 TB as well as on DVD.

When the HD is nearly full I disconnect it from my PC and store it in a cabinet. How long do you think this external drive will last on the shelf?

stelios

Re: DVD vs Hard Drive
 

"How long do you think this external drive will last on the shelf?"

Not as long as you would think. Hard drives are like car engines, they need to be kept active otherwise they will just lock up. Generally you should spin up a hard drive at least once or twice every six months.

Re: DVD vs Hard Drive
 

Damned if you do and damned if you don't. Put hard drives on the shelf and lubricants dry out - but probably more deadly is the fact that bits rot away quietly. I've seen a lot of drives that were fine after long periods of inactivity - but also a lot where even though the drive powered up OK the data was corrupted.

In older disk drives there was an issue with the heads coming to rest on the drive surface and surface lubricant wicking into the head disk interface and effectively gluing the heads to the disk surface. It was known as stick-tion and sometimes it was so effective that the head sliders were torn off the suspensions. This was not good.

Modern drives typically "unload" the heads ie pull them clear of the surface on shutdown to prevent this.

Re: DVD vs Hard Drive
 

Many of the issues of digital are in this report by the Academy of Motion Picture Arts and Sciences

The Digital Dilemma | Science & Technology Council | Academy of Motion Picture Arts & Sciences

Re: DVD vs Hard Drive
 

Correction: YCM separations.

Re: DVD vs Hard Drive
 

After all these info, this weekend I have a lot of precious photos to print...

stelios

Re: DVD vs Hard Drive
 

You just say that tape drives suffer from the possible change of attachment protocols, and then you criticize HDD for the exact same problem!

Even if your tapes are readable and data are good, you might be lucky to find a drive that will read them but then there is no guarantee that you can plug the drive into any current system. Same problem as any other storage media suffer from.

BR, Erik