How long time do you keep back-ups?
 

Hi,

I am going through my systems and the difficult question comes when to delete back-ups of earlier projects. Would be interesting to hear how much back-ups you save? Example: dvd projects sold to customers.

Long ago I made the decision that when I'm done with a project I'm done. Delete and move on. I will save the Premier Pro project files since they're small and in theory I can reconstruct it from the project file, but I've never done that.

I'll keep edited masters on tape forever and I'll do the same for DVD .iso files. Since I need to make additional DVDs of past projects, I'll keep the .iso files on hard disk for about a year then I'll move them to DVD for archiving.

I tend to be more of a pack rat. I'll take the source footage, project files, and finished output and archive to a combination of tape and DVD. I store these items in my safe deposit box at the bank, so that I have an off site backup in case something happens.

I've been exploring using Jungle Disk software to backup to the "Cloud". It's fairly inexpensive and provides an easy solution. YMMV.

Varies.

For paid jobs, I keep the edited master files forever. Source stuff deleted after a few months. For my junk, I tend to delete stuff sooner. I just don't have the space I wish I had. But I've got stuff for work that's been brought forward from VHS cassettes. Nature of the business.

I just keep buying more firewire drives and save pretty much everything... I have about three milk crates stuffed full of them :)

Archive vs Backup
 

I've always thought of back-ups as what you keep to protect against a "system" failure of some sort - be it a hard drive failure, a mistaken mouse click, I/O controller failure, etc.

RAID systems are pretty good about saving your bacon in the even of a drive failure, but controller failures can and do happen. And saying "yes" to an "OK to delete this file?" message is more common than you might suspect.The bulk of file restoration requests in most companies is due to human error of some sort rather than hardware failure.

Another way of thinking of it would be that a back-up is what you do to protect work in process that's on your system

Archives, on the other hand would, I think, be a better description of what people want to keep around for years, or forever and would typically be preserved across the lives of several systems.

I make the distinction because I think it's important to separate the concepts of protecting against failure/accident and preserving assets for potential future use.

Having said all that, I'd really like to hear what your thoughts are regarding the archiving of source and edited material, and in a perfect world, what you'd like as back-up and archive solutions.

My interest is more than academic. As I've mentioned several times in past posts, I've been involved in the business end of the development and manufacture of LTO tape drives for over ten years, working with one of the "big two" tape companies. I've also worked on the development and marketing of high-end enterprise and open system RAID subsystems

As hardware prices fall and capacities increase, I think there may be opportunities for reasonably priced systems that would largely automate the back-up and archive process, and frankly, I'm trying to make a business case to some of the companies that I work with to investigate product opportunities for such systems. The more evidence I can collect, the better

You got a point there, Jim. I guess it's archiving we're discussing here. Archiving for potential re-use of material, both for clients and ourselves. A consistent way of keeping a small and effective stock. But I guess back-up is a very close, related subject...

Yes, in many ways they're very close. But I think the main thing that sets archiving apart is, as the anticipated time frame for retention and the intended usage, which puts different requirements on the choice of media.

Historically (well, at least within the 50 years I've been in the computer business) backups are not retained "forever" as the underlying files are still changng, ie today's customer data base file will have different information than tomorrow's version of the file. It has been quite common for example to take a full backup once a week, then take backups of just the changes to the file(s) for a week, then take another full backup and discard the intervening incremental backups and maybe keep the original full backup for another week as a grandfather so to speak. There are a lot of variations on this theme of course - sometimes keeping a couple of weeks of incrementals, and a few weeks of fulls, etc and also some variations involving storing some or all backups off-site.

Until recently, almost 100% of industry backup was to tape and tapes were physically moved offsite every day, again with a lot of variations.

More recently, with disk prices falling, backups to disk have become quite popular. Also, the sophisticated hardware controllers in high-end systems can automatically communicate over a network with other controllers and distribute copies of data directly

There have also been schemes involving logging of transactions to tape (or disk, but in the days of refrigerator or pickup truck sized disk units, offsite disk was undreamed of!) in such a way that the tapes could be played back and recreate the state of the database at any point in time.

Anyhow, the key to back up is that it's a way of restoring some or all data back to what it was at some point in time, although the underlying data has changed - perhaps in error!

Thankfully for the tape business, the US government has started requireing companies to keep e-mail and other records for 5 years or more. Besides helping keep me employed, this is a great example of an archive - not intended to be used to restore data to a previous state, but to preserve data for potential "repurposing" at a later date.

Another great example of archiving is what the petroleum companies do by keeping the tapes from the survey ships for years and years. As computers get faster and algorithms get better, they can periodically reprocess tha data and actually find new or previously overlooked oik deposits. I believe there is still a repository in Stavanger Norway where they keep copies of all the tapes recorded on the Norwegian shelf of the North Sea. Some of the petroeum industry tape repositories used to at one time run into the hundreds of millions of tapes.

I remember when I was still at IBM that when we brought out a new technology for tape drives we put a drive on one of the exploration ships and let it run in the Gulf of Mexico for a couple of months to be sure that the low frequency vibrations from the seismic arrays (they use compressed air not explosives to make the "boom") wouldn't disrupt the head servo mechanism

And to get to the point, I think another good example of archiving is what we do by keeping our tapes or project files etc around so we could if need be use the material in another production or re-issue the material, or make new copies for our customers, etc. On a larger scale than most of us, think of Disney wanting to re-release "Snow White" every 20 or 30 years or so.

Hard drives, for all their merits, are not designed to sit still on the shelf for years and years, but tape is designed with archival storage very much in mind - down to things like the chemical formulation of the coatings on front and back of the tape in such a way as to have the fronts be repulsive to the backs - in order to prevent fronts from sticking to backs when kept tightly coiled for years and years.

Long rant, I know, but hopefully it helps to clarify the differences between backup and archive.

Eek. (grin)