SDHC long term storage reliability
 

Does anyone know about the storage reliability of SDHC cards compared to, say, backing up to Blu-Ray or DVD?

I am going to trial using SDHC cards like I used to use XDCAM discs. Buy them and keep the footage on the shelf, or reuse when the footage isn't needed anymore.

This would eliminate the need for cumbersome backup to Blu-Ray and other discs, and also turn the recording medium into a disposable rather than an asset. They are so cheap now that it would be easy to get two cards and back up the footage that way.

Sandisk have already launched a write once version of SD that is said to have a storage life of 100 years. So I wonder if standard SD and SDHC is pretty good anyway. If normal SDHC can match the 50 years predicted lifespan of XDCAM disc then it will still be better than going to DVD etc.

Simon, very good question. Of course, nobody knows for sure and all life spans are estimates not by guessing but based on lab tests. Artifical ageing is the key word in that context. The main question however, is, are these estimates "conservative" (rather shorter life span) than not.
With the current pace in development I'd say 10 years is already good, 20 years would be marvelous. Everyhing beyond is beyond my imagination, because can I hope to be still alive then? So this question at a certain point turns more into a philosophical one then a technical.

Nevertheless - very good question. Thank you.

Simon, SanDisk is going to release (in 2009) a new WORM card with enough capacity for video storage. The shelf life is reported to be 100 years. They are supposed to make the announcement the first part of '09, whatever that means.

No doubt it will be expensive.

I, for one, am lookig forward to it.

Check out this thread:
http://www.dvinfo.net/conf/sony-xdca...deo-files.html

The storage life of the media is one thing, but whether the equipment to play the media still exists in ten, twenty, thirty years time or whenever is another.
If the playback equipment still exists will the media be editable with future editing programs?

It's a very good question indeed, especially with SDHC costs/GB reaching near equivalence with XDCAM discs. Surely there must be a research document somewhere? Before I committed to such for long term storage, I'd like something fairly official to reassure clients with.
Nothing is absolute, but I'd say that an SDHC or Compact Flash card is about as good as that can get. You could store a reader along with the media, and I'd be far more confident of it working in many years time than anything with moving parts. Maybe the USB connection will be the thing that will cause the most obsolescence problems? How long will computers have that?

I still have memory cards for my Playstation one that are from 1995/6 and they still work.. I have carts for my Jaguar that are from 1994 that have flash memory in them that still hold high scores.

I don't think that this is an issue. After all we are talking about files that can be copied faster than real time. Although it might be a pain you can still copy those files over to whatever new storage systems arrive on the scene.

But with SDHC that won't be for a long time simply because there are so many SD devices in circulation. Nothing like that ever happens over night.

The other thing that is favourable is that, assuming I understand things right, SD cards do not need to keep a voltage in order to hold data. So things like voltage leaks over time leading to loss of data shouldn't be a problem.

A loooooong time, simply because even with USB 3 there will be backwards compatibility, or adaptors to create compatibility.

Sandisk talks about a lifespan of 100 years. ok.

Please turn your memory 100 years backwards: 1908.

Electricity. Yes, in its infancy.

Telephone: Nope

Radio. Mr. Marconi, when did that happen first?

Airplanes. Will men ever be able to fly like a bird?

Computers. What is that?

Digital media? Even not in science fiction novels.

In 1908 my grand dad was an 8 year old boy. When he died,
he was 72. And he never understood what the Apollo landing
on the moon in 1969 really was.

So the real question here is, how long will be tools available
to read the data from the memory. And additional means to
decipher the meaning of that data, consisting of 0s and 1s.

Compare that with old papyrus fragments from ancient egypt,
some 2500 to 3000 years old. Your eyes can see the hyroglyphes
and your brains may be able to decipher their meaning.

It's the virtuality of modern digital data that makes it so
vulnerable to ageing, even more vulnerable than from the physical ageing
of the layer it is stored upon.

-end of philosophical rant.

The first large scale mains distribution systems started to roll out in the 1880's in the US and Germany, the telephone came into existence in 1876 with phones becoming widespread by the end of the 1880's. First true radio transmission was by Oliver Lodge in 1894. One hundred years ago they were transmitting across the atlantic.

What should be considered is how difficult it is becoming to find good Video 8 decks or betacam decks yet these are technologies that only went out of production 10 years or so ago. Whatever you use today to store your material it is likely that in less than 10 years time you will want to transfer it to something else before playback becomes problematic.

I bet 5 years ago PCMCIA users felt that was future proof. Give it another 5 years and it's going to be difficult to find a computer with a PCMCIA slot and I doubt there will be many people making adapters.

Peter, Simon has already addressed your "concern."

Yep. Just copy files to the new storage.

Another thing to consider is whether or not people in 100 years time would want to watch a video demonstrating how to use a carrot peeler from 2008 ;-)

Simon, with all due respect, things can be but they need not be as easy as you put them.

While you can project old 16mm b/w films from the 30s of the last century today, you'll find it difficult to read a 5,25" floppy disc from 1982 that contains a simple letter.
And given the amount of data that piles up with every new digital video project, who has the time and the money to pay for "to copy the files to a new storage".

While i can store 8mm films more than 50 year old in a dry dark cold room (less than 50% humidity), with low temperature (less than 5 degrees centigrade) and no
natural light for another 50 years, the same does not apply to digital data.
All major movie films are separated into RGB and printed to b/w films for archiving. Not the digital data they have been derived from, but simple old b/w film.

Yes, but one thing you are forgetting here is that we are talking about digital data. Films from 100 years ago are one thing, but here is one fundamental question. Where is the original footage with which to make a new edit of those films?

I'll tell you where. No where. It doesn't exist. So quite simply there is no parallel to the situation we find ourselves in now. But what I can say is that the situation is a lot more manageable. We have the internet so we have a lot of advanced warning about what technology is around the corner, and we can therefor prepare for it.

Difficult, NOT impossible.

You make the analogy of 5.25" diskss. Well I was using them and I managed to copy my data to 3.5" disks and then eventually hard disks. It wasn't hard. Like I said, if you keep up with technology and adapt your data there shouldn't be a problem.

Hell I can still get hold of a working ZX-81 if I wanted to! You can always get hold of old technology if there is *really* a need. But like I said, c'mon people, what is the likelyhood you will actually need *your* video in even 10 years time, let alone 25, 50 or 100!?

Well, I don't know about you, but I fully anticipate MY carrot peeling video to be discovered in some archive, several centuries in the future. They'll discover that my haphazard and rudimentary use of digital recording equipment and available light to be "novel" and "brilliant" and "either ahead of or behind my time" - but they won't quite be sure which.

I'm planning ahead for this, and am archiving every byte in stone with a hammer and chisel.

ROTFLMFAO!

Happy Christmas Ted, you just made my night! :-)

Coincidentally, after not looking at these forums for a few days, exactly this point came up yesterday with some friends we visited over Christmas. They (like many other people) bought a video camera when their first daughter was born, and now that she's coming up to 21 had the thought of looking at the videos again. That was when they realised that the tapes were S-VHS-C, all they had as a player was a VHS machine, and the camera's long gone.

I’ve ended up promising that if they can find the C adaptor, I can dub them from S-VHS to DVD, but the point is that when they bought the camera in the first place and spent a lot of time using it, they just assumed that there would be no problem in viewing the material in years to come – no more than opening an old photo album.

OK, if I hadn’t got the facilities, there are businesses which can do the conversions at present – but it would be at quite a cost, and that cost is likely to be more as materials get older and the replay equipment gets scarcer. I’d bet a lot that a great many AVC-HD cameras get sold every day for people to film such as babys first steps or words, and I’d equally bet that the people who buy them just assume that the images will be accessible whenever wanted in the future.

This all assumes the recordings themselves are fine, and it's just a question of getting the right hardware. I've recently looked at quite a few tapes from around that period, and so far, no problem. But what about newer technologies?
The best link I’ve found about flash memory principles is: Flash memory - Wikipedia, the free encyclopedia and the basic principle is that data is stored by the presence or absence of electrons on a floating gate, the state being read by how their electric field influences a transistor gate. That article says little about how long data will remain uncorrupted, other than:
So the big question becomes just what “many” translates to? I’d be pretty sure the data would be OK in a years time, and expect there not to be a problem in 5 or 10 years based on experience, but what about 25 years, 50 even?
Do you have a reference for that, Peter? I ask not just through academic interest, a client is trying to put a proposal together for a project where archivability is probably the most important single factor. The problem being faced is that any proposal has to be backed up with sound, referenced evidence. So a Sandisk salesman saying “oh, you can expect them to last 100 years!” is no use, a Sandisk White Paper saying the same thing with evidence would be hugely useful.

The other thing I’ve learnt to be careful of is the difference between physical lifespan and data lifespan. If you talk about a device having a lifespan of 100 years, it’s conceivable that you may still be able to write and read data to/from it in 100 years time, but any given set of data may only last a lot shorter period of time.

The biggest difference here though is that the information that have is stored on a linear storage device in analogue form. Whereas with digital files it will always be easy to transfer information as new technology comes along. Also the way in which new versions of Firewire and USB retain backwards compatibility means that for a long time yet information could be retrieved from such devices and cards.

Yes, there is that question, which is why if the data really is important that established industry proved archive solutions should be used such as DLT and LTO. This doesn't help the average family, but I think that is just a case of being vigilant, making more than one copy, and checking things regularly.

I'd imagine they have a white paper somewhere. The 100 year thing was in their press launch for the product.

Simon, you young whippersnapper, I remember using 8" floppy disks. And believe me, at that diameter, they were pretty floppy. (hehe)

Ok seriously, I subscribe to the same ideas about archiving. Just remember to keep transferring your digital media to newer forms of storage. I think the biggest issue facing us in the future is miniaturization getting absurd. It wouldn't take much to lose the micro-SD cards we already have available!

-gb-

Yes, the size is getting silly, and came up in a conversation the other day. Size is something that is as important as robustness in a production environment. It would be great if someone could manufacture an SDHC card in an Express card body for this very reason.

As you well know, that's exactly what P2 is. Albeit with a bit more circuitry to handle the RAID functionality. 4 SD memory devices inside although I doubt they are in their usual plastic enclosure. Which brings up another interesting memory of mine...

A few years back, there was an MP3 player on the market using a CF Microdrive for storage. When word got out among digital photographers, people were cannibalizing the players for the memory drive because it was cheaper to buy the whole player than the CF Microdrive card in stand alone form.

-gb-

Agreed in principle, but it is assuming that even with the right hardware, the original files are still readable. A bit of degradation on an analogue tape recording may mean a bit more hiss, or a few seconds missing. A bit of degradation on a digital file may mean "corrupt file, cannot be read"!
For a large broadcast organisation, then yes, DLT or LTO and active management, but as you say, not much help to an average family. Sometimes the best family memories are the ones that are come across by accident, after lying undisturbed for years. Unfortunate if it's a photo album where some of the pictures are a bit faded, but better that than a totally unreadable digital media?

The advice about making a copy and checking regularly is sound, but how many families do you think get told it when they purchase their AVC-HD SD camcorder? And how much does it say in the instruction books about media longevity? That's why it would be so valuable to have even an idea of the lifetime of the data on a standard flash card.
I've found the Sandisk white papers section, and can't find anything about this topic. Anything said about reliability seems solely to concern short and medium terms.

I took the 100 year figure to refer to normal flash memory, but now understand it refers to the newer write-once cards - SanDisk | Business Products | Flash Memory Cards | SanDisk 3D One-Time-Programmable (OTP) Memory – and not data on standard flash cards. If the OTP cards work as some are hoping, then they will indeed be close to the Holy Grail for many in the industry. To quote from a page linked to by Sandisk (SanDisk | Business Products | Flash Memory Cards | SanDisk 3D One-Time-Programmable (OTP) Memory ) “To program the memory, a permanent physical change is made to the structure, as opposed to storing charge, eliminating reliability concerns from charge leakage and data corruption. To program the memory, a permanent physical change is made to the structure, as opposed to storing charge, eliminating reliability concerns from charge leakage and data corruption.”

But the assumption seems to be that one of these cards will be able to be placed in a camera like the EX, and a standard (but unalterable) recording made. I’m not sure this is the case, since although Sandisk talk about compatability with existing flash devices, that may be from the point of view of reading from them, all that Sandisk say is:
When they talk about writing to the memory, they seem to imply that specialized technology is necessary:
I’d love to be proven wrong, and I suppose the best thing to do is write directly to Sandisk, but I strongly suspect that 3D-OTP is intended more at markets like memory for portable GPS devices, to store such as mapping data on. Come the upgrade, you just get a new OTP SDHC card and throw the old one away.

As far as video use, they could also be very useful as a Blu-Ray alternative.......