storage solution at DVinfo.net

Dan Farzad · July 24th, 2006, 05:14 PM

I have been running my small production company for about a year and half now.
Within this time I deciced to focus on corporate and non profit organizations. The major problem that I have with these projects is space. Unlike most wedding businesses where when your done editing you delete the files, for most part I have to hold on to all the datas of a project in order to use it for years to come in making their future videos and making the job easier but space is a problem.
till now I offered my client to provide me with an external HD where I create the project from beg to ending on it. I dont feel it efficient. and extra $250 for the client. past that issue I dont feel safe about any external HDs, every tme I turn one on, I'm affraid of it crashing etc since 2 have gone bad on me.
What do u guys suggest and what do u do?

Glenn Chan · July 24th, 2006, 05:38 PM

Some options would be:
A- Burn onto optical discs. The problem right now is that this takes forever.

B- Put material onto two different hard drives. Wait for rebate deals on hard drives. Enclosures can be had for a fairly low price now... even eSATA I believe.

C- Setup a storage server on a PentiumIII or better machine (yes, a pentium III will be fine). With a compatible RAID controller card, you could setup a RAID 5 which will protect somewhat against drive failure (although sometimes two drives will go down at once). RAID doesn't protect against user error.

D- There are computer tape backup solutions. High initial cost, but they let you have multiple backups and things like that.

Dan Farzad · July 24th, 2006, 08:02 PM

can i compress the data after it it done so it takes less space on a HD and reexpand?
how>?

Glenn Chan · July 24th, 2006, 09:27 PM

You could enable NTFS compression on the drive, or use an archival program along the lines of winRAR. However, you will get extremely little compression with video files. DV for example already employs DCT compression, so you will not gain much compression over that. Animation and Huffyuv codecs already utilize lossless compression, so file sizes after re-compression may be slightly (and insignificantly) larger.

It's generally not worth the time.

Matt DeJonge · July 25th, 2006, 02:54 PM

Dan, I agree with Glenn on compressing your files. The only advantage or using WinRAR on video files (using the "Store" option, which takes VERY little time) is that it will allow you to chop up the video file to 4.6 GB files for spanning to multiple DVDs. I do this once in a while when needed, and it works great. The only downside is the slow burntime, but newer DVD-RW drives are pretty quick now.

I find that storing the data on external HDs is pretty inexpensive and easy to implement, and you only need the drives on when you want to write or retrieve data, otherwise leave them turned off. That'll save on your "mean time before failure".

I will soon be getting a PC set up with SATA drives and will be going RAID-0. I've decided to mount (2) 500GB drives into 1 RAID-0 config (attached to a controller), then mount another 2 drives in a RAID-0 config (to a second controller), etc. This is cheap from the controller stand-point (you can get controllers PCI for about $65.00) and you'll have some redundancy because every second drive is striped. You can also build this machine up as needed. The only downsides are the size of the machine (it's a whole computer after all), and the possible noise. However, that's not too big of a deal, as I'd only boot the machine when needed (hibernat mode helps this).

Glenn Chan · July 27th, 2006, 06:42 PM

Quote:

I will soon be getting a PC set up with SATA drives and will be going RAID-0. I've decided to mount (2) 500GB drives into 1 RAID-0 config (attached to a controller), then mount another 2 drives in a RAID-0 config (to a second controller), etc. This is cheap from the controller stand-point (you can get controllers PCI for about $65.00) and you'll have some redundancy because every second drive is striped.

That's only if you do RAID 10... which is two pairs of RAID 1/0 sets, which are then RAIDed together as a RAID 1/0 array.

RAID 1+0 and 0+1 would be more accurate terminology, which indicates the order of the 'RAIDing'.

Matt DeJonge · July 28th, 2006, 12:42 PM

I think I might have misled you all with my wording. Actually, I'm using two seperate RAID controllers (two drives per card) and treating them as sperate logical drives (e.g., D: & E:). Controller "A" mirrors LUN 0 and 1, and controller "B" mirrors LUN 2 and 3. I am not mirroring the raid controllers' data between each other.

Denis Danatzko · July 28th, 2006, 08:25 PM

Quote:

Originally Posted by Dan Farzad

till now I offered my client to provide me with an external HD where I create the project from beg to ending on it. I dont feel it efficient. and extra $250 for the client. past that issue I dont feel safe about any external HDs, every tme I turn one on, I'm affraid of it crashing etc since 2 have gone bad on me.
What do u guys suggest and what do u do?

My first suggestion is to read these articles, then strongly consider tape. The articles aren't very long, and the points have been somewhat proven through personal experience: by sheer coincidence, I recently dug up the original VHS master tape of a wedding I did in 1989. I copied it to miniDV tape, and brought it in to my new PC so I could learn Premiere Pro 2. As part of that purchase, the vendor spent a couple hours with me to get me started. He commented repeatedly about the high quality of the image for a VHS tape that was created so long ago. And it was shot using only a Minolta Master consumer camera. (which I still have, BTW).

http://www.infoworld.com/article/06/...ifespan_1.html

http://photocollector.net/TipCD.htm

Brendon Whateley · July 28th, 2006, 10:05 PM

Quote:

Originally Posted by Matt DeJonge

I think I might have misled you all with my wording. ...

So, to check if I understand: You have two RAID 0 arrays? In otherwords doubling the drive access speed at the expense of making total data loss twice as lightly. As one who has suffered many HD failures, I'd not do that myself!

Matt DeJonge · July 31st, 2006, 08:26 AM

Quote:

Originally Posted by Brendon Whateley

So, to check if I understand: You have two RAID 0 arrays? In otherwords doubling the drive access speed at the expense of making total data loss twice as lightly. As one who has suffered many HD failures, I'd not do that myself!

You know, I'm a frigging idiot. I'm set up for RAID 1, not 0. Duh. Absolutly duh. I don't know what I was thinking when I posted that. I totally see the confusion now. I'm mirroring LUNs 1 & 0 and the mirroring LUNs 2 & 3. I'm not spanning the drives. Sorry about that.

Steven Davis · July 31st, 2006, 09:16 AM

Just buy hard drives and swap out. But beware of the MYBOOKS from Western Digital, I recently had an annoying experience with one, went back and picked up a lacie external instead. But I'm at that point where I'm going to just start buying internals and swapping. It's cheaper.

July 24th, 2006, 05:14 PM	#1
Dan Farzad Regular Crew Join Date: May 2005 Posts: 107	storage solution I have been running my small production company for about a year and half now. Within this time I deciced to focus on corporate and non profit organizations. The major problem that I have with these projects is space. Unlike most wedding businesses where when your done editing you delete the files, for most part I have to hold on to all the datas of a project in order to use it for years to come in making their future videos and making the job easier but space is a problem. till now I offered my client to provide me with an external HD where I create the project from beg to ending on it. I dont feel it efficient. and extra $250 for the client. past that issue I dont feel safe about any external HDs, every tme I turn one on, I'm affraid of it crashing etc since 2 have gone bad on me. What do u guys suggest and what do u do?

July 25th, 2006, 02:54 PM	#5
Matt DeJonge Regular Crew Join Date: Apr 2006 Location: Los Angeles, CA Posts: 37	Dan, I agree with Glenn on compressing your files. The only advantage or using WinRAR on video files (using the "Store" option, which takes VERY little time) is that it will allow you to chop up the video file to 4.6 GB files for spanning to multiple DVDs. I do this once in a while when needed, and it works great. The only downside is the slow burntime, but newer DVD-RW drives are pretty quick now. I find that storing the data on external HDs is pretty inexpensive and easy to implement, and you only need the drives on when you want to write or retrieve data, otherwise leave them turned off. That'll save on your "mean time before failure". I will soon be getting a PC set up with SATA drives and will be going RAID-0. I've decided to mount (2) 500GB drives into 1 RAID-0 config (attached to a controller), then mount another 2 drives in a RAID-0 config (to a second controller), etc. This is cheap from the controller stand-point (you can get controllers PCI for about $65.00) and you'll have some redundancy because every second drive is striped. You can also build this machine up as needed. The only downsides are the size of the machine (it's a whole computer after all), and the possible noise. However, that's not too big of a deal, as I'd only boot the machine when needed (hibernat mode helps this). __________________ AMD Athlon 64 X2 Dual Core 4200+ / 2GB RAM / nVidia 7900GT Dual Head, 3x500GB HD Adobe CS3, LightWave 3D 9.3

July 28th, 2006, 12:42 PM	#7
Matt DeJonge Regular Crew Join Date: Apr 2006 Location: Los Angeles, CA Posts: 37	I think I might have misled you all with my wording. Actually, I'm using two seperate RAID controllers (two drives per card) and treating them as sperate logical drives (e.g., D: & E:). Controller "A" mirrors LUN 0 and 1, and controller "B" mirrors LUN 2 and 3. I am not mirroring the raid controllers' data between each other. __________________ AMD Athlon 64 X2 Dual Core 4200+ / 2GB RAM / nVidia 7900GT Dual Head, 3x500GB HD Adobe CS3, LightWave 3D 9.3

July 31st, 2006, 09:16 AM	#11
Steven Davis Inner Circle Join Date: Jun 2004 Location: Richmond, VA Posts: 3,065	Just buy hard drives and swap out. But beware of the MYBOOKS from Western Digital, I recently had an annoying experience with one, went back and picked up a lacie external instead. But I'm at that point where I'm going to just start buying internals and swapping. It's cheaper. __________________ What happens if I push the 'Red' button?

July 24th, 2006, 05:38 PM	#2
Glenn Chan Inner Circle Join Date: Jun 2003 Location: Toronto, Canada Posts: 4,750	Some options would be: A- Burn onto optical discs. The problem right now is that this takes forever. B- Put material onto two different hard drives. Wait for rebate deals on hard drives. Enclosures can be had for a fairly low price now... even eSATA I believe. C- Setup a storage server on a PentiumIII or better machine (yes, a pentium III will be fine). With a compatible RAID controller card, you could setup a RAID 5 which will protect somewhat against drive failure (although sometimes two drives will go down at once). RAID doesn't protect against user error. D- There are computer tape backup solutions. High initial cost, but they let you have multiple backups and things like that.

July 24th, 2006, 08:02 PM	#3
Dan Farzad Regular Crew Join Date: May 2005 Posts: 107	can i compress the data after it it done so it takes less space on a HD and reexpand? how>?

July 24th, 2006, 09:27 PM	#4
Glenn Chan Inner Circle Join Date: Jun 2003 Location: Toronto, Canada Posts: 4,750	You could enable NTFS compression on the drive, or use an archival program along the lines of winRAR. However, you will get extremely little compression with video files. DV for example already employs DCT compression, so you will not gain much compression over that. Animation and Huffyuv codecs already utilize lossless compression, so file sizes after re-compression may be slightly (and insignificantly) larger. It's generally not worth the time.