To RAID or not to RAID? at DVinfo.net

Go Back   DV Info Net > Cross-Platform Post Production Solutions > High Definition Video Editing Solutions

High Definition Video Editing Solutions
For all HD formats including HDV, HDCAM, DVCPRO HD and others.


Reply
 
Thread Tools Search this Thread
Old June 4th, 2009, 03:50 PM   #1
Wrangler
 
Join Date: Sep 2002
Location: Los Angeles (recently from San Francisco)
Posts: 954
To RAID or not to RAID?

I finally put together a nice quad-core 3.0 GHz 4 Gig RAM (soon to be 8) editing machine so I can handle HDV more easily, and picked up an Intensity Pro on sale, just for fun.

I was trying to capture component to mjpeg and found that, at either 1080i or 720p, I could only capture for between 30-45 minutes at a time. Both the capture software included with the Intensity Pro and Premiere Pro CS4 would simply stop capturing, and always at a different place.

I tested my drive speed and, though it's fast enough according to the Intensity Pro specs, it just barely bests the required speed. I'm thinking of going to some sort of RAID solution to improve capture performance. Here are my choices:

1. External eSATA-based RAID device, probably only 2-drives.
2. RAID card using 2 external eSATA drives (I already have the card, so this is the most appealing, economically speaking).
3. Internal RAID using the motherboard's built-in RAID capabilities.

Is there any benefit from the standpoint of getting the Intensity Pro to capture component for more than 45 minutes or so?

Thanks!
Paul Tauger is offline   Reply With Quote
Old June 4th, 2009, 06:13 PM   #2
Major Player
 
Join Date: Aug 2007
Location: Goleta, CA
Posts: 299
Honestly for I would do the internal RAID 0. If you have all of your stuff backed up on tapes and your project is saved to a different drive then there isn't much need to get an external RAID unless you want terabytes of storage, 8+ terabytes depending on your motherboard, or speed and reliability from a RAID 5 which is too much for HDV.
James Brill is offline   Reply With Quote
Old June 5th, 2009, 04:00 AM   #3
Trustee
 
Join Date: Aug 2006
Location: Rotterdam, Netherlands
Posts: 1,832
The two pictures below show you a single drive and a 12 disk raid30.

Noteworthy is the performance degradation with increasing fill rates on the single disk, but none of that on the raid30. The same performance degradation happens with a 2 disk raid0. Notice that a 12 disk raid30 is for most of us overkill, but a 4 disk raid3, possibly with a hot spare added, is definitely a serious performance improvement. Not many raid controllers support raid3 however, so the alternative may be to use raid5 or even raid6, although for video editing raid3 is probably the best.

Maybe you need some drive cages to fit those disks in your system, but that is IMO better than externally. No cables, no extra power supply, no external housing.

Maybe also have a look at these guides, there is valuable information in there that may be of interest to you.
http://forums.adobe.com/thread/436215
Attached Thumbnails
To RAID or not to RAID?-hdtune_benchmark_areca___arc-1680-vol-000.png   To RAID or not to RAID?-hdtune_benchmark_samsung_hd103uj.png  

Harm Millaard is offline   Reply With Quote
Old June 8th, 2009, 09:32 PM   #4
Major Player
 
Join Date: May 2006
Location: Toronto ON Canada
Posts: 731
Quote:
Originally Posted by Paul Tauger View Post
I tested my drive speed and, though it's fast enough according to the Intensity Pro specs, it just barely bests the required speed. I'm thinking of going to some sort of RAID solution to improve capture performance.
What is the required speed for your needs?

You are looking for a RAID to use strictly as scratch space? Are you looking strictly for performance or security (or balance between the two)?

You're sticking with HDV or planning on moving up to something with a richer data rate?
__________________
Mike Barber
"I'm laughing to stop myself from screaming."
Mike Barber is offline   Reply With Quote
Old June 9th, 2009, 07:36 AM   #5
Inner Circle
 
Join Date: Mar 2003
Location: Ottawa, Ontario, Canada
Posts: 3,942
Firstly, if you have already recorded HDV there is no point in using the Intensity Pro. The Intensity Pro is only of use to record directly live from the camera from the HDMI connector. Once recorded to tape the signal has been compressed and the cleanest way of getting this video to the PC is to capture with iLINK as this is just a file copy.Using Intensity will invoke two more conversions. Firstly in the camera to HDMI and then in the Intensity/PC to whatever format is selected. HDV requires about 3.5MBps. Most modern drives should easily do 50MBps even 100MBps, even external USB 2.0 can manage 20MBps. Even 5400rpm laptop drives will be adequate to capture HDV. IF you want to ease the editing load on your CPU then you could convert the captured file to Cineform or if your CPU ( not your hard drive) is fast enough convert to Cineform on capture. If you do use Cineform then the file is bigger and you will now need a fast hard drive to feed the easier to edit Cineform file to your editing software. But even for this a single stream is easily handled by a modern hard drive. Secondly using a single 2 disc RAID 0, under some circumstances can be slower than two individual drives if it must both read and write at the same time, as it would if it were the only storage. Bottom line is if your editing just one stream of HDV you do not need a RAID or Intensity Pro.
For your information I run Q9450( 2.66Ghz), 8G RAM, Vista 64, 250G OS, 250G TEMP, 2x 750G for video editing and several external 1T drives as needed on eSATA. I have no RAID and regularly ( weekly at times) edit using EDIUS multicam with two tracks HDV native and 2 tracks of Canopus HQ converted from AVCHD. I have no speed problems and a check of performance always shows the CPU as the bottle neck. I also use Vegas 8 and this too has no problems using HDV native even AVCHD native though does slow the frame to the preview monitor at times. Using a RAID 0 would be convenient to not have to select drive for the files but I have found placing the files on the drives to be no real problem( read from one drive and write to another).
I too got an Intensity Pro as a means of editing my AVCHD video when this was not well supported. The Intensity has been on the shelf for over a year now. I guess I should sell it!!!

Ron Evans

Last edited by Ron Evans; June 9th, 2009 at 08:34 PM. Reason: Harm pointed out I had a typo in typing GBps not MBps.
Ron Evans is offline   Reply With Quote
Old June 9th, 2009, 09:31 AM   #6
Trustee
 
Join Date: Aug 2006
Location: Rotterdam, Netherlands
Posts: 1,832
Ron,

You have made some typo's I think. In the numbers you mentioned G should be M, as in MB/s (mega bytes per second).

While I agree with your argument about speed, there are some advantages to raids. Not only do they increase the speed and responsiveness of the system, but the R of redundancy (notice that I call a Raid0 an AID0 because of the lacking redundancy) is a significant advantage. Sure disks fail at some moment, but with single disks you lose all data, with a Raid when you lose a disk, replace the disk and it will be rebuilt in the background, without losing any data. With raid levels 6, 10, 30 or 50 you can even lose two disks at the same moment without losing data (to be completely correct, in the case of 10, 30 or 50, one from each array), and how often do you lose two disks at the same moment? The safety can be further increased by using hot spares.

With the low prices of hard disks, there is no reason not to invest in raids. More speed, more storage, more security.
Harm Millaard is offline   Reply With Quote
Old June 9th, 2009, 10:01 AM   #7
Inner Circle
 
Join Date: Mar 2003
Location: Ottawa, Ontario, Canada
Posts: 3,942
OOps. Yes Harm you are correct mixed up hard drive size with speed. My point about RAID is that you need more than two discs to make it worthwhile. Also for DV or HDV it really isn't worth it for speed as a single disc is fast enough by quite a margin. RAID makes life easier as there is no need to worry about where to put the files ( they are all on a very big disc) but for RAID 0 with just 2 drives ( which a lot of people use) there is likely no performance advantage ( could even be slower) and an increased risk to data ( all will go if either of the discs fail) rather than just one discs worth if left as two discs. A move to larger disc arrays may be worth while if there is a need for redundant storage or speed for uncompressed video etc. I contend that for the amateur or semi amateur as lots are on this forum, with only two or three discs in their systems RAID adds complexity and vulnerability for very little if any speed advantage. It makes managing files easier on one big disc for a downside of vulnerability. Your comments are valid for someone prepared to install upwards of 6 discs plus OS discs in their system. I have gone the other route by keeping it simple and backing up important files to tape. I have no speed problems using HDV and AVCHD. The CPU is still the bottleneck on performance. Biggest issues are encoding/transcoding, CPU and memory are the bottleneck there for me.

Ron Evans
Ron Evans is offline   Reply With Quote
Old September 26th, 2009, 06:44 PM   #8
Major Player
 
Join Date: Nov 2003
Location: Bay Area, CA
Posts: 477
Is there a link any of you can post as to what all the different RAID levels mean and do?

Will much appreciate it!!
Andrew Clark is offline   Reply With Quote
Old September 26th, 2009, 07:36 PM   #9
Trustee
 
Join Date: Aug 2006
Location: Rotterdam, Netherlands
Posts: 1,832
AC&NC - RAID.edu - RAID Tutorial & Benchmarking Tools
Attached Thumbnails
To RAID or not to RAID?-raid-overview.jpg  
Harm Millaard is offline   Reply With Quote
Old September 26th, 2009, 09:13 PM   #10
Trustee
 
Join Date: Jan 2008
Location: North Conway, NH
Posts: 1,745
Unless you just want more speed out of your computer, there is no reason to go RAID for HDV and DV. I can do a multi-cam edit in Premier Pro with three streams of HDV from a single drive and a fourth QT/Anim stream coming from another drive with good performance. It will drop some frames from the display but that doesn't effect my editing capabilities. Scrubbing the footage is pretty effortless.

I'd be an idiot to go contrary to Harm's comments because he's right in what he says. My thought is, do you really want to add to the complexity and maintenance requirements of your system for a little more speed? As with everything, there's a price/prize equilibrium that will work for you. Figure out your current and short-term future needs, and factor in how much money you want to part with and how much system admin time you want to spend and you should have your answer.

My guide is, if I don't need it, I don't get it. If I need it in the future, I wait. It'll likely be cheaper then anyway.
Tripp Woelfel is offline   Reply With Quote
Old September 27th, 2009, 06:16 AM   #11
Major Player
 
Join Date: May 2009
Location: Arta, Greece
Posts: 342
I use Vegas to capture HDV (I consider it much more stable for the particular job than Premiere) and most times I capture on external USB drive! As mentioned, HDV is a pretty low-demanding format, since its data rate is even slower than USB's. Needless to say, a good SATA drive is more than enough to work with HDV with excellent speeds (never had a single problem, unless the color correction begins, but thats a CPU/memory issue).
Dimitris Mantalias is offline   Reply With Quote
Old September 27th, 2009, 06:38 AM   #12
Trustee
 
Join Date: Aug 2006
Location: Rotterdam, Netherlands
Posts: 1,832
Tripp,

As I said in post #6, there are three advantages to a raid configuration, the triple S law:

Speed, safety and space. There is no drawback in terms of maintenance or complexity.

Safety is a major advantage that outweighs IMO the trouble of setting up a raid. If you use single disks any of them can fail and occasionally do. That means you just lost all your data on that disk. If you have a raid, a disk can also fail, but after replacing the disk with a new one, you have not lost your data, it will be reconstructed in the background.
Harm Millaard is offline   Reply With Quote
Old September 27th, 2009, 08:45 AM   #13
Major Player
 
Join Date: May 2007
Location: KLD, South Africa
Posts: 983
I have 5 drives in RAID 5 and it works great! I've lost three drives this year on separate occasions due to the firmware issue Seagate had, luckily I lost no data, swapping out the faulty drive was all I had to do. Data is money, get a redundant raid system! It's faster and makes sense.
Nicholas de Kock is offline   Reply With Quote
Old September 28th, 2009, 07:07 AM   #14
Trustee
 
Join Date: Jan 2008
Location: North Conway, NH
Posts: 1,745
Quote:
Originally Posted by Harm Millaard View Post
Tripp,

As I said in post #6, there are three advantages to a raid configuration, the triple S law:

Speed, safety and space. There is no drawback in terms of maintenance or complexity.
Yeah... I think I went a little deep on the maintenance overhead point. To be honest, it's been over a decade since I used a RAID and longer than that since I was configuring software and hardware RAIDs. Undoubtedly, things are much lower maintenance now.

To go to the OP's issue of the capture stopping prematurely, a RAID may or may not subvert the problem but it will not resolve his underlying system problem because any contemporary machine should be able to capture DV or HDV without a RAID. Thinking a RAID would resolve the problem is like thinking a poultice will fix a broken leg.
Tripp Woelfel is offline   Reply With Quote
Old October 1st, 2009, 04:42 AM   #15
Trustee
 
Join Date: Mar 2007
Location: Sherman Oaks, CA
Posts: 1,259
I think the speed argument actually can go out the window when only two physical drives are used. If your media is stored on drive #1 and you render to drive #2, I believe this will be a faster arrangement than if you were to RAID dives #1 and #2 together into a RAID0 configuration and store media on and render to the same RAID array.

Of course this doesn't hold if more drives are added to the equation. But as rule you want to keep your media on a different physical drive than you render to. So depending on how many drives you want to throw at the problem, RAID may or may not be the correct choice. If the answer is you're only going to use two physical drives for video editing, then RAID is not a good choice, FWIU.
__________________
Avid Media Composer 3.1.3. Boris Red and Continuum Complete. Vegas 8.0c. TMPGEnc Xpress Pro 4.0

Last edited by Peter Moretti; October 1st, 2009 at 06:46 AM.
Peter Moretti is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Cross-Platform Post Production Solutions > High Definition Video Editing Solutions

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 09:47 PM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network