RAID 0 or RAID 5 ? at DVinfo.net
DV Info Net

Go Back   DV Info Net > Windows / PC Post Production Solutions > Non-Linear Editing on the PC

Non-Linear Editing on the PC
Discussing the editing of all formats with Matrox, Pinnacle and more.

Reply
 
Thread Tools Search this Thread
Old September 9th, 2012, 09:27 AM   #1
Major Player
 
Join Date: May 2007
Location: Bethesda, MD
Posts: 404
RAID 0 or RAID 5 ?

Hello,

I have a couple of questions about choosing my next storage solution.
I know exactly how RAID 0 and RAID 5 work. So I guess, I'm asking for feedback and opinions from users.

Would you use internal storage or external storage?
In each case would you go with RAID 0 or RAID5

I've noticed that some manufacturer only offer RAID 0. For example, the G-Technology 4TB G-RAID External Hard Drive Array with Thunderbolt, which goes for around $ 600 offers only the RAID0 configuration. From what I read online it's impossible to also have access to individual drives inside the bay. So in RAID0 if one drive fails, I lose everything. Wouldn't you say that the fact that this bay is only RAID0 is a mistake? In other words, what are the chance of a failure?


Another manufacturer seems to offer a better solution? Promise Technology 4TB Pegasus R4 RAID Storage with Thunderbolt (4x 1TB) goes for around $ 1000 and allow RAID5 and access to individual drives.I understand that if ONLY one drive fails in a RAID5 I'll be able to recover my data.
Isn't this way better?

So what would you do? Go with RAID0 or RAID 5, both in thunderbolt? Or should I bother about the extra speed of thunderbolt at all?

Should I go with internal storage instead? And if I choose this way, RAID0 or RAID5
I'm PC based using Vegas, should I save money and go with external RAID using USB3 and not worry about some small added speed offered by thunderbolt?

I'm aware it's a lot of questions.
As usual, thank you so much for all your coming answers
Larry
Larry Secrest is offline   Reply With Quote
Old September 9th, 2012, 09:35 AM   #2
Inner Circle
 
Join Date: Aug 2006
Location: Efland NC, USA
Posts: 2,322
Re: RAID 0 or RAID 5 ?

Here are a few questions to answer:

1. What is your application's bandwidth requirement?

2. How much fault tolerance is required?

If you can answer those two questions the choice for RAID and which flavor will become very clear.
__________________
http://www.LandYachtMedia.com
Chris Medico is offline   Reply With Quote
Old September 9th, 2012, 09:54 AM   #3
Major Player
 
Join Date: May 2007
Location: Bethesda, MD
Posts: 404
Re: RAID 0 or RAID 5 ?

Chris,


1. What is your application's bandwidth requirement?

I have no idea! I'm going to edit footage from a HDV cam. The footage is converted to Cineform at capture and edited in Cineform. I'd love no frozen frame and I'd love everything to run smoothly.


2. How much fault tolerance is required?
Well, I'd like NOT to lose any data!
Larry Secrest is offline   Reply With Quote
Old September 9th, 2012, 10:18 AM   #4
Inner Circle
 
Join Date: Aug 2006
Location: Efland NC, USA
Posts: 2,322
Re: RAID 0 or RAID 5 ?

You need to answer those questions before you can choose the correct product.

To answer the bandwidth question you can determine how many simultaneous streams of video you need to run. Take the bitrate of the codec for each stream and multiply that by the number of simultaneous streams. With that info you can see if you need RAID at all.

If you can handle a single drive failure you can use RAID5. If you are extra paranoid and want to mitigate a 2 drive failure you can use RAID6.

Regardless of which flavor you choose get an external solution with a hardware RAID controller. That will give you the best performance and fault tolerance.

See if this is something that looks like it will handle your application:

http://www.bhphotovideo.com/c/product/740549-REG/G_Technology_8TB_G_SPEED_eS_PRO.html
__________________
http://www.LandYachtMedia.com
Chris Medico is offline   Reply With Quote
Old September 9th, 2012, 10:23 AM   #5
Trustee
 
Join Date: May 2007
Location: Minneapolis, MN
Posts: 1,313
Re: RAID 0 or RAID 5 ?

Larry, does your PC even have a Thunderbolt connection?

I would say eSATA or USB 3 on a hardware-based RAID5 enclosure (4 drives) is your best bet for speed and safety. You're correct in being wary of a RAID0 array for your main disk - if one of the drives fails, things could get bad pretty quick. RAID0 setups are great for render files and converted footage though, in other words, things you can re-create if things go bottom up.

Remember, if you can't afford to lose it, make sure it's backed up - multiple copies in different locations if you're able.

Cineform HDV material is up to 14MB/s, and my 4-bay eSATA RAID5 clocks in at around 200MB/s, so unless you're layering a dozen tracks of video on top of one another, you should be perfectly fine using a connection like eSATA or USB 3.

What's nice about a hardware-based setup if that the drive repairs itself if there is any inconsistency between the drives - a software system (like the G-Tech) uses your computer resources to do this.

EDIT: Chris hit it spot on - sorry if I stepped on your toes as I was writing at the same time! Also, here's what I've been using for the last couple of years: http://www.bhphotovideo.com/c/product/812241-REG/OWC_Other_World_Computing_8_TB_Mercury_Elite.html
Not as fancy as the G-SPEED, but I don't have a PCI slot so it works for me!
__________________
Nate Haustein PXW-FX9 / FCPX
www.flightcreativemedia.com
Nate Haustein is offline   Reply With Quote
Old September 9th, 2012, 10:42 AM   #6
Major Player
 
Join Date: May 2007
Location: Bethesda, MD
Posts: 404
Re: RAID 0 or RAID 5 ?

Chris,
http://www.bhphotovideo.com/c/product/740549-REG/G_Technology_8TB_G_SPEED_eS_PRO.html
This can certainly handle my applications. But my wallet can't handle it.
I'll give some considerations to RAID6

Nate,
I'm going to build a computer from scratch, I do so every 5 years, so it will have a thunderbolt connection. You might be right though, thunderbolt my be overkill for what I'm doing, and I'm not a pro. So I think I might go with USB3 if there is a big savings here.

Also, I was thinking that in the perfect machine I'd have
SSD as boot drive and OS hosting drive
Two internal drives configured in RAID0 as scratch disks
and 4 drives configured in RAID5, either internal or external, probably USB3 connection, I guess.


Nate, you said: " What's nice about a hardware-based setup if that the drive repairs itself if there is any inconsistency between the drives - a software system (like the G-Tech) uses your computer resources to do this."
Are you saying that what you use, OWC / Other World Computing 8TB (4x2TB) Mercury Elite Pro Qx2 Quad Interface 4-Bay RAID Solution B&H Kit, is not software based? What about the Promise Technology 4TB Pegasus R4 RAID Storage that I mentioned in my first post, same as the G-Tech?
L.
Larry Secrest is offline   Reply With Quote
Old September 9th, 2012, 10:54 AM   #7
Trustee
 
Join Date: May 2007
Location: Minneapolis, MN
Posts: 1,313
Re: RAID 0 or RAID 5 ?

+1 for the SSD boot drive, it makes SUCH a difference!

Your setup sounds pretty sweet, I don't think you'll have any issues at all editing your footage. If you have the case space, perhaps you could even set up 4 internal drives as a RAID0 pair and a RAID1 pair to start. Put your raw footage and masters on the redundant RAID1 volume and use the RAID0 volume for your scratch/render files.

If you got 4 matching copies of the same bare drive right away, you could always pull those 4 and put them in an external enclosure at a later time, replacing your internal drives with something else if necessary.

Not saying a RAID5 enclosure isn't a great investment, but if cost is a factor and you're not entirely sure of you needs right away, why not save some bucks! I honestly think that you could edit your Cineform HDV material off internal RAID0 and RAID1 SATA connected volumes without a problem. Your thoughts Chris?
__________________
Nate Haustein PXW-FX9 / FCPX
www.flightcreativemedia.com
Nate Haustein is offline   Reply With Quote
Old September 9th, 2012, 11:08 AM   #8
Major Player
 
Join Date: May 2007
Location: Bethesda, MD
Posts: 404
Re: RAID 0 or RAID 5 ?

Nate,
Yes of course SSD for boot and OS drive.

What you've just said makes so much sense, I'll go with the internal drive config you suggested and if I need to I'll add an external bay later. Of course. Jesus, sometimes I think I have tunnel vision!
Larry Secrest is offline   Reply With Quote
Old September 9th, 2012, 11:15 AM   #9
Inner Circle
 
Join Date: Aug 2006
Location: Efland NC, USA
Posts: 2,322
Re: RAID 0 or RAID 5 ?

My advice would be to skip the RAID in the beginning if you don't have the budget to get a proper hardware based RAID setup. You may find you don't need it at all.

I don't trust the pseudo-raid controllers motherboards offer. I have not had good success with them. Avoid software RAID like the plague for video applications. They steal much to much of your computers resources and your overall system performance will suffer.

If you do need the speed I strongly recommend going to a video proven RAID system. A bad RAID is worse than no RAID. I've proven that from personal experience.

I suspect you don't need RAID at all for your compressed video workflow. I'm not currently using it on my system. I am using video bitrates up to 220Mb/sec (DNxHD) and have no issues with speed using internal disks in the edit system. If I am going to use a lot of high bandwidth sources I spread them out across 3 media drives in the computer. Its not RAID but having multiple drives to draw from simultaneously works well. I backup all my media to a separate server so if I loose a drive in the edit machine recovery is not painful.
__________________
http://www.LandYachtMedia.com
Chris Medico is offline   Reply With Quote
Old September 9th, 2012, 11:25 AM   #10
Major Player
 
Join Date: May 2007
Location: Bethesda, MD
Posts: 404
Re: RAID 0 or RAID 5 ?

So in other words, to edit HDV captured through HDLINK and converted into Neoscene or Cineform Studio Premium, I don't need RAID?

Just a nice processors, very nice mobo, very good ram and very decent drives should do the trick ?
Larry Secrest is offline   Reply With Quote
Old September 9th, 2012, 11:31 AM   #11
Trustee
 
Join Date: May 2007
Location: Minneapolis, MN
Posts: 1,313
Re: RAID 0 or RAID 5 ?

Thanks for your thoughts, Chris - I trust your judgements. My earlier suggestions were simply a "way" of doing things, but perhaps there are some flaws as you mentioned. I also think you're spot-on with using single, high-quality drives for the time being. The band-width requirements Larry is working with shouldn't exceed what a single drive can offer.

The only thing I wonder about (and it's something I would want personally) is a good way to automatically back up the data on the "master footage" drive to a second copy. It doesn't need to be RAID1, but what software is there to regularly backup the data on one internal drive to another without having to think about it too much?

I don't think it would hurt to have a third internal drive for a render disk as well, storage is cheap and it's nice not to worry about running out.
__________________
Nate Haustein PXW-FX9 / FCPX
www.flightcreativemedia.com
Nate Haustein is offline   Reply With Quote
Old September 9th, 2012, 11:57 AM   #12
Major Player
 
Join Date: May 2007
Location: Bethesda, MD
Posts: 404
Re: RAID 0 or RAID 5 ?

ok,

Please do NOT take this the wrong way. I am obviously NOT a specialist in computer, so I'm just checking several sources.

This is what the Videoguys say on this page on their point 5:
Videoguys Blog - Videoguys NLE Video Storage FAQ (Aug 2012 update)

QUOTE
If you are putting together a new machine for video editing we recommend selecting a motherboard with a built-in SATA RAID controller. Not only will this work great, but we've found they often include very easy to use RAID configuration software. QUOTE

So you disagree with them? From your experience I should NOT go with the SATA RAID controller offered by the mother board?
Larry Secrest is offline   Reply With Quote
Old September 9th, 2012, 12:18 PM   #13
Inner Circle
 
Join Date: Aug 2006
Location: Efland NC, USA
Posts: 2,322
Re: RAID 0 or RAID 5 ?

Nate, I'm far from a MacOS expert. I've helped a friend set up a TimeMachine routine on his computer which backs up his media nightly onto a NAS we setup on his network.

You can get more flexibility in scheduling the events with an app like this - Change Mac Time Machine Default Backup Time Interval - TimeMachineScheduler

I'm on a PC and have set up a scheduled event using the Windows SyncToy utility. It backs up all of the changed media files across all the drives in the edit machine onto my media server. It does a straight copy so restoring everything is really easy. Just copy the folders off the server to the replaced drive and keep going.

My comments regarding the onboard RAID options stem from multiple attempts to use it and suffering failures. I decided it wasn't worth the trouble and purchased an external setup. That system still works flawlessly after many years. I am not currently using it since I retired the older edit system and I am not doing any uncompressed video editing projects at the moment. If I did start one I wouldn't hesitate to drag it out and connect it to the new system.

+1 on the recommendation to have a dedicated render drive. In my machine I have 3 media drives, 1 OS drive, and 1 render drive. If I was going to use a motherboard based RAID for anything it would be on the render drive. I did some testing a few years ago and posted it on the Avid forum. It showed editing performance improved more by using a RAID0 setup on the render drive than any other combination I tried. Using RAID on media drives made no measurable performance improvement with codec bitrates under 440Mb/sec. Using RAID0 on the render drive reduced render times by more than 15% over using a single drive for rendering. The bottleneck moved from the drive bandwidth to the processor. Using a single render drive the processor would never exceed 80% utilization. With a RAID0 render drive the processor would load to its max. That was tested on a Intel Q6600 machine. Not the fastest thing compared to what you can get today.

Back to the subject at hand. Larry, if you are shooting HDV what bitrate are you transcoding that into? You shouldn't need to go more than about 150Mb/sec on a good intraframe codec such as Cineform. If you are using a bitrate in that range then RAID for your media drives would certainly not be needed. If you are transcoding it to RAW or a very high bitrate (over 450Mb/sec) then RAID would become necessary. That would be severe overkill based upon your acquisition format.
__________________
http://www.LandYachtMedia.com
Chris Medico is offline   Reply With Quote
Old September 9th, 2012, 12:24 PM   #14
Inner Circle
 
Join Date: Aug 2006
Location: Efland NC, USA
Posts: 2,322
Re: RAID 0 or RAID 5 ?

Quote:
Originally Posted by Larry Secrest View Post
ok,

Please do NOT take this the wrong way. I am obviously NOT a specialist in computer, so I'm just checking several sources.

This is what the Videoguys say on this page on their point 5:
Videoguys Blog - Videoguys NLE Video Storage FAQ (Aug 2012 update)

QUOTE
If you are putting together a new machine for video editing we recommend selecting a motherboard with a built-in SATA RAID controller. Not only will this work great, but we've found they often include very easy to use RAID configuration software. QUOTE

So you disagree with them? From your experience I should NOT go with the SATA RAID controller offered by the mother board?
I agree that it can be easy to configure but I know that it isn't as robust as a real hardware RAID setup. To put it in perspective a decent quality RAID controller will cost more than your motherboard and have similar processing horsepower. Its not practical to expect the RAID controller on your motherboard will be in the same league of performance as a dedicated RAID controller. That was the reason why I asked what your tolerance to loss is. Since you expressed a strong aversion to loss I wanted you to know where the weaknesses were and how to best achieve your goal.
__________________
http://www.LandYachtMedia.com
Chris Medico is offline   Reply With Quote
Old September 9th, 2012, 12:26 PM   #15
Inner Circle
 
Join Date: Feb 2004
Location: switzerland
Posts: 2,133
Re: RAID 0 or RAID 5 ?

it is silly to compare Raid 0 to raid 5, since they are not intended for same use.
Raid 0 is just a way to merge several disk space into one, like all raid are supposed to be.
but Raid 0 stops here.
You get better performance as well and it could be a reason to go that way.

Raid 5 is made to add some redundancy so you can expect your data to be safer.
the drawback is you loose a disk capacity (so if you raid5 4 disk of 1 terabyte, you got only 3 terabyte diskspace.). So the cost is higher.

in between you got raid 1 (mirror) that offer total redundancy, but does not offer increased disk space (you totally loose one disk ) and get only partial speed incrase (when reading).

Raid1 looks like a bad idea, but it is very simple, efficient (see it like a permanent instantaneous backup),
and by these times where disks come in huge size (2 tera is a lot) and great performance for cheap, it is a very interesting solution. And you can replace a disk at any time with another one , even if not of the same type.

if you just need a huge disk space, consider JBOD (just a bunch of disk) that merge several disks into one. No advantage here

For mission critical purpose, Raid+ is great. You take any Raid with feature you like (like Raid 0 for speed)
and add another Raid (like raid 1 for redundancy) . It is called Raid 0+1 or raid 1+0 or raid 10.
if you run a nuclear powerplant, you would probably use something like raid 51 or 50.

Raid 5 is very finnicky about controller, disks.
Giroud Francois is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

B&H Photo Video
(866) 521-7381
New York, NY USA

Scan Computers Int. Ltd.
+44 0871-472-4747
Bolton, Lancashire UK


DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Windows / PC Post Production Solutions > Non-Linear Editing on the PC

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



All times are GMT -6. The time now is 11:26 PM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2024 The Digital Video Information Network