eSata for connecting external RAID drive sets at DVinfo.net

Go Back   DV Info Net > Windows / PC Post Production Solutions > Non-Linear Editing on the PC

Non-Linear Editing on the PC
Discussing the editing of all formats with Matrox, Pinnacle and more.


Reply
 
Thread Tools Search this Thread
Old July 27th, 2011, 12:25 AM   #1
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
eSata for connecting external RAID drive sets

I'm looking at purchasing an external RAID system and I'm mindful that the newfangled USB 3 is much faster than eSATA (which itself is still quite speedy).

The question is ... am I ever likely to exceed the bandwidth that you get with an eSATA connection to the external array? Am I getting worried over nothing? Should I go for a unit that supports USB 3 instead?

(I have a feeling that USB3 will tax the CPU a bit, but I need to research that one to know for sure.)

Thoughts?

Andrew
Andrew Smith is offline   Reply With Quote
Old July 30th, 2011, 08:32 AM   #2
Major Player
 
Join Date: Apr 2006
Location: Warsaw/Poland
Posts: 716
Re: eSata for connecting external RAID drive sets

The question is what kind of files, resolutions, and compression you're going to work with, which eSata (1.5/3/6 Mbit) version you're using, what kind of motherboard you have, and how many other drives are connected via eSata.
__________________
Creative Impatience - The Solace of Simple Solutions. A few useful plugins for Adobe users, and my remarks on the tools and the craft in general.
Bart Walczak is offline   Reply With Quote
Old July 31st, 2011, 01:46 AM   #3
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

Thanks Bob,

The computer is yet to be built, but will be in the very near future. I'm keen to do it right and this is part of the research.

I've purchased my upgrade to Adobe Production Premium Suite 5.5 and I'm about to purchase a Matrox MXO2 (with Max technology for super speedy h264 encoding). The MXO2 is connects via PCIe.

I've got a large collection of HDV recordings and will continue to record to this format with my Sony V1 camera. From time to time there will also be content in the AVCHD format from the Sony HXR NX5 camera.

With the Matrox MXO2 gear, there is the option to capture to 25, 50 and even 100Mbit/sec mpeg2 files for super quality work. I think this option is there with a view to having higher quality video to do good chromakey work with, etc.

Andrew
Andrew Smith is offline   Reply With Quote
Old July 31st, 2011, 02:07 AM   #4
Major Player
 
Join Date: Nov 2003
Location: Sweden
Posts: 221
Re: eSata for connecting external RAID drive sets

We use Highpoint RocketRaid eSATA cards with connections to two cabinets each holding 4x SATA drives running RAID5. No performance issues for editing several 4:2:2 iframe 150 mbit/s streams.
Problem is not in disc read/write, bottleneck is processing/CPU power.

HighPoint Global website
/m
__________________
Magnus Helander, Crossmediageek on G+
Magnus Helander is offline   Reply With Quote
Old July 31st, 2011, 04:21 AM   #5
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

Interesting. Amazing what we have access to these days. Not that I am expecting to be editing raw uncompressed streams.

Has anybody had any experience with the G-Tech units? See Videoguys.com - G-TECH from Hitachi

Andrew
Andrew Smith is offline   Reply With Quote
Old July 31st, 2011, 07:17 AM   #6
Major Player
 
Join Date: Apr 2006
Location: Warsaw/Poland
Posts: 716
Re: eSata for connecting external RAID drive sets

Quote:
Originally Posted by Andrew Smith View Post
I've got a large collection of HDV recordings and will continue to record to this format with my Sony V1 camera. From time to time there will also be content in the AVCHD format from the Sony HXR NX5 camera.

With the Matrox MXO2 gear, there is the option to capture to 25, 50 and even 100Mbit/sec mpeg2 files for super quality work. I think this option is there with a view to having higher quality video to do good chromakey work with, etc.
With HDV and AVCHD footage you'd best work natively, without any transcoding, and no capture will make the footage better than it already is compressed, so you can just get those files via firewire and/or USB in their original format. HDV and AVCHD have pretty low bitrates, so I highly doubt you'll ever get to saturate your eSata connection. Even Matrox'es codecs shouldn't be much of a problem if you choose to capture live, although personally having had my own share of problems with Matrox cards, I tend to avoid their solutions.
__________________
Creative Impatience - The Solace of Simple Solutions. A few useful plugins for Adobe users, and my remarks on the tools and the craft in general.
Bart Walczak is offline   Reply With Quote
Old July 31st, 2011, 08:18 AM   #7
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

I've used Matrox products for a very long time now. With all the internal resource intensity that editing video demands of a computer there are plenty of ways that an editing experience can go wrong.

What I've observed with Matrox is that they took it upon themselves to do extensive testing with various motherboards, sound cards etc to validate what will work successfully in a video editing system with one of their cards. Additionally, they only release their drivers for the latest edition of PremierePro etc when they are ready for production and not before. I've seen it consistently over many years, and it's the sort of thing that takes a privately owned (not shareholder profit orientated) company to do.

The last time I built an editing computer, I used components from their tested/validated list and spent well on the editing box as a whole. The result for me was a rock solid editing machine which has served me faithfully.

Thus here I am, researching so that I can spend well on my next edit computer.

Andrew
Andrew Smith is offline   Reply With Quote
Old July 31st, 2011, 08:20 AM   #8
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

Also, thanks for the feedback regarding the potential for saturating the eSATA channel.

Andrew
Andrew Smith is offline   Reply With Quote
Old July 31st, 2011, 11:49 AM   #9
Major Player
 
Join Date: Nov 2003
Location: Sweden
Posts: 221
Re: eSata for connecting external RAID drive sets

I forgot to mention, we use Stardom cabinets for the drives - Taiwanese all aluminium with zero issues.
STARDOM STORAGE SOLUTIONS - SOHOTANK FEATURE
/m
__________________
Magnus Helander, Crossmediageek on G+
Magnus Helander is offline   Reply With Quote
Old August 1st, 2011, 12:12 AM   #10
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

Forgive me if I'm a bit clueless ..... but I'm beginning to get the idea that some of the external RAID units are merely physical enclosures to hold and power the hard drives, and some additionally have the hardware included for handling the RAID functionality internally?

I've been looking through all the G-Tech section on the VideoGuys site and it's a little bit hard to figure out as to why some units are priced in the hundreds of dollars and yet others are priced in the early thousands of dollars.

Andrew
Andrew Smith is offline   Reply With Quote
Old August 1st, 2011, 12:32 PM   #11
Major Player
 
Join Date: Nov 2003
Location: Sweden
Posts: 221
Re: eSata for connecting external RAID drive sets

Here is my limited understanding of the situation to why the costs differ:

Challenge is how to connect the external drive bays/units in one or several cabinet/s to your motherboard so that you can get the data from the drives at the highest possible rate.

I know of three kinds of connectivity
- SAS - Serial Attached SCSI o
- SATA - Serial ATA
- iSCSI over Gigabit Ethernet

You need to get the data from drives to your CPU over the PCI-express PCIe bus - PCIe is operating at 250 megabyte/sec per channel. Videocards operate at full 16x speed, RAID cards only needs 1x, as you will have a hard time to get 250 megabyte / sec transfer rate from your drives.

So, some have all the RAID logic in the cabinet and present a single logical drive to the operating system across eSATA or a (possibly, i'm not sure...) SAS connection.
Others put a RAID card/logic in the PCIe bus and then connect "dumb" drives in cabinets.

So, it depends if you have SAS / SATA / iSCSI connectivity to your drives, and if you need a PCIe card, and ... well.. the overall performance.

Factors to consider
- Read & write rates for the physical drives
- Combined read & write of the logical RAID unit/drive
- Ability to expand.
- RAID 5 or other configuration...

And check for "burst" which is what the unit can deliver as a single peak/burst, and "sustained data rate" which is what it can deliver continually during playback

This is a nightmare, so ask some storage professionals, I work with people who is fascinated by this stuff - I just want it to work. 150 mbit / sec I-frame 4:2:2 is approx 19 megabyte / sec datarate.

/magnus
__________________
Magnus Helander, Crossmediageek on G+
Magnus Helander is offline   Reply With Quote
Old August 2nd, 2011, 10:34 AM   #12
Sponsor: Electronic Mailbox
 
Join Date: Mar 2002
Location: Glen Cove, NY
Posts: 751
Re: eSata for connecting external RAID drive sets

A very good thread. Thanx to Andrew for giving me the heads up. As was posted, the G-tech RAIDs come in several levels.

G-Raids & G-Speed Q attach vie eSata (or firewire800/400/USB). eSata is the fastest connection and the one we recommend. We plug ours directly into the integrated eSata of our DIY motherboards. G-RAID and G-Speed Q offer the same throughput and datarates. The only difference is that you can configure the 4 drives in the Q as a RAID 5 for data protection. This protection costs you 25% of you total storage. So an 8TB Q with 4/x2TB drvies gives you 6TB of storage. A Graid is a RAID0, which gives you the full amount of storage of the 2 drives, but an drive failure results in all data lost.

G-Speed eS uses a G-Tech PCIe x4 RAID controller which gives you higher throughput capable of multi-stream ProRes 422 playback, but also ads costs.

G-Speed Es Pro uses an expensive mini-SAS high performance PCIe x8 IOP RAID card. Even higher thoughputs and capable of multi-stream ProRes 422 HQ playback.

We have a video storage FAQ on our website with some additional info Videoguys Blog - Videoguys NLE Video Storage FAQ

Gary
__________________
Check out http://www.videoguys.com 800 323-2325 We are the video editing and live video production experts! DV InfoNet members save 5%! Use Coupon Code DVINFO5OFF
Gary Bettan is offline   Reply With Quote
Old August 2nd, 2011, 11:48 AM   #13
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

Thanks Gary.

To be honest I've had my heart on a RAID5 system whereby you get both the throughput speed as well as the protection that you mentioned.

On the other hand, I've had none of my regular consumer HDDs fail in the previous 6 years that they have been running in the computer. (I know, I know ... getting a bit old for hard drives.)

I was thinking: I could possibly just purchase a couple of 'enterprise' level drives and RAID 0 them together and it might be some sort of a solution? Just a thought.

(off to read the storage FAQ now)

Andrew
Andrew Smith is offline   Reply With Quote
Old August 2nd, 2011, 12:23 PM   #14
Sponsor: Electronic Mailbox
 
Join Date: Mar 2002
Location: Glen Cove, NY
Posts: 751
Re: eSata for connecting external RAID drive sets

Here is the thing about hard drives. They don't just all of a sudden fail out of nowhere. Either an event causes them to fail, such as a power surge or physical damage, or they die a slow death.

What I mean by this is that if you pay attention to your system performance, you will know something is going wrong with your storage. Look and listen for any of these:

- clicking or loud whirring
- drives spinning hard for no apparent reason
- delays in starting clips or opening files
- windows explorer acting kind of sluggish
- heat or fans having to work extra hard to keep cool

When you see the first signs of any of these symptoms, back up your work and start planning on replacing the drives. The sooner the better.

Gary
__________________
Check out http://www.videoguys.com 800 323-2325 We are the video editing and live video production experts! DV InfoNet members save 5%! Use Coupon Code DVINFO5OFF
Gary Bettan is offline   Reply With Quote
Old August 2nd, 2011, 12:39 PM   #15
Inner Circle
 
Join Date: Feb 2009
Location: Brisbane, Australia
Posts: 2,258
Re: eSata for connecting external RAID drive sets

Gary,

In the spirit of not having to pay 10% GST on items brought in to Australia from overseas when the value of the goods is less than $1000 AUD .... it is possible to purchase the 4-drive G-Speed eS unit with no drives in it? (Would also make it lighter for freight too, I imagine.)

Andrew
Andrew Smith is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Windows / PC Post Production Solutions > Non-Linear Editing on the PC

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 07:03 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network