RAID 1 but which Block Size should I do? at DVinfo.net

Go Back   DV Info Net > Apple / Mac Post Production Solutions > Final Cut Suite

Final Cut Suite
Discussing the editing of all formats with FCS, FCP, FCE


Reply
 
Thread Tools Search this Thread
Old August 21st, 2009, 01:18 AM   #1
Regular Crew
 
Join Date: Aug 2005
Location: Torrance, CA
Posts: 58
RAID 1 but which Block Size should I do?

Hello all,

I've got two Western Digital Caviar "Black" 1TB drives added to my Mac Pro, and I'd like to do a Mirrored RAID set (RAID 1). I've been doing some research online but I am still seeing many different views as to how to properly set this up.

My workflow for now will be capturing or editing clips as ProRes 422. Pretty much no online editing just offline editing these clips for things like SD video / DVD production as well as some 720p HD editing in the near future.

Questions:

1. With the two drives, then, should I set the "block size" to 256k as opposed to 128k or anything smaller than that? The drives mainly will hold video files, image assets like .tga files and of course the FCP, Motion or other pro app project files.

2. Should I format as Mac OS Extended and IGNORE journaling? I've read that journaling isn't necessary for video editing. Or is this incorrect?

Thanks in advance all, I love this forum.
__________________
James Wong / www.hindsightproject.com

Last edited by James Wong; August 21st, 2009 at 01:19 AM. Reason: Added Question
James Wong is offline   Reply With Quote
Old August 23rd, 2009, 10:31 AM   #2
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
You want 128K

Assuming you're creating a software-RAID in Disk Utility this is something I've been testing for years on various configs and have the answer you need:

Unlike a hardware RAID there is overhead created in the OS to merge and maintain multiple drives in the RAID setup, and there is a diminishing point of return. Anything beyond 128K becomes too heavy a workload for a software RAID and I/O speeds drop significantly.

What's more interesting is that because there are no communication parameters that you can modify in Disk Utility for RAID setup (other than RAID type and block size) you'll notice very little performance difference between 16K and 128K however because you're working with large video files it will be less work for the HDD's to stay at 128K.

Different brands of HDD's also behave differently with respect to throughput speeds and unfortunately these values aren't constant; as HDD models evolve their performance grades also change.

Currently the fastest responding HDD's for any RAID is either Seagate or Hitachi (depending on the model), as both companies produce TB drives that perform at or above the 100MBs mark in I/O speed. (As tested with the AJA and BlackMagic disk testers). WD makes stable drives and they consistently test slightly slower than their competitors.
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old August 23rd, 2009, 11:04 AM   #3
Major Player
 
Join Date: Mar 2006
Location: Chicago
Posts: 706
Since you're doing RAID 1 for safety, I suggest leaving journaling on. You wouldn't notice a difference. As far as block size for video files, I would start with 128 and go from there. I would like to see some specific test for FCP too.

On mac raid 0 and 1 have little impact on CPU overhead, as there is no parity calculation. If you don't have enough throughput, the boot drive can be placed in one optical slot and then do RAID 1 + 0 with four drives.

If you need to add a sata card, buy a good one. The disk controller in the Mac Pro is very good (6 sata, 1000mBps) . Adding a card is adding disk controlling hardware, and it doesn't make sense to put a cheap card in a good machine. In broad terms an add-on card with an Intel IOP running at 800mhz will do 500 mBps, and one running at 1.2 ghz will do 1000 mBps. For mac I like the highpoint cards.
Don Miller is offline   Reply With Quote
Old August 23rd, 2009, 08:13 PM   #4
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
Quote:
Originally Posted by Don Miller View Post
The disk controller in the Mac Pro is very good (6 sata, 1000mBps) .
I don't know what benchmarking utility you're using to get these kind of numbers but they're far from accurate. The fastest any RAID into a single Mac desktop has ever tested - even using 4Gb Fiber cards such as the ATTO 42ES is around 450 MB/s. That's using the AJA System Test and the BlackMagic benchmarking utility - both of which accurately replicate the I/O environment used in video editing.

There are some eSATA RAID's that start out with very high READ speeds (above 500 MB/s) but since there is no cache available on the controller cards (the only cache available lives on the HDD's themselves) that speed can't stay constant and typically falls to around 280 MB/s during the course of editing/renders.

Ultra-high I/O numbers such as those posted in glossy ad campaigns are *theoretical* only and not real-world numbers.
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old August 24th, 2009, 12:00 AM   #5
Regular Crew
 
Join Date: Aug 2005
Location: Torrance, CA
Posts: 58
Quote:
Originally Posted by Robert Lane View Post
Assuming you're creating a software-RAID in Disk Utility this is something I've been testing for years on various configs and have the answer you need:

Unlike a hardware RAID there is overhead created in the OS to merge and maintain multiple drives in the RAID setup, and there is a diminishing point of return. Anything beyond 128K becomes too heavy a workload for a software RAID and I/O speeds drop significantly
Thank you for the information, Robert. Very useful. I had the RAID setup at 256k block size, however I have only been running basic tests (and backing up my projects/video on another drive as well) - so I can reformat the two drives as such. For the record they are 1TB WD Caviar Blacks.... not Seagates (I've usually purchased Seagate but some recent drives have had higher rates/reviews on failure for various reasons).

RAID via Disk Utility is my only option at the moment until I can find additional editing/authoring projects than the one I have currently :) Thanks again for the useful info!
__________________
James Wong / www.hindsightproject.com
James Wong is offline   Reply With Quote
Old August 24th, 2009, 08:07 AM   #6
Major Player
 
Join Date: Mar 2006
Location: Chicago
Posts: 706
Quote:
Originally Posted by Robert Lane View Post
I don't know what benchmarking utility you're using to get these kind of numbers but they're far from accurate. The fastest any RAID into a single Mac desktop has ever tested - even using 4Gb Fiber cards such as the ATTO 42ES is around 450 MB/s. That's using the AJA System Test and the BlackMagic benchmarking utility - both of which accurately replicate the I/O environment used in video editing.

There are some eSATA RAID's that start out with very high READ speeds (above 500 MB/s) but since there is no cache available on the controller cards (the only cache available lives on the HDD's themselves) that speed can't stay constant and typically falls to around 280 MB/s during the course of editing/renders.

Ultra-high I/O numbers such as those posted in glossy ad campaigns are *theoretical* only and not real-world numbers.
I didn't say anything about a raid system. I was talking about one component. If each of the six sata channels could be saturated with a single device, a raid 0 system on a Mac Pro would be limited by the controller hardware to about 1000 MBPS. Currently the best SLC based SSD drives could do about 800 MBPS with six drives in raid 0.

Plenty of high end raid system run over 500 MBPS. SATA drives are hardly high-end Again, I was talking about a component.

Also, Fiber channel and a separate multi user file server is going to slow things down. Two highpoint 4322 PCI cards each connected to eight SAS drives will be considerably faster. There's a test with 16 Intel SML SSD drives with two adaptec cards that use the same iop as the 4322 that do 2.2GBPS.

I'll add that most of the better RAID system sold for video editing are simply the Highpoint 4322, two high quality SAS to SATA cables, and a nice box like the ProAvio. That's $600 + $120 + $500 + $800 to $1200 for drives. I guess the 3X-4X markup is for burn-in and support. If the support is very good, perhaps it's worth it for some people.
Don Miller is offline   Reply With Quote
Old August 24th, 2009, 12:24 PM   #7
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
Quote:
Originally Posted by Don Miller View Post
... Two highpoint 4322 PCI cards each connected to eight SAS drives will be considerably faster. There's a test with 16 Intel SML SSD drives with two adaptec cards that use the same iop as the 4322 that do 2.2GBPS.
The H.P. cards are amazing - and reliable - as we've tested a handful. Unfortunately not everyone can give up (2) PCIe slots to make this uber-fast I/O environment which is why we always test around a single HBA.

I will say, that for all the shortcomings of eSATA that the dual 4322 setup is the best bang for the buck with respect to IO speeds; it's too bad they haven't figured out a way to put cache on the controller cards so that they can hold onto those speeds instead of ramping down over time.
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old August 31st, 2009, 06:16 PM   #8
Regular Crew
 
Join Date: Aug 2005
Location: Torrance, CA
Posts: 58
A lot of the latter discussion is beyond my resources, but was interesting to read nonetheless :-) Thanks for the responses everyone.
__________________
James Wong / www.hindsightproject.com
James Wong is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Apple / Mac Post Production Solutions > Final Cut Suite

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 01:18 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network