Computer store guy said Firewire and USB 2 are not fast enough for HD video editing
 

Hi, sorry if this was brought up before, I've been away.

I was at a computer store today looking at harddrives. The computer store guy asked me what it's for, I told him to put into an external enclosure for editing.

Then he goes on to say that to edit true HD, that USB 2 and firewire are not fast enough, that I'll need a SATA or eSATA drive.

I was thinking of using an external drive to do my editing off. But is he right? Something tells me that he's not, I recall firewire to be 50mb/s or something and it exceeds the the cable throughput of HD video.

But hey, I can't remember. Can someone quickly clear this up?

I'm shooting AVCHD (17 or 24 MB quality) on a Canon HF200, external drives are either USB 2 or firewire.

He's right. You can get it to work, but it won't be fun. Adobe recommends against it.

I ran into the same problem on my dual core. I got a PCI eSata card (cheap) and a 1 TB LaCie drive for all the video files. Works fine. From what I've read the big slow down is in the processor. I've heard the guys running the new i7 quad cores do pretty well.

so don't bother trying to do it off firewire is what you guys are saying? Do it on my internal drive?

The singular case of 'my drive' is worrisome. For HD editing you need at least three physical 7200 RPM SATA or eSATA drives. USB2 and FW400/800 are good for backups, that is all.

Oops, to clarify in my case, I will be editing the AVCHD video in Cineform Neo Scene (intermediate) and it seems to work fine.

All the more reason that 3 disks is a minimum. AVCHD is taxing on any system, Neoscene may lessen the CPU load, but it will increase the burden on your disks.

Most of the replies to this question are a bit too broad for my tastes.

The equation is pretty simple - number of streams times bitrate of codec being edited = continual read/write bandwidth required. The maths you can do yourself by just looking up the numbers for your individual case, there isn't a catch all answer.

You don't need to RAID disks to edit HD if you have a low bitrate codec. YOU WILL need fast CPU generally, and there CAN be a bottle neck in serving the CPU once you are doing multiple effects/grading on an external drive, which makes rendering slower, but doesn't make HD editing impossible.

Firewire 800 drives or e-SATA are preferable, raid arrays are useful once you are working with lots of streams at less compressed codecs. But HDV for example, is exactly the same bitrate as DV, so drive speed only really becomes a hurdle when rendering, not on playback/editing, and even then CPU is often the bigger bottle neck.

For best results you want to have dedivated video storage. we recommend a raid configuration.

G-Tech G-raid is an exernal raid- that connects via USB2, Firewire 400/800 and eSata

Check out our Video storage FAQ for lots more info on the subject

Videoguys Blog - Videoguys NLE Video Storage FAQ

Gary

Agreed, if you also take into consideration the bandwidth required for OS, programs, house keeping, pagefile, other services running, etc.

It is far too simple to say a video stream uses 3.6 MB and I have 5 streams, so the equation is 5 x 3.6 = 18 MB, thus a USB disk is sufficient. You disregard all the other tasks that need be performed by the OS, the editor, other programs in the background, the pagefile, etc. You also conveniently forget about fill rates, defragmentation, bus congestion and the like that all have serious impact on effective throughput.

The base line is: NO USB or FIREWIRE or GREEN DISKS for editing. EOS (End of story).

Harm, how do you then explain the many times I have successfully used firewire 400 drives for editing HDV footage?

Also - I'd never recommend USB disks for editing ever. Firewire I have used with success, but USB 2.0 as a standard was not designed with editing in mind.

Note - I'm running Final Cut generally (on an older iMac at times, it's one of our side machines compared to our main editing machines which are quad or eight cores, so I'll admit it's something we go to when there are no other) PC options may have more constraints because of the architecture.

It's not ideal - but there if someone is working to budget constraints, or even simple physical space constraints because of travelling and people have to make do then such drives can and do work.

If you are building a system, then I think the answer is optimize your system. The cost of e-SATA and a Raid config is not all that high and is well worth the inclusion if your setup can handle it.

I like to keep my ingested files stored on an external hard drive, and Ive found that to ingest using USB 2.0 WILL cause corrupted files and frame drops. Going to an external disc via firewire and all is well with the world.

But

As I give the client the raw files on hard drive, Ive found there to be a great scarcity of firewire enabled drives. So I usually ingest to an internal drive, then copy to the external.

Editing from the external drive poses no problem.

Ben

Graig,

As a very rough rule of thumb you will get these average transfer rates with different disks, but be aware that there are a lot of factors influencing this:

USB2: 18-22 MB/s
FW400: 30-35 MB/s
FW800: 50-60 MB/s
(e)SATA: 80-100 MB/s

These are very rough figures and are also dependent on the number of USB devices attached, the fill rate of the disk, etcetera. As you can see FW400 is almost twice as fast as USB2, so that may explain why you got by.

Why would anybody choose for 3.5" 7200 disks with a USB2 or FW interface, when there is no (significant) price difference with an eSATA interface?

If their machine had no way of getting an eSATA connection (such as an iMac. ;) )

To be fair, when I buy enclosures, I make sure they have e-sata aswell... It's just not all the machines they are being used on have e-sata as an option so they also have firewire AND usb (in case the go to clients with no firewire - which does happen still.)

I am sure the original poster gets the message - eSATA is good, a raid is better, firewire will do in a pinch in the right setup with the right codec, USB won't

Nuts, I edit with a Dell Inspiron 1520. Now eSata connection port on it. I think Firewire should be fine.

I've never had the "luxury" of a second internal drive in any of the computers I've used for editing video. From a single core emachines with a 1.6GB drive editing Digital8, to a single core hyperthreading HP with 512MB RAM to a dual core HP with 2GB RAM editing HDV (and with that one I started capturing to and rendering on external USB drives some), to my current Q6600 and Core i7 Dells editing AVCHD natively, I've encountered no problems other than the Q6600 processor being a bit slow.

if you just look at plain number, no hardisk , even in RAID, is able to support video editing.
fortunately , nobody use "uncompressed full bandwith" to edit video.
So the real numbers are the one used to record compressed video. (between 4 to 100 Mbytes/sec, but usually well under 50)
And since there is no tape , disk or memory that can be run fastest than a good harddisk, you can expect that Disk on Firewire or USB could fit the task.
Now, while any car can fit the task of bringing you somewhere, for sure, some will do it with better safety, speed , comfort or ease. same with computer equipment.
Depends your budget and expectations. Just make sure both will meet.

A simple rule that I used for years is do not read and write to the same disc at the same time, one stream per disc. This rules out using the OS drive for any video storage or temp files and also means having at least three discs, preferably four or more. This also means that external USB is just fine under these circumstances. IF you have more than one external make sure they are not on the same controller which for some motherboards may not be possible. I have used external USB2 for DV, HDV and AVCHD with no problems. Though now have no need to do so with my present system.
My current configuration is 250G OS, 250G temp, 2x750 RAID0 video storage, 1TB drive for finished projects. Backup to DLT03 tape drive. Also have several bare 1T drives that I use with eSATA when needed using Thermaltake disc dock.

Ron Evans

I use an external (well, several) FW400 external discs for capturing and editing Apple ProRes compressed video sourced at HDV720P60 and I'm writing and pulling 1GB per minute (ProRes 720P60) with no problems. Currently I'm on an iMac until Snow Leopard comes out so I don't have the luxury of eSATA or internal drives. My last edit machine had 4 internal drives: one for the OS, one for video capture, one for audio capture and one for render. Worked VERY well until drives started dropping like flies...

I edit HD off of a FW800 external all the time.
Does that make me the exception to the rule?

what is a green disk?

Doesn't that have an express card slot? If so, esata can connect to it.

Green disk are disk that use low power, the drawback being the lower performance mainly due to lower speed rotation. Since video editing usually requires above average performance and green disk are average or under, you can expect problems.

I’ve made due with FW400 in the distant past for DV – it was ok.

Now, for (sans highest setting) Cineform, HDV, AVCHD or DV, a simple second sata works well enough. Processor speed is more of an issue with these compressed formats than modern hard drive speeds. Most hard drive these days are more than fast enough. I’ll also mention that I don’t commonly work with multiple streams

For those who usually work with multiple streams, would a straightforward software raid work?

He's not entirely wrong, but he's not entirely right either.

Firewire will give you better performance than USB 2.0, and FireWire 800 (the new firewire) will give you even better performance than firewire. But eSATA gives you performance equivilant to an internal HD, which is why I prefer to use them.

You will experience slowdown, especially with AVCHD at those bitrates, on a USB 2.0 connection. USB 2.0 operates in bursts, and can't maintain sustained speeds; the problem is not reading one file at a time, but when you have multiple files that need to be read simultaniously. Like, for example, putting more than one video together in the same project. And any dissolve effect will need to read more than one video file at once.

USB 2.0 drives are still useful, however, if you're proxy editing OR you're willing to live with stuttery/slow playback during the editing phase. Vegas is actually pretty good about USB 2.0 drives, Final Cut Pro is not... for various reasons.

Honestly, you have a choice of enclosures out there, I'd go ahead and get one that can operate as EITHER USB 2.0 or eSATA. If USB 2.0 works for you, use that. If not, go with eSATA.

In my Experience with HDV and MXF files, USB is no good and 1394a (FW 400) can be problematic at times.

1394b (FW800) drives and eSATA are no problem at all. As to the why use FW800 over eSATA? In my case the main reason is daisy chaining. I can run 4 FW drives off my FW PCI 2.0 card(2 inputs) of mixed drive sizes with no issue. My eSATA card can only run 2 drives and they must be the exact same size.

I love the eSATA for content back-up and tend to use it as such. I have clients that provide me with content on FW800 drives for editing or capture and I can just chain them in without disabling any of my drives. If it were an eSATA drive they would have to provide me with one that is the exact same size and I would have to disable one of my drives to plugin their drive.

If you have a open 5.25" drive bay and a spare internal SATA connection, go with this kind of trayless removable drive system. It gives you virtually unlimited storage.

Trayless Removable Harddrive System

Tom's hardware just ran an interesting comparison of a 3 year old, 200G 7200RPM drive against a current, 1T, 5400 "green" drive. The newer drive seemed to be more capable on virtually every analysis, simple because technology continues to improve.

This has me debating new drives - I already have a new 500G for a new build, and it's very noticeably zippier, which leads me to suspect that the same would be true while pulling things such as AVCHD files off the disks - a faster drive might improve throughput enough to give better overall performance.

Optimizing a computer becomes an interesting task, as ALL the subsystems can affect performance, and you're only as fast as the slowest (bottleneck) subsystem... meaning CPU speed, FSB, Memory, hard disk and (not so much for video editing) video, each play a part.

I'm not too sure about anything some "computer store guy" says... unless they've got direct experience, it could be some secondhand hearsay from a web forum (and aside from DVi, we know how accurate THOSE are!). YMMV...

Another point to be aware of is that once you are using a file based video then the problem should be improved from DV or HDV and these would mainly have problems on capture. Once the files are on the hard drives editing will just have slowdowns etc but no losses. Issues will then arise on export to tape.
Only capture and export are critical that there is contiguous file transfer and where continuous drive throughput is important.
IF you are editing DV or HDV then the drive issues are real. IF you are editing AVCHD then drives are less of a problem but CPU power is VERY important and is the bottleneck. IF you are using an intermediate like Cineform or Canopus HQ the transfer rates will be much higher and thus drive speed is again important for editing( but you will not loose anything just editing preview quality). When you render your final file again drive speed is not important. This is purely a computational task.
Bottom line. For DV and HDV ( they are the same data rate) capture to a fast drive, any modern internal or a single external should be fine. Export to tape from a fast drive. These are the only times where realtime performance is critical. IF you are capturing HDV with realtime conversion to Cineform or Canopus HQ then drive speed and CPU power will be important.
As far as I am concerned it doesn't matter too much for AVCHD as this is a file copy process either from a hard drive in the camera ( which will be slower than all the hard drives on your PC) or from a memory card ( again likely slower than your hard drives)

Ron Evans

It's really about data density. For performance we want the largest capacity on the fewest platters. This setup moves the most data under the head per unit of time.

I would not buy "green" drives except for backup. The smallest drive I would buy today is 1TB. As soon as the 2TB drives get up to 7200 rpm I will buy those. Remember too that as drives get full they slow down, especially with raid. For the main edit drives I shoot for 2x capacity to keep under 50% full. OSX is good about using the outside fastest part of the drive first. Not sure what windows does.

Esata is just sata with better connectors. If an esata drive attached to a card is slower than internal drives it means the PCI card is cheap.

I am editing 1080i and 720p footage on a Firewire800 drive without problems. Seems like an exception, too! ;)

For those who are saying FW is too slow, I did want to note that i had an issue with FW when capturing on a laptop via a PCMCIA adapter card. I had the camera plugged into the FW400 port and a FW800 drive in the 800 port next to it. Many dropouts. This was an issue with the PCMCIA slot bus and card and not the interface. I have never had issue with my desktop system in capturing to FW800 externals or editing with them. Perhaps they too were using cards that the input from the camera and output to drive resulted in conflict? Maybe they can elaborate on the experience.

True HD??? I wish that term was banned. People generally don't know what it means and it is too often mis-used. If he means Uncompressed HD then he is correct, but who the hell edits Uncompressed HD?

Judging that you are shooting with a HF200, i will assume you are not editing HD for pro use. So if that is the case then Firewire drives are completely fine for editing HD (HDV or AVCHD). I have even used Lacie firewire 800 drives for professional use in the field editing on a laptop in ProRes and have had no issues.

Just some clarification for some.

For capturing FW drives are OK, even USB can be sufficient, for instance with OnLocation.

For editing things change drastically. As the number of tracks increase, effects and transitions added and audio tracks added, your disk requirements in terms of sustained transfer rates increase significantly.

Given that eSATA drives are about twice as fast as FW800 and about three times as fast as FW400 for approximately the same price, the choice is obvious...

A few points.

True HD. Back when I was using 3CCD SD cameras and HD was just a wet dream in the filmmakers mind, we would call anything with 1280 X 720 or higher True HD,... because of the resolution... these compression codecs didn't even exist back then so no need to geek out on it.

PCMCIA card adapter - I have a dell Inpsiron 1520, it's just over a year old and has FW400 (I think). Where can I get a good adapter, or should I just upgrade my laptop to one that has a eSata connector?

FW800 - how do I check to see which FW connector I have and that of the external drives I have?

My Canon HF200 doesn't even have a FW port, it has USB, so no problem with file transfer there.

FW400 had either a 4-pin mini or a 6-pin connector. FW800 has a 8-pin (or was it 10-pin?) connector.

Correct, ASSUMING that you have enough COMPUTING horsepower to process multiple streams at once. The second you run out of CPU/GPU processing power and have to render, you're back to one stream. And it also depends on how many stream NEED to play concurrently: if I layer 99 video tracks (discounting audio for this example) on my FCP timeline at 100% size/speed/opacity, I can STILL play my timeline in real time without rendering BECAUSE only the top layer is visible, therefore only one stream NEEDS to play back. Now, start playing with blend modes, opacity, backgrounds, foregrounds and things change again...

Don,

Are you saying I could connect my G|TECH RAID3 to an empty SATA (not eSATA) connection on my mother board w/ some type of an connector adapter?

This link seems to indicate that's the case:

http://www.nextag.com/esata-to-sata-cable/compare-html

Thanks much!

Yes. My Gigabyte motherboards came with the adapter to provide two eSata connections. Just like this ANTOnline.com - StarTech.com ESATAPLATE2 StarTech.com ESATAPLATE2 - Serial ATA internal to external panel - Serial ATA 150/300 - 7 pin Serial ATA - 7 pin external Serial ATA - 1 ft

Ron Evans

If I do this, I imagine I have to be sure to turn on the external drive before booting the computer, so the BIOS can recognize it. And turn off the drive after powering down the computer. Do you think that's correct?

Thanks much.