Dispelling Rumors - Vegas 8.0 v 8.1 - Compression Tests
 

For quite some time now, I've been hearing how using .MOV file types in Vegas is painful. How quicktime is "crippleware" in Vegas, etc. I also knew that I was seeing results in my work that seemed to refute what I was reading.

Last week, I stumbled upon some informal benchmarking on the Sony Creative Suites forum called HDVRender Test. Essentially, the idea was to take a known .veg of generated media, and render it on various machines so people could compare performance numbers. Great idea, so I tried it. Today, I decided to take that one step further. I wanted to use that test to see how various codecs stacked up, and how the two current shipping versions of Vegas stacked up. These are my results:

Machine:

Dell Precision M6300 Laptop
2003 XPPro x64 SP2
Core2 Duo T7250 @ 2.00 GHz
4GB RAM

Vegas Versions:

Vegas 8.1 build 171
Vegas 8.0b build 217

The Codecs: Mpeg-2 HD @ 25 Mbps, Uncompressed, Lagarith, Sony AVC, Avid DNxHD, JPEG2000.

So how do they compare:



Test 1: Stock HDVRender Test

V8.0 MXF 05:46 1440x1080x32 29.970i mpeg-2 HD 25 Mbps(CBR) 48KHz16 bit Stereo 15,810Kb

V8.1 MXF 05:03 1440x1080x32 29.970i mpeg-2 HD 25 Mbps(CBR) 48KHz16 bit Stereo 15,810Kb

** As expected, the 64bit version was significantly faster.


Test 2: Uncompressed AVI HDVRender Test

V8.0 AVI 05:46 1440x1080x24 29.970i Uncompressed 48KHz16 bit Stereo 684,548

V8.1 AVI 05:03 1440x1080x16 29.970i Uncompressed 48KHz16 bit Stereo 456,809

** 64bit render only offered 16 bits per pixel! Speed was faster


Test 3: Lagarith AVI HDVRender Test

V8.0 AVI 05:49 1440x1080x24 29.970i Lagarith 48KHz16 bit Stereo 13,176

V8.1 AVI 05:04 1440x1080x16 29.970i Lagarith 48KHz16 bit Stereo 13,176

** Again 64bit only offers 16 bits per pixel


Test 4: Sony AVC HDVRender Test

V8.0 MP4 02:58 1440x1080x32 29.970i Sony AVC 20 Mbps/512 Kbps 48KHz16 bit Stereo 1,881

V8.1 Would not Render

** This was unexpected as I tried to choose codecs that were in both versions


Test 5: Uncompressed Quicktime HDVRender Test

V8.0 MOV 06:18 1440x1080x32 29.970i Uncompressed 48KHz16 bit Stereo 912,193

V8.1 MOV 03:15 1440x1080x32 29.970i Uncompressed 48KHz16 bit Stereo 912,193

** File size is larger than AVI because it offers 32 bits per pixel. Render speed in 64 bit is nearly twice as fast as 32 bit and much faster than AVI even with increased bit depth.


Test 6: DNxHD 220x 10 bit HDVRender Test

V8.0 MOV 06:01 1440x1080x32 29.970i DNxHD 220x (10bit) 48KHz16 bit Stereo 135,343

V8.1 MOV 01:50 1440x1080x32 29.970i DNxHD 220x (10bit) 48KHz16 bit Stereo 135,343

** Render time is less than 1/3 that of 32bit version and over twice as fast as any AVI.


Test 7: DNxHD 45 8 bit HDVRender Test

V8.0 MOV 06:01 1440x1080x32 29.970i DNxHD 45 48KHz16 bit Stereo 28,542

V8.1 MOV 01:50 1440x1080x32 29.970i DNxHD 45 48KHz16 bit Stereo 28,542

** Same performance gain as 10 bit version


Test 8: JPEG2000 HDVRender Test

V8.0 MOV 06:16 1440x1080x32 29.970i JPEG2000 (32Mbps) 48KHz16 bit Stereo 20,056

V8.1 MOV 03:52 1440x1080x32 29.970i JPEG2000 (32Mbps) 48KHz16 bit Stereo 20,129

** Performance abot 60% better in 64bit version.


In terms of compression:

AVI Files:

Lagarith offers a 52:1 advantage over uncompressed


MOV Files

DNx 220x offers a 6.5:1 advantage
DNx 45 offers a 32:1 advantage
JPEG2k offers a 45:1 advantage



So while this test is hardly scientific, it does seem to offer up some very interesting trends.

1. 64 bit Vegas is CLEARLY faster on the same material using the same codec as 32bit

2. 64 bit Vegas has some significant issues with .AVI files and bit depth

3. The conventional wisdom that somehow Vegas is optimized for .AVI may be true in 32 bit, but the 64bit version blows that out of the water.

So it seems in terms of rendering, that the most speed is available by working with and rendering .MOV file types in 64 bit Vegas. Which is what I've been saying for months now. If the Avid DNxHD codec is used, the render times are faster than anything else around (though I would like to test Cineform) and they can be traded off to Macs or other PCs without cost.

This test took a couple hours to run. I'd be curious to see it replicated on one of the quadcores or dual quadcores out there. Disk speed didn't seem to be a big factor as the render times didn't change much even when writing out uncompressed files versus the far smaller highly compressed intermediate files.


Comments welcome.

Nice work Perrone - I've settled with Vegas Pro after my own "non-scientific" testing of various NLE's - it appears that 64bit Vegas Pro seems to offer some clear advantages as a basic cutter type NLE, but you lose much of the ability to work with plugins compared to VP 8.0c.

Any suggestions on a dual workflow methodology?

I dont use many plugins. But basically, I work in 8.1 whenever I can, and 8.0 when I have to.

I'm beginning to wonder if my issues with Quicktime being crippleware is a result of using Vista 64? I just restored an acronis image of Xp x64 on my laptop after having given Vista 64 close to two months of using and TBH - my laptop was marginal in performance at best. I have basically the same specs as you do (Dell D620 with T7200 Core2Duo and 4GB RAM)

After restoring the Acronis Image, my laptop has sprung back to life so I have an unsubstantiated feeling that QT is sluggish on Vista 64 - at least it has been for me.

I"m going to redo my testing of the DNxHD codec after restoring my x64 XP Pro on my desktop today and see if that's the case.

Here's the test file

CLiff, if you are going to run Vista 64 make sure the Vista goodies are all turned off so that it looks like XP then it works really well. Turn off user account control, turn off aero visual effects etc, use Classic theme and set for performance.

Ron Evans

Did all those things - Quad core Q6600, OS on separate 7200rpm Seagate sata drive, separate drives for video, audio, etc

All I can say is that after 2 months of really giving Vista 64 a shot, I don't see any benefit to using it - I think I'm going to hold out for Windows 7 and see if M$ can get its act together on streamlining their code some more.

Hard drives seek is laggy, numerous processes running that affect performance, twice as much memory being used while idle - My impressions now are that Vista is the equivalent of Windows ME - some profess it works very well for them - I gave it a fair shot and I'm left unimpressed.

Perrone,

I was the one who came up with the "rendertest-hdv.veg" file and started the thread on the Sony Vegas forum. It's turned out to be a very popular thread with over 325 responses.

I would like to suggest that you cut & paste the results from above and post it to the Vegas forum thread. I'm sure everyone there would be very interested in your results and I'm certain a few people would take the challenge of replicating your test on their dual-core, quad-core and i7 machines. Perhaps this particular test methodology deserves its own thread on the Vegas Forum since it expands on my original single-purpose test.

John Cline

John,

I am not a member of that forum. Please take this post and put it over there with my blessing. I'll check in some time in the next week or two.

Has everybody been complaining about .mov timeline performance or render performance? It's been my impression others (including myself) have been frustrated with placing .mov files on the timeline and experiencing less than steller performance during the editing process.

Jon

For what it's worth, I just finished rendering the test file in Quicktime uncompressed in version 8.1 as an .mov file. Rendertime on my i7 clocked at 3.0Ghz took 0:53 seconds. My resulting file size however was 934,007.

the Jpeg2000 version just as yours came in at 0:51 seconds. File Size = 20,533

Jon

People seem to have differing tolerances for performance on the timeline. Some people are happy with AVCHD editing on similar hardware to people saying it's unusable. When I see that post houses handling multi-million dollar features are generally still editing in SD, I have to wonder what people are talking about.

If things are too slow, render proxies and work. If you want to cut online HD then buy 8 core machines like Hollywood does, and use light codecs and 8-way+ RAID like they do.

For me, editing on the timeline, is a pretty important factor. I often am dealing with multi-cam edits with 3 or 4 video streams (cams) at once previewing in Vegas and if I can't get at least 20+ fps on each clip, making cuts gets difficult and just downright frustrating. This on top of the fact that video quality has a tendency to vary greatly from sometimes crisp and clean video to other times grainy video with muted colors. Again, makes color correcting difficult.

How can you tell what FPS you are getting in the preview window? Also, what frame size (1080/720/480) do you typically edit with?

For me, cutting in SD makes sense because I don't have an HD monitor. For others, it's probably different.

I edit on a dual monitor system. I work almost exclusively in 1080i HDV (1440x1080). I have dual 24" monitors running 1920x1080, so I have the ability to run my second display as a full screen preview however I find that completely unnecessary - so I set my display window to run at half resolution (720x540 - par 1.33). I find that 1/2 resolution is just.

Vegas tells you at the bottom of the window the playback rate, typically it's pegged at 29.97fps but if you start adding heavy color correction, transitions, etc, etc it can start to degrade...

Jon

Core2Quad vs Corei7 on render test...
 

Jon--Just for a general comparison of my core2quad system to your corei7 system, I ran the same render test for same qtmov and I can see what I will gain when I eventually upgrade.
My basic sys specs: qx6700 @ 3.2ghz on x38 chipset w/4gb ddr2 @800mhz. Vista 64, Vegas 8.1...
Perrone's core2duo system took 3:15, my quad system came in at 2:12, and yours blew both of them away in :53...very impressive gains to had here with the new intel chip! I'm wondering if there'll be even more speed with Windows 7.

Good evening,

I down loaded the file for the rrender test. I ran it on my I7, 2.66 ghtz with 6 gigs ram.

I down loaded 8.1 and run it in vista 64. I used the same specs as far as I know.

mpeg 2 71 seconds
Mp4 sony 75 seconds
AVI 37 seconds
QT 102 seconds
mp4 main concept 37 seconds

I ran the task manager at the same time so I could follow the performance:

Interestingly, Main concept is the only one where it used 100% of the cpu and 3.7 gigs of memory.

the others used 5 to 20% and averaged about 2 to 3 gigs of memory.

What is that about???

I do not know if these are good or bad. I have been working on leaning out my computer and still have more to do I reckon.


I could also run them on my older dual core, but not sure if there is apoint to that.

Yes, it really does show the superiority of the i7 architecture when coupled with a software application that can make good use of the technology... i7 reigns superior here and me overclocking the cpu (safely and comfortably) to 3.0Ghz from the standard 2.66 seems to yield that much more of a punch in performance. It's for this reason on this machine I'm really not that concerned about optimizing it for performance. Sure I spent 20 or 30 minutes doing some of the obvious tasks such as turning off Aero, moving the page file, etc, etc... but at this point, I just don't feel the time spent to try and eek out another 1-3% in performance is justified..

Everything I hear about Windows 7 screams more efficiency and better performance. Early beta versions of the software are supposedly performing at Vista's levels now and you can typically expect performance improvement as the software nears release. I believe Windows 7 may very well end up being the best and most successfull operating system ever released by MS when it's all said and done.... Let's keep our fingers crossed.. :)

Jon

seems perky enough...
 

Jon, I've been running the Win7 beta 7000 on one of my other machines (q6600) and it definitely seems to run well, but I haven't run Vegas 8.1 on it since I don't have a license for another pc. Other than having to play a couple of compatibility games to get it to recognize the intel chipset drivers (p35), it was an easy installation and it's run all the apps I've loaded on it to this point.
I hear the i7s OC really well--I've had to slap a peltier cooler on my QX6700 to get it to 3.2ghz, but being the first iteration of the quad (Kentsfield) and consuming 130 watts at the default 2.66ghz, it's understandable that it runs really hot when it's OC'd.
By the time I'm ready to run an i7, they should be plenty cheap, and hopefully the ddr3 will come down in price, too.

I have never overclocked a machine, is it complex, or can an average user actually do it???

If you've never overclocked a cpu or system before, and you depend upon your system's stability, don't do it. You should spend a few bucks on a faster cpu before you make your first attempt at overclocking. But if you have a spare system sitting around and want to play, overclocking is a good thing to know, not to mention you'll learn all kinds of things about your system as a whole. There's usually plenty of headroom on core2duos, quads (and I've heard the i7s too), to squeeze some extra speed out of them, providing you can keep them and the rest of your system cool enough.
Since I have the qx6700, as opposed to the q6700, the multiplier on the chip is unlocked, and overclocking is simply a matter of changing the multiplier, as opposed to raising the front side bus frequency. Subsequent versions of the core2duoquads, for the most part, were better overclockers, especially when then Intel put out the Penryn, which had a higher FSB and 45nm die vs 65nm. So I wound up getting a Peltier cooler, which runs current through bimetal plates to transfer heat, and can really cool things down, as long as you don't cool it low enough for condensation to occur, and you exhaust the transferred heat efficiently.
Since it's not a subject short enough to explore in a single posting, (not to mention that I'm getting WAY off thread here), I'll defer you to an OC forum if you want to explore this more. [H]ard|OCP - www.hardocp.com

Once again, please don't do your first overclock on a system you depend upon for business...you can fry your hardware easily, or at the very least void your warranties. With prices relatively low these days, upgrade your hardware instead!

I generally agree with this comment above however in the case of the 920 i7, I completely disagree.. The chip appears to have *SO* much headroom that doing a very mild overclock should be completely safe yet yield signficant performance advantages. I've been building systems since I bought my first XT computer with 256K of RAM and no hard drives. I've mostly always steared clear of overclocking but with the i7, I say go for it as long as you are comfortable manipulating BIOS settings.

To use an analogy it would be like buying a brand new car and saying "Don't break the 65mph speed limit - it's not safe, if you get an accident you could be liable because you were breaking the law. It just doesn't pay"

Again, though in concept I agree, I think going 70-75mph on the open highway is not going to kill anybody when your car was built to travel at 100 easily.. :)

Jon

your analogy...
 

Since he's never overclocked before, a more correct analogy would be: like you just got your license and and you bought a brand new car, etc...
I wouldn't advise anyone, overclocking for the first time, to practice on their critical system. I wanted to respond to his post but keep him cautious.
Even though the i7 is extremely overclockable, if the chip doesn't have unlocked mulitpliers he'd be changing his system bus, which would affect other system components also--how many depending on his moboard's flexibility.
Are you overclocking your i7?

Okay, I'll conceed a little on that one Vincent..

I do overclock my i7 but as I mentioned before, overclocking is not something I typically do for the reasons previously mentioned. The i7 is multiplier locked however manipulating the new bus (It's not FSB anymore) is far more flexible and forgiving then the old days, so going from 2.6 to 3.0Ghz is a really easy overclock. It's almost as if Intel along with Mobo manufacturers are encouraging you to do so..

Jon