Insane render times with HDV?
 

I thought the problem with my HDV render times had to do with not using the Aspect Plug in. I just downloaded the new demo and now I realize that HDV must just render much much much slower than DV.

I load a clip in DV and add the same color and effects and it takes only 30 minutes to render.

Now I take the same exact clip in HDV using Aspect 3 and it takes 4 hours to render with the same color and effects.

Is this normal?

Yes and for logical reasons. HDV at 1080i is 4.5 times larger than a frame of DV. Every pixel has to go through the same process (in your example -- color and effects), so it simply has to take at least 4.5 times longer. There are other factors that make HD encoding slower (per pixel) than SD encoding. These is the complexity of MPEG encoding vs DV (MPEG is MUCH harder.) Then there is the issue PC cache sizes which aren't suddenly 4.5 times bigger. This means the larger HDV frames are being processed in slower system memory vs the smaller DV frames which fit better in L2 CPU cache. Basically the 8:1 ratio you measure is perfectly reasonable. This is why so many are upgrading their PCs for HD work, HD takes a lot of CPU.

Ok, thanks David. I guess I will look into using Vegas 6 and setting up a render farm.

Just as a note - I've been doing some effects work and colour correction with HDV footage since November. A huge chunk of render times is dealing with the GOP. While standard colour-correction etc. will obviously take 4.5x as long to perform by shear geometrical arguments - you are much better off converting to a lossless intermediate (i.e., uncompressed avi/bmp frames) before attempting a lot of frame-by-frame work and conversions.

There's no point wasting CPU cycles decoding the GOP for every frame - do it once at the beginning and never worry about it again.

-Steve

David, are the render times for DVCPRO HD similiar to HDV? Should I, could I use a DV25 proxy files for the straight video until final render? (Steven mentioned rendering to image sequences for VFX stuff, I feel that may suffice for video as I do this with animation).

If CPU L2 Cache determines the speed of rendering, as to whether or not a frame can be calculated outside system memory. Is it possible to split the frames amoungst dual or multicore processors? Is the software written to take that into account, or in the future?

If I had a small renderfarm, would it make sense or be possible to utilize that?

Pete

Peter,
Compressing DVCPRO-HD will be faster than native HDV as DVCPRO-HD has no motion compression, but it is still HD res so it still takes much longer than DV. Using proxy files is the offline vs online debate that as a CineForm employee I have to say using an online HD immediate is better. :)

Your caching idea only help if you do a lot of heavy processing on one frame across multiple CPUs, the problem with HD is you typically do a lot of light to moderate processing over many frames -- so there is little room to improve the cache performance. You always need to load the next frame, whether using a single or multiple CPUs.

lol!

Well, DVCPRO-HD would take less time to process than HDV, simply because the resolution is lower (i.e., 1080 HDV = 4.5x NTSC resolution, 1080 DVCPRO-HD = 4.0x NTSC resolution)... But once you've got the footage in a decent editing codec (Cineform! woot!) render times will be pretty much equivalent.

-Steve

Obiously I need to educate myself on DVCPRO HD a little further. But from what I gather, in laymans terms, DVCPRO HD is a faster data-rate than HDV, but a smaller frame size. DVCPRO HD has a larger color space (which is the extra data) than HDV. DVCPRO HD because of the up and down rezzing, quickly degrades and is best considered a capturing format, and should be rendered to something of higher or uncompressed quality for both mastering and edits.

Both DVCPRO HD and HDV are processor intensive vs. disk speed (where uncompressed formats benefit).

If I render to an uncompressed format, then likewise the data-size is four times ntsc, and would require 4x the disk speed?

Maybe I should learn more about HDCAM SR...

Pete

DVCPRO-HD should be excellent as a first-generation editing codec, just like DV. Because of the I-frame only compression, you don't lose anything if you don't recompress until you start adding movement, transitions and colour changes... but for the first edit, it should be fine. As soon as you have to render anything, you would be wise to switch codecs to something like Cineform, or go uncompressed.

-Steve

Thanks Steve. I have a better idea of what to do now. I'll look into the 10bit Prospect solution. I think with DVCPRO HD, I can stay within budget.

Pete

Re: Steven's comment...
 

Hi,

Could Steve or someone else explains in more detail what this means, and how to do this?

______

Steve wrote:

you are much better off converting to a lossless intermediate (i.e., uncompressed avi/bmp frames) before attempting a lot of frame-by-frame work and conversions.

There's no point wasting CPU cycles decoding the GOP for every frame - do it once at the beginning and never worry about it again.

-Steve

________

Lets see, I tried my first long render yesterday. I have a dual xeon, 3.6 gig, 2M L2, with 4 gig of memory. I took a one hour clip and rendered to wm9
1080i 60i. The darn think took 24 hours to render. And then when I looked at the clip, it was all messed up in color, etc.

So, if these are normal render times, it sure means I need multiple computers so I can still do work.

Dave

Dave, the math of 4.5x longer render that DV25 is correct. I have a dual xeon 3gig, so I feel your pain. I'm pricing a dual-core dual opteron solution, and can't wait for 64bit, that should help. But until then, ugh. (We're back to rendering overnight again).

It may be faster/more efficient if you just render a cineform avi and use the stand alone windows media encoder in post. I have a 1.5hr, 720p job. It took me about 19 hours (7 for the avi, and 12 for the wmv) . I usually render to more one than one format, mpeg2, etc. For that I use squeeze, and it's best to have a master an avi and do a batch.

I know it's temping to just run a wmv off the timeline, if you only rendering just one resultant file, so be it. But waiting all the time to check the end result is frustrating. Best thing is that you test a small sample clip first. 30 seconds is bearable, and you'll have results in 10 minutes.

Peter, thanks, thought maybe it was just me.

Good idea about making an AVI file.

Never thought I would be looking for a new computer since what I have is so much better than anyone else I know. Trying to see if I can find any
Pentinum D stuff floating around.

Now, if I could just load some of this s/w on a I/A 64 box. I work on the big stuff at work so I could get like a 128 CPU system put together under Windows.

I only did the 1 hr encoding to see how much space it took. Am trying to see what I can get onto a single layer DVD. Now, I have heard HD is only 20 minutes. This is why I can to compare wmv9 output vs m2t to compare the size. The I/O Data player is supposed to be able to play either.

Dave

Not every project will be done in Hi-Def as of yet. My Dell 650 is now barely adequate to handle the HD work. At the very least, capturing the raw material in Hi-Def would help in matters of shelf life and repurposing.

There is a savings going Prospect HD and capturing via HD-SDI were conversion happens in realtime. That's appealing. At the expense of more memory, going uncompessed would also save time -you only need a fast disk array.

Regardless, when it comes time to publish to a digital file, your still gonna face that 18-24 hour render for every hour of HD on the timeline. Better make doubly sure your titles and edits are correct before you hit that render button.