Render Farm!

[Anhar Miah]

I think now that HDV has been thrust upon us, we need to think about rendering,

I was rendering the other day normal DV to DVD and it was an absolute nigtmare in terms of time, if only i could have more than real time....

I was then looking into render farms, and got thinking if it would be possible to hook up some old's PC say three to four and use a "mini" render farm, after alot of searching it seemed there are lots of people wanting the same thing, however it was rather complicated....(linux + a PHD in computing... :) joking....

anyway i wanted a simple solution Guess what!

you guys probably know, but vegas 5 has network render feature!! just install vegas on your main computer and intsall the client app on the other "slave" computers and make a small home network using XP/win2k whatever and then VOLA, you have a render farm, the best thing is you can add as many compuiters you like..

imagine rendering DV (or HDV) for that matter in miniutes instead of hours/day !


anyone has tried this? (i know its a little early)

[Barry Green]

Network rendering works fine, but currently the licensing arrangement doesn't allow you to network-render MPEG-2 content. So it would be of no benefit for HDV material, or making the render for DVD material.

[Dustin Cross]

I think network rendering is a great way to go.

With Premiere or FCP & Automatic Duck, import your project into After Effects and use the AE network render engine to render out a 16bit image sequence. Then you put that Image sequence back into your NLE, sync up the audio, and make the MPEG. This way any color correction or effects that take up most of the render time will take advantage of the network render and be at a better render quality than Premiere or FCP.

You can put the AE network render engine on all the extra computers you have laying around.

This is what we are doing with the feature I am working on right now. We are cutting it into multiple short reels in FCP, which we then bring into AE for color correction and any effects and then render it to a 16 bit image sequence.

Since FCP doesn't do 5.1 we will need to export our audio to something else for that.

Hopefully by the time we finish all this the next version of FCP and quicktime will be out and support 5.1 audio, so we can sync everything back up and make our MPEG from FCP.

We are making this up as we go.

[Robert Mann Z.]

if you have $700 to spend buy edius pro 3, you can work on serveral layers of hdv in realtime (NOT PREVIEW, or FRAME DROP, BUT REALTIME)...

i have been doing so all week now, with transitions dissolves and color correction...now its true edius is not vegas feature wise, but for simple editing with rt perfomance from a software app on a 3gig laptop thats not bad...

if you have $1,200 and a free 64bit slot buy the hardware and software and edit even more layers in real time...

with my laptop its nice to edit hdv on my couch with out having to start up a render farm everytime i want to edit...but i have to admit there is cool type of gizmodo factor to having a render farm at your fingertips...

[Zack Birlew]

Question, is the slot for EDIUS 3's hardware actually a 64-bit slot or is it PCI Express?

If it's a 64-bit slot, then the user would have to buy a server board (dual Xeon or Opteron solution)

If it's PCI Express then a regular desktop computer can use it (currently nothing for AMD64 or AMD FX processors until early next year and that leaves Intel's LGA775-socket boards)

I'm very interested in the EDIUS hardware (preferably EDIUS SP). But I'm not too sure on the slot issue. =P

[Jason Brunner]

When I was young, (about five years ago) I knew a guy who had a fantasy about making a giant render farm out of a bunch of play stations or x-boxes or something like that. Has anyone come up with a set up like that? If you could get it to work it would be alot of horsepower for the buck. I don't know if its feasible but it sounded interesting......

[Robert Mann Z.]

64 bit

[Rob Lohman]

Barry: I don't see how not mpeg-2 encoding is a license issue over
the network. Let me get down and dirty a bit.

MPEG-2 compression is an inter-frame compression method. Or in
other words, it stores (usually) only the differences from the
previous (and following) frame. An individual node in the render
farm does not have access to this information since it is not
rendering the entire MPEG-2 file. Not there might be some scheme
to make this work with sharing files, but then you will get into
problems with different encoding speeds and such.

So I really do not see MPEG-2 encoding happening over a render
farm network anytime soon (the big boys might have such solution
with calibrated machines [speed wise] and a sync system to make
sure each node has data to work with).

Rendering can be broken down into three parts:

1. rendering of the content (effects, color correction, letter boxing etc.)

2. video encoding (MPEG-2 etc.)

3. audio encoding (Dolby Digital etc.)

Now the first is brilliant for network rendering and is also the most
used one. This is where you will see the most benefit with Vegas
when doing a network render.

After the rendering is done 2 & 3 are multiplexed together into
one file.

The problem with MPEG-2 encoding is that it takes a pretty long
time (if you want good results), there is no real way around this
(personally I have my reservations about the quality on those
realtime hardware MPEG-2 encoders). Usually I do some MPEG-2
tests (or if you've done enough encodes you will KNOW which
settings will produce good results or have a template) and then
just let it render overnight.

If rendering time is really that important it might be best to invest
in a new ultra fast AMD64 CPU with the fastest mainboard and
harddisks you can find? Then at least you know it is being encoded
as fast as you can and you should be ready for when the 64 bit
encoders should hit.

Until then...

[Kevin Shaw]

Could network rendering work for output to the Windows Media or DivX HD formats? If so that could actually be useful for HDV work, as these are two of the more promising output formats and are reportedly painfully slow to encode on current single computers. Another way to approach this would be to simply break long projects into chapters and encode each chapter on a separate computer, which is something I would probably try if I was looking at a 24+ hour render time for a 90-minute HD DVD.

[Rob Lohman]

From the Vegas 5 manual:

Distributed network rendering splits the video into segments that are rendered by multiple computers. In this mode, each computer renders a portion of the project, and the rendered sections are then reassembled into a single file by the one computer (called the stitch host).

Distributed rendering is a good way to reduce the time it takes to render a project containing a significant amount of processed video (video effects, transitions, panning/cropping, track motion, and compositing). However, distributed rendering requires increased disk space and network traffic because each segment must be saved before the final output file be generated.

Nontemporal video output formats, such as DV or uncompressed AVI, are also well suited for distributed network rendering because segments can be reassembled without re-encoding.

I'm not sure whether this will be supported with WM or DivX for
example, but you should easily be able to test this for yourself
(I'd say you have more chance with DivX than WM).

However, it is certainly true that perhaps the easiest and in this
case best method might be to render individual sections on each
computer and merge these together (this is basically only possible
[without re-encoding] if you are going to DVD through MPEG2, or
use uncompressed / DV compression).

I'm not sure how the Vegas license works in these cases. If I'm not
mistaken you can add 2 network rendering computers, but if I
remember correctly those Vegas installations behave differently
than your main machine (ie, certain features might not work like
editing on such a machine).

I would suggest you run some tests to see how each method
works and what best fits your workflow. This way you can also
identify problems etc. early on.

[Dennis Adams]

Vegas network rendering for temporal or licensed formats can be done by using an intermediate codec on the render machines. There's an uncompressed 4:2:2 codec which is useful for this. The final "stitch" is done to the final output codec, on a single machine. This is really only useful for when the render portion of your project outweighs the encoding, which is NOT the case for "regular" editing. So, for HDV editing with WMV or MPEG-2 final output, network rendering would not be useful. Network rendering pays off when you're using a non-temporal codec (such as AVI DV) and/or when you've got a lot of compositing or video FX.

///d@
Sony Media Software

[Rob Lohman]

Dennis: thank you for your response. As you no doubt know
DiVX and XviD are non-temporal codecs as well (I'm not sure
about Windows Media). So would it work if you want to output
a DiVX AVI for example? Ofcourse the benefits aren't that great
as you said, I totally agree.

[Dennis Adams]

Rob:

I'm pretty sure those are, in fact, temporal (interframe) codecs, otherwise they wouldn't be as efficient as they are.

The codecs that I know of that do no-recompress stitching are AVI/DV, some M-JPEG formats, and the Vegas YUV 4:2:2 codec.

///d@

[Joshua Starnes]

When I was young, (about five years ago) I knew a guy who had a fantasy about making a giant render farm out of a bunch of play stations or x-boxes or something like that. Has anyone come up with a set up like that? If you could get it to work it would be alot of horsepower for the buck. I don't know if its feasible but it sounded interesting......

It has been done by a couple of university computer science department - but it's not for the faint of heart or non PhDs :)

I don't know about the PS2 computer, but the XBox one runs on a specially created version of Linux in order for it to do what they want it to do. It's some 30 or so Xbox's networked together - supposed to be quite a nice supercomputer, though, by the nature of the beast, there's not muc that it can actually do.

[Rob Lohman]

Actually, the whole xbox renderfarm etc. is not economical anymore.
The machine's are quite slow by today's standards and will not
give you enough power for what they cost. It is cheaper to get
a couple of high performance (build yourself) PC's instead.