Multi Core Rendering at DVinfo.net

Go Back   DV Info Net > Apple / Mac Post Production Solutions > Final Cut Suite

Final Cut Suite
Discussing the editing of all formats with FCS, FCP, FCE


Reply
 
Thread Tools Search this Thread
Old September 29th, 2009, 08:46 PM   #1
Regular Crew
 
Join Date: Jan 2009
Location: Wellington, New Zealand
Posts: 82
Multi Core Rendering

Well, Here we go again, from what I've seen in the world everyone has issues with this sorta stuff and FCS. Im wanting to render some motion graphics throught compressor, and thats fine, but when i send it to my cluster it still only uses 2 cores and not even 100% (averaging maybe 60-70%

I found a number of tutorials for multicore rendering, but I can't get it to work with me.... Here are the details:

MacPro 2.8ghz 8 core (2006 or 7 I think)
4gb Ram
FCS3
OS X 10.6
Separate OS X and Scratch disks

Attached are my screenshots, hopefully someone will know how to fix this, or if I need to upgrade anything - Ram is already on my list of things to do....

While I'm here, every time I open Q-Master in the sys prefs it has to close and open again. Normal? Bug? Just Me?


Thanks
Ryan
Attached Thumbnails
Multi Core Rendering-screenshot1.jpg   Multi Core Rendering-screenshot-2.jpg  


Last edited by Ryan Hansen; September 29th, 2009 at 08:47 PM. Reason: Duh - attachments... :)
Ryan Hansen is offline   Reply With Quote
Old September 30th, 2009, 11:04 AM   #2
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
Simple fix:

There are 2 issues here, one is the setup:

You've setup your cluster to work as if you're spreading the rendering jobs across a network allowing other computers to share your resources, that's not what you're after. You don't want to *share* you want to *Manage*.

Uncheck Share and check Manage instead, then under "Options for selected service" make sure Compressor is selected in the list above then hit the "options..." button and select up to *half* of the available cores on your machine. Then when you submit your job to Compressor you check the box "include unmanaged services".

BUT, only MPEG-2 encodes will spread across multiple cores, that's because MPEG-2 processing is by default multi-threaded. AVCHD and H.264 are NOT multi-threaded codecs so even with distributed processing turned on you will only use one of your CPU cores, period. Unless someone creates a multi-threaded version of those codecs that behavior will never change. So in fact what you are seeing in your Activity Monitor is correct: the H.264 encodes are only using a single core. There's nothing more you can do in Compressor or Qmaster to speed that up unfortunately.

That is one reason Matrox and BlackMagic have created hardware cards that accelerate H.264 encoding because you can't do it in software. If you're planning on working with H.264 encodes a lot it would probably be beneficial for you to check into the hardware encoding options.

For details on this setup pick up the book from Brian Gary, Compressor 3 Quick Reference Guide.
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old September 30th, 2009, 04:57 PM   #3
Inner Circle
 
Join Date: Dec 2005
Location: New York City
Posts: 2,615
If you are using Snow Leopard, the QMaster sys pref window will open/close/reopen because that particular system pref is still in 32-bit code!

I just set up a new 8-core and had a question about Qmaster, Final Cut and multi-processing. When I render in FCP, usually four cores activate to about 60 to 75% while the other cores usually remain below 5% as if they are being used for background computer processes. Sometimes a fifth or sixth core pops up for a few seconds. Is that a limitation of the way FCP renders or the fact that I only have the stock 6gb of RAM installed? In the searches I've done in the Apple literature, Qmaster is never associated with FCP only Compressor. Have I missed something here?
__________________
William Hohauser - New York City
Edit/Camera/DCP production/Animation
William Hohauser is offline   Reply With Quote
Old September 30th, 2009, 05:05 PM   #4
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
That's right, Qmaster is for sharing resources for Compressor encoding only, the idea being that you could turn multiple Mac Pro's into a render-farm by distributing the workload across dozens of cores to finish a large-scale job. However the promise of Snow Leopard being able to boost render performance in FCP7 only occurs with newer GPU cards that are actually designed to leverage OpenCL technology.

Only mainframe style systems such as Quantel allow shared processing for rendering on the timeline because each workstation is literally being fed it's assets via fiber from a huge rack of data servers with TB's of RAM and massive processing power behind them.
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old September 30th, 2009, 05:10 PM   #5
Inner Circle
 
Join Date: Dec 2005
Location: New York City
Posts: 2,615
Does GPU performance come up in the Activity monitor graphs?
__________________
William Hohauser - New York City
Edit/Camera/DCP production/Animation
William Hohauser is offline   Reply With Quote
Old September 30th, 2009, 05:35 PM   #6
Regular Crew
 
Join Date: Jan 2009
Location: Wellington, New Zealand
Posts: 82
Quote:
Originally Posted by Robert Lane View Post
Simple fix:

You've setup your cluster to work as if you're spreading the rendering jobs across a network allowing other computers to share your resources, that's not what you're after. You don't want to *share* you want to *Manage*.
Duh.... I should have seen that... Done it now :) Thank you for your help.

Interesting what you say about the H.264 codec only being able to encode on 1 core, I would almost expect someone to come up with one one day since everything is heading multi-core.... I was always under the impression that the software / hardware did all the work and that the codec was just the "settings" and specs that the software / hardware was working to..... I'm Guessing that its something to do with the way the codec processes the files - I've heard that some codecs do some funky stuff with the order of frames and sync with the audio and all sorts of stuff....
Ryan Hansen is offline   Reply With Quote
Old September 30th, 2009, 05:56 PM   #7
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
Quote:
Originally Posted by William Hohauser View Post
Does GPU performance come up in the Activity monitor graphs?
No the current AM in 10.6 only shows CPU, RAM and Disk monitors. There may be another utility that talks to OpenCL threads but I haven't seen one yet. But that also would require one of the newer ATI or nVidia cards that has GPU firmware to *talk* to OpenCL for it to do it's offloading.

What's interesting - and something that I can't speak to directly as to "why" - is that Premiere Pro, Avid and Media 100 already use built-in software threading to leverage any GPU the system is connected to (assuming the card has 256MB or more of VRAM). Edius does this as well but that's a PC-only app. Why FCP7 will only do this under OpenCL and only with these specific cards is a total mystery. Maybe the spec will open up as FCP7 matures and more GPU's become available.

This retail white-paper enclosed on OpenCL gives little insight into the how and why of the card-specific code and stuff on ADC is technically beyond my understanding since I'm not a code monkey, but it's definitely GPU-firmware specific; if the card wasn't written for it then OpenCL get's unused. Which is a real shame considering people who just purchased Mac Pro's prior to Snow Leopard and opted for the more expensive cards can't use OpenCL. It's another head-banger from Apple.
Attached Files
File Type: pdf OpenCL_TB_brief_20090903.pdf (338.0 KB, 417 views)
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old October 2nd, 2009, 05:02 AM   #8
Regular Crew
 
Join Date: Jan 2009
Location: Wellington, New Zealand
Posts: 82
Does anyone know of any 3rd party apps that monitor GPU? or even a llist of supported hardware? Ive got a nVidia 8800gt, but I can't find any definite answers....
Ryan Hansen is offline   Reply With Quote
Old October 2nd, 2009, 01:39 PM   #9
Go Go Godzilla
 
Join Date: Nov 2004
Location: Scottsdale, AZ USA
Posts: 2,739
Images: 15
Quote:
Originally Posted by Ryan Hansen View Post
Does anyone know of any 3rd party apps that monitor GPU? or even a llist of supported hardware? Ive got a nVidia 8800gt, but I can't find any definite answers....
You don't need any third-party apps, take a look at this thread and use the Developer Tools built into OSX:

http://www.dvinfo.net/forum/non-line...rformance.html
__________________
Producer/CEO
Reel Lane Films LLC
Robert Lane is offline   Reply With Quote
Old October 6th, 2009, 11:58 AM   #10
Regular Crew
 
Join Date: Feb 2004
Location: Seattle WA
Posts: 28
Ive been trying to set up my Mac Pro QMaster this way. I found this article that explains it with the share option selected. Any idea why?

Compressor - Enabling multiple Cores
Ryan McBride is offline   Reply With Quote
Old October 6th, 2009, 05:25 PM   #11
Regular Crew
 
Join Date: Jan 2009
Location: Wellington, New Zealand
Posts: 82
Yea that's the one I went off, I think shared is more for network rendering

Alot of good stuff on that site as well :)
Ryan Hansen is offline   Reply With Quote
Old October 7th, 2009, 12:02 PM   #12
Regular Crew
 
Join Date: Feb 2004
Location: Seattle WA
Posts: 28
Thats what I thought. I also came across this thread that shows a slightly different set up as well. Using Compressor with multiple cores - Digital Rebellion Blog
Ryan McBride is offline   Reply With Quote
Old October 19th, 2009, 10:38 AM   #13
Inner Circle
 
Join Date: Dec 2005
Location: New York City
Posts: 2,615
Reopening this thread for some more questions and observations.

1) Setting my 2.66 eight-core to 4 instances in Qmaster works great. All cores light up during MPEG2 and even h.264 rendering in Compressor. Actually the Activity Monitor shows 16 cores. However once the task shifts to ac3 audio files it drops down to 1 to 3 cores at 40% at most. Anyway, 1 hour of DV quality material took 25 minutes to render at the default high quality 2-pass DVD setting Apple provides. Setting Qmaster to anything over 4 instances didn't work and the computer would use the default render settings as if the Quickcluster hadn't been set up.

2) Sending a sequence with real-time (dark green "Full") effects to the Quickcluster through Compressor causes an immediate render failure although sending the same sequence to render under the "My Computer" option works perfectly. Creating a reference quicktime and bringing that in Compressor fixes this problem since the real-time areas are rendered at this point. The Share function has the same problem unless you render everything before choosing the Quickcluster. This seems to be a programming mistake that should be corrected.
__________________
William Hohauser - New York City
Edit/Camera/DCP production/Animation

Last edited by William Hohauser; October 19th, 2009 at 09:54 PM. Reason: Corrected info about the share function.
William Hohauser is offline   Reply With Quote
Old October 23rd, 2009, 01:56 AM   #14
Regular Crew
 
Join Date: Jan 2009
Location: Wellington, New Zealand
Posts: 82
I take it your using 10.6 nd fcs3?
Ryan Hansen is offline   Reply With Quote
Old October 23rd, 2009, 09:45 AM   #15
Inner Circle
 
Join Date: Dec 2005
Location: New York City
Posts: 2,615
Yes, the computer and programs are completely updated.... except for the Pro Apps update that just showed up today.

The end effect, even with the rendering issues, is that I used to run a real-time dub to a DVD recorder to save time on certain projects. Now I can make DVDs for clients in the computer in less time.
__________________
William Hohauser - New York City
Edit/Camera/DCP production/Animation
William Hohauser is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Apple / Mac Post Production Solutions > Final Cut Suite

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 10:15 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network