Best compression for internet transfer -> editing at DVinfo.net

David Lei · June 26th, 2008, 12:15 PM

Hello! First post! I am working on a project in which I need to take delivery of ~3 hours of HDV footage and edit it in FCP. The ideal would, of course, be to get the original tape. However, that would be a bit inconvenient given the circumstances. Transferring, say, ProRes would be impractical. What compression should I use to optimize file size <> quality for editing purposes? I know I'll have to make sacrifices on quality ... transferring a few (3 or so) gigabytes is within reason. Finally, the final product would mainly be streamed online, possibly on Vimeo. Thanks in advance for any advice you can offer.

Mike Barber · June 26th, 2008, 12:33 PM

Quote:

Originally Posted by David Lei

I need to take delivery of ~3 hours of HDV footage and edit it in FCP. […] What compression should I use to optimize file size <> quality for editing purposes? I know I'll have to make sacrifices on quality ... transferring a few (3 or so) gigabytes is within reason.

If I am reading this correctly, you want to compress 3 hours of HDV down to 3 GBs? Is that the right way to summarize?

Glenn Fisher · June 26th, 2008, 08:06 PM

Do you have visualhub? If you do, then you can set it to compress everything to a format of your choice (I don't know all that much, but I would probably go with the "iTunes" tab, set to "All Devices", with "H.264 Encoding" checked, and set to the "High" quality), then you can go into the advanced settings and click "Fit each video in: __ MB" and then you can specify the size you would like for each individual video. This would work really well if all of the videos were of the same length, because if you had 3 one-hour files, then you could just take whatever file size you would like to transfer over the internet and divide it by 3, and each file should (in theory) be compressed the same amount. If your files aren't all of the same length, then you might want to run files of similar length through visualhub to ensure that the amount of compression is roughly the same on each one..although that could get complicated!

If you don't have visualhub, well, I would personally highly recommend it. I love visualhub, because it's become so useful to me, especially since I run a video podcast for my local school, and it's so easy just to drag and drop a video right onto visualhub and have it convert everything for me to an ipod compatible format! If you're interested, you can pick up visualhub from here: http://www.techspansion.com/visualhub/

I'm not sure if there is any other easy way. I would be curious to see if anybody else out there has any good recommendations on how to do this through compressor.

Oh, the idea just hit me, I don't think compressor will allow you to set an output file size for compressed media, but I do believe that is tells you an estimation of how big the file will be. If you experiment a little you could set up your own preset and just manually add it to all of the videos in compressor, then do a little math by adding up the estimated file sizes and see if it's reasonable to send over the internet. Again, if you were working in compressor, I would personally recommend using an H.264 codec just because they seem very effective in keeping a high amount of quality with a comparatively low file size.

Well, good luck, and I hope that might help!

Mike Barber · June 26th, 2008, 09:46 PM

The problem with H.264 is that it is a delivery codec and is not meant for editing.

David, tell you the truth, you either need to get the tapes (have them makes dups and send those), get them to send you the footage on a FW drive, or rethink this whole thing. Transferring that much footage over the web is crazy. In order to make that practical, you'll need to down-rez the picture as well as todd tons of picture and sound quality out the window.

Just the audio data alone must be almost 9 Gigs (3 hours of 16/48k audio)!

David Lei · June 27th, 2008, 07:53 AM

Yeah, this isn't an ideal situation. I'll see if getting the tapes is possible. Even though H.264 wasn't meant for this, couldn't I do it anyway? I could convert it back to ProRes to work with. I'd lose some quality but the question is how much?

Mike Barber · June 27th, 2008, 08:10 AM

Quote:

Originally Posted by David Lei

I'll see if getting the tapes is possible.

That really is your best route. If they are serious about the project being done right and looking good, then they should be providing you with the footage in a manner that ensures the best possible outcome.

Quote:

Originally Posted by David Lei

Even though H.264 wasn't meant for this, couldn't I do it anyway? I could convert it back to ProRes to work with. I'd lose some quality but the question is how much?

Well, I don't for for certain. I think it maybe can be done, but like I said before: you're going to have to shrink it down -- and I mean tiny -- and throw out so much data (read: quality) that it isn't going to be worth it -- and we're talking more than just "some" quality, I think we're talking about a significant amount. Converting back to ProRes isn't going to restore what was tossed out.

Dave Christensen · July 6th, 2008, 01:58 PM

Just another quick consideration to throw into the mix: h264 tends to be very processor-intensive particularly with HD. What this means is once you start editing multiple h264 streams, they're probably getting decoded and rendered into your intermediate codec in the timeline. All this to say, you'll need quite a beefy system to keep up with this if you want to keep editing well in real-time. So, to reiterate, h264 won't be the best option. :)

David Lei · July 6th, 2008, 03:04 PM

Thanks for the tips. I ended up taking Mike's advice and got my hands on the tape. Too bad, actually, I was kind of curious if it could work decently enough. I will have to test it out for kicks when I get the chance.

Dave Christensen · July 6th, 2008, 03:09 PM

David - Keep the post updated if you figure out another workflow in the future. I'm actually researching the same issue right now, where all my HDV footage is coming from literally the other side of the world and needs to be delivered and edited locally. I'd love to hear any creative solutions that turn up. I wonder if the big wigs in Hollywood just overnight tons of HDDs via Fedex for their dailies, or always do the dailies on location.

June 26th, 2008, 12:15 PM	#1
David Lei New Boot Join Date: Jun 2008 Location: Brooklyn, NY Posts: 9	Best compression for internet transfer -> editing Hello! First post! I am working on a project in which I need to take delivery of ~3 hours of HDV footage and edit it in FCP. The ideal would, of course, be to get the original tape. However, that would be a bit inconvenient given the circumstances. Transferring, say, ProRes would be impractical. What compression should I use to optimize file size <> quality for editing purposes? I know I'll have to make sacrifices on quality ... transferring a few (3 or so) gigabytes is within reason. Finally, the final product would mainly be streamed online, possibly on Vimeo. Thanks in advance for any advice you can offer.

June 26th, 2008, 09:46 PM	#4
Mike Barber Major Player Join Date: May 2006 Location: Toronto ON Canada Posts: 731	The problem with H.264 is that it is a delivery codec and is not meant for editing. David, tell you the truth, you either need to get the tapes (have them makes dups and send those), get them to send you the footage on a FW drive, or rethink this whole thing. Transferring that much footage over the web is crazy. In order to make that practical, you'll need to down-rez the picture as well as todd tons of picture and sound quality out the window. Just the audio data alone must be almost 9 Gigs (3 hours of 16/48k audio)! __________________ Mike Barber "I'm laughing to stop myself from screaming."

July 6th, 2008, 01:58 PM	#7
Dave Christensen Regular Crew Join Date: Dec 2006 Location: Tempe, AZ, USA Posts: 32	Just another quick consideration to throw into the mix: h264 tends to be very processor-intensive particularly with HD. What this means is once you start editing multiple h264 streams, they're probably getting decoded and rendered into your intermediate codec in the timeline. All this to say, you'll need quite a beefy system to keep up with this if you want to keep editing well in real-time. So, to reiterate, h264 won't be the best option. :) __________________ Final Cut FTW

July 6th, 2008, 03:09 PM	#9
Dave Christensen Regular Crew Join Date: Dec 2006 Location: Tempe, AZ, USA Posts: 32	David - Keep the post updated if you figure out another workflow in the future. I'm actually researching the same issue right now, where all my HDV footage is coming from literally the other side of the world and needs to be delivered and edited locally. I'd love to hear any creative solutions that turn up. I wonder if the big wigs in Hollywood just overnight tons of HDDs via Fedex for their dailies, or always do the dailies on location. __________________ Final Cut FTW

June 26th, 2008, 08:06 PM	#3
Glenn Fisher Regular Crew Join Date: Dec 2007 Location: Bridgewater, NJ Posts: 65	Do you have visualhub? If you do, then you can set it to compress everything to a format of your choice (I don't know all that much, but I would probably go with the "iTunes" tab, set to "All Devices", with "H.264 Encoding" checked, and set to the "High" quality), then you can go into the advanced settings and click "Fit each video in: __ MB" and then you can specify the size you would like for each individual video. This would work really well if all of the videos were of the same length, because if you had 3 one-hour files, then you could just take whatever file size you would like to transfer over the internet and divide it by 3, and each file should (in theory) be compressed the same amount. If your files aren't all of the same length, then you might want to run files of similar length through visualhub to ensure that the amount of compression is roughly the same on each one..although that could get complicated! If you don't have visualhub, well, I would personally highly recommend it. I love visualhub, because it's become so useful to me, especially since I run a video podcast for my local school, and it's so easy just to drag and drop a video right onto visualhub and have it convert everything for me to an ipod compatible format! If you're interested, you can pick up visualhub from here: http://www.techspansion.com/visualhub/ I'm not sure if there is any other easy way. I would be curious to see if anybody else out there has any good recommendations on how to do this through compressor. Oh, the idea just hit me, I don't think compressor will allow you to set an output file size for compressed media, but I do believe that is tells you an estimation of how big the file will be. If you experiment a little you could set up your own preset and just manually add it to all of the videos in compressor, then do a little math by adding up the estimated file sizes and see if it's reasonable to send over the internet. Again, if you were working in compressor, I would personally recommend using an H.264 codec just because they seem very effective in keeping a high amount of quality with a comparatively low file size. Well, good luck, and I hope that might help!

June 27th, 2008, 07:53 AM	#5
David Lei New Boot Join Date: Jun 2008 Location: Brooklyn, NY Posts: 9	Yeah, this isn't an ideal situation. I'll see if getting the tapes is possible. Even though H.264 wasn't meant for this, couldn't I do it anyway? I could convert it back to ProRes to work with. I'd lose some quality but the question is how much?

July 6th, 2008, 03:04 PM	#8
David Lei New Boot Join Date: Jun 2008 Location: Brooklyn, NY Posts: 9	Thanks for the tips. I ended up taking Mike's advice and got my hands on the tape. Too bad, actually, I was kind of curious if it could work decently enough. I will have to test it out for kicks when I get the chance.