Compressing h264 videos for archiving at DVinfo.net

Go Back   DV Info Net > Canon EOS / MXF / AVCHD / HDV / DV Camera Systems > Canon EOS Crop Sensor for HD

Canon EOS Crop Sensor for HD
APS-C sensor cameras including the 80D, 70D, 7D Mk. II, 7D, EOS M and Rebel models for HD video recording.


Reply
 
Thread Tools Search this Thread
Old April 23rd, 2010, 07:24 PM   #1
Regular Crew
 
Join Date: Jan 2010
Location: new york city
Posts: 113
Compressing h264 videos for archiving

So I've been fooling around with crunching these quicktimes down for archiving, to see how small I can get them without any noticeable compression. "Noticeable" is subjective, of course, so I'm putting out what I've figured so far in case people want to chime in.

Now, I realize that the professional answer is "how can you compress your raw footage? That's insane!" And, yes, for certain uses, I would never do that- but the reality is that this stuff takes up SPACE and it fills drives up quickly. So until I can get a 400 TB drive for $10, I'm trying to find a way to crunch the less-than-vital footage I've shot and will be shooting. The fact is, a good chunk of HD video will not be blown up to a 50 foot screen or be used in high-end compositing and green screen effects, stuff where this compression starts to really matter.

Basically, I'm trying to find the perfect balance between quality and economy. If your comment is "don't compress," please provide a really great reason other than the obvious one, such as, compressing these files even slightly will cause Earthquakes in Uganda.

My limited results, using footage shot with the 50mm 1.4 canon lens, so far-

Original File - 47 mbs - looks great
Approx 22 mbs - looks great. The loss is so miniscule, it's work the 50% space saving.
Approx 12 mbs- looks really, really good. Most people would never notice. In fact, I only noticed after frame by frame comparisons. Playing the video, you'd never tell. Tempting at 1/4 of the space!
9 mbs - still looks great but I start to see the softness pretty easily. But it's such a nice softness!
6 mbs - looks amazing for the size, softer, but still impressive, that algorithm!
4 mbs - impressive for the size but it's getting a little too soft even for me.

I haven't tried something around 35 mbs, but that 22 mbs is so great, I'm curious what others think. Of course, other footage, lenses, lighting scenarios could provide much different results, this shot was using natural light indoors during daytime, the lens was pretty wide open but with a low ISO, shooting closeups going in and out of focus with the shallow depth of field.

Thoughts? Yea or Nea? It's amazing, to me, that even at 10% the already-compressed footage looks as good as it does. Okay, shoot!
David St. Juskow is offline   Reply With Quote
Old April 24th, 2010, 07:50 PM   #2
Major Player
 
Join Date: Apr 2010
Location: Chelmsford England
Posts: 287
I am glad you posted on this, I am mulling over exactly the same issue. I am shocked how quickly I fill up those 8GB cards now! I have a Canon HF100, which has a max bitrate of 17mb/s, and looks great to me.

The majority of my footage is personal/family stuff, so the image quality / file size trade off is less critical. For almost all of this type of stuff, 12-17 mb/s is a level I think I'm happy to archive with.

Incidentally, I am encoding with Mainconcept/mp4. I have tried to get Vegas Pro 9 to use the VFW x264 codec, but I can't get it to work. What are you using?
James Donnelly is offline   Reply With Quote
Old April 24th, 2010, 08:25 PM   #3
Regular Crew
 
Join Date: Jan 2010
Location: new york city
Posts: 113
I'm using Compressor, which comes with Final Cut Pro. I used to experiment a lot with encoding back, what, 10 years ago now, using Media Cleaner Pro, trying to do the same thing, but I hadn't had much need for that in the interim, so technology zipped by me. Now I'm back, and I've tried a couple of compression programs like Sorensen Squeeze, but Compressor is pretty good and, of course, is part of FCP so I haven't found need to try something else as of yet. I use it to convert the files to Apple ProRes for editing as well, it can add visual timecode to clips if people need to watch as dailies for notes, etc. It's pretty versatile.

Anyway, good to hear that bitrate seems to work for you, too. Curious about others...
David St. Juskow is offline   Reply With Quote
Old April 24th, 2010, 09:13 PM   #4
Regular Crew
 
Join Date: May 2007
Location: Boston MA
Posts: 167
RAW Files?

Since the original files are already compressed and there are no RAW files, I chose to keep the originals. I copy them onto DVD-Rs and store them as data files. One DVD-R accommodates ~4.3GB or roughly 13 minutes of 1080p video. That is the maximum size of an individual clip you can generate in these cameras anyway. If we consider one DVD-R costs $0.20 and accommodates 4GB, then 1TB of data storage costs ~$50.
Pedanes Bol is offline   Reply With Quote
Old April 24th, 2010, 10:48 PM   #5
Major Player
 
Join Date: Oct 2003
Location: Richardson, TX
Posts: 842
And with 1TB drives hardly more than that I just save them on a hard drive and skip the hassle of generating all those Dvds.
Bryan McCullough is offline   Reply With Quote
Old April 25th, 2010, 04:54 AM   #6
Major Player
 
Join Date: Apr 2010
Location: Chelmsford England
Posts: 287
A hard drive for archiving? Aren't you worried about if an when it will fail?

I have some mirrored external 500gb drives for storing work in progress, but I would need about 20 of these to store everything!

Waiting for the price of Blu-ray to come down. Quality media costs about 2 a piece bought in 10's, and the cheapest writer is about 150. So, 2 for 25Gb = 6 DVD-R's. A bit more money than DVD for a lot less hassle.

Hmm, seems I'm talking myself into another purchase!
James Donnelly is offline   Reply With Quote
Old April 25th, 2010, 05:16 AM   #7
Regular Crew
 
Join Date: Feb 2008
Location: Thessaloniki, Greece.
Posts: 179
Quote:
Originally Posted by Pedanes Bol View Post
Since the original files are already compressed and there are no RAW files, I chose to keep the originals. I copy them onto DVD-Rs and store them as data files. One DVD-R accommodates ~4.3GB or roughly 13 minutes of 1080p video. That is the maximum size of an individual clip you can generate in these cameras anyway. If we consider one DVD-R costs $0.20 and accommodates 4GB, then 1TB of data storage costs ~$50.
My experience all of those years as for archiving is to follow this one. DVD-R's and HDD at the same time. I keep all of my important projects like this. A copy at a HDD and then to a DVD-R or a Dual Layer. BD's in a while will make this even better.
__________________
My Films My Works FB
George Angeludis is offline   Reply With Quote
Old April 25th, 2010, 11:20 AM   #8
Major Player
 
Join Date: Oct 2003
Location: Richardson, TX
Posts: 842
Quote:
Originally Posted by James Donnelly View Post
A hard drive for archiving? Aren't you worried about if an when it will fail?
We keep backups of all our hard drives. I've had more burned disks go bad after a few years than I've had hard drives fails. No backup system is perfect, but I've found archiving to hard drives and having redundancy backups is the way that works best for us.

Now I can see eventually also having blu-ray backups of those drives since 20 disks per TB hard drive would be much easier to deal with than 200 disks.
Bryan McCullough is offline   Reply With Quote
Old April 25th, 2010, 01:17 PM   #9
Regular Crew
 
Join Date: Jan 2010
Location: new york city
Posts: 113
Forget the cost of media- I don't know where you keep all that storage. Maybe being a New Yorker has made me space-conscious, but that a LOT of whatevers {hard drives, dvd spindles, etc.) that will pile up fast using this camera, even if they're still functional in 20 years. I know there's no perfect system yet and we all pick our battles, but I guess that's why I'm trying to pack in as many clips onto the same amount of space, to have less crap in my home!
David St. Juskow is offline   Reply With Quote
Old April 25th, 2010, 02:12 PM   #10
Major Player
 
Join Date: Oct 2003
Location: Richardson, TX
Posts: 842
Yeah, I'm in Texas. Space ain't a concern. :)
Bryan McCullough is offline   Reply With Quote
Old April 25th, 2010, 04:36 PM   #11
Major Player
 
Join Date: May 2009
Location: Amsterdam, NL
Posts: 217
Quote:
Originally Posted by George Angeludis View Post
My experience all of those years as for archiving is to follow this one. DVD-R's and HDD at the same time. I keep all of my important projects like this. A copy at a HDD and then to a DVD-R or a Dual Layer. BD's in a while will make this even better.
my projects generally run over the 100gig mark, often a lot more.. dont you guys find the same??

i personally wouldnt be a fan of sitting at home all day burning 30 discs! So far I'm archiving everything on cheap(er) and slow usb 2.0 drives with 2 copies of all my active or semi active work..
Manus Sweeney is offline   Reply With Quote
Old May 1st, 2010, 09:16 PM   #12
Regular Crew
 
Join Date: Mar 2009
Location: United States
Posts: 81
After a month of shooting interviews and my little movie review show, I think I'm up to about 200 gigs. Of course that's in Pro Res after log and transfer or transcoding with Streamclip. And that's everything, finished shows have been running about 5 gigs or so depending on show segment length (max around 9 minutes before a short commercial break).

I've got a cheap usb 500 gig drive that I'm going to fill up and then I'm planning on buying a new one every few months. And a bud of mine told me to keep the box that the drive comes in, because you put the drive in the box and store it when done. Even use the plastic bag that's in the box when storing it.

I've got stacks of dual layer dvds that I've archived shows to over the years. And that takes so much time. Right now, hard drive is the only real solution. I spoke with a guy at CNN who said that they had been using some kind of tape system until recently. I suspect that Blu-Ray will be great when the price comes down and speed increases for burning. There's just something comforting about having a hard copy on disc.
Jonathan W. Hickman is offline   Reply With Quote
Old May 2nd, 2010, 02:46 PM   #13
Regular Crew
 
Join Date: Nov 2006
Location: Sacramento, CA
Posts: 177
Quote:
Originally Posted by Pedanes Bol View Post
Since the original files are already compressed and there are no RAW files, I chose to keep the originals. I copy them onto DVD-Rs and store them as data files. One DVD-R accommodates ~4.3GB or roughly 13 minutes of 1080p video. That is the maximum size of an individual clip you can generate in these cameras anyway. If we consider one DVD-R costs $0.20 and accommodates 4GB, then 1TB of data storage costs ~$50.
I've used this method for single project archive using WinRAR to make easily burned DVD5's.

My primary backup is my DroboPro. Not super quick, but dual redundancy is nice.
Being able to edit directly off the DroboPro is possible in a pinch, but not for day-to-day work.
__________________
Rusty
Rusty Rogers | Films
Rusty Rogers is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Canon EOS / MXF / AVCHD / HDV / DV Camera Systems > Canon EOS Crop Sensor for HD

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 01:02 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network