Documentary Archive solution needed! at DVinfo.net

Go Back   DV Info Net > Special Interest Areas > Documentary Techniques

Documentary Techniques
-- Discuss issues facing documentary production.


Reply
 
Thread Tools Search this Thread
Old September 23rd, 2005, 04:08 PM   #1
New Boot
 
Join Date: Sep 2005
Posts: 16
Documentary Archive solution needed!

HELP…please!
I have been unsuccessfully thrashing around in the dark for the past few months trying to come up with the right solution to my production problem:
I am scheduled to begin shooting a documentary series in July next year. The entire series will be shot on and around a small 33 foot boat sailing through the northern and southern Pacific.
As you are probably aware, pressure is being put on producers by broadcasters like Discovery, Travel and National Geographic to shoot on film or HD. The conditions under which I will be shooting this series preclude me from using a DVW or other full size type EFP unit and film is out of the question. I need the flexibility and maneuverability of a smaller "handicam. So it's down to Sony or Panasonic…with Canon a possibility.
Here's where it gets difficult… I don't believe that Sony's HDV 1080i format will compare with Panasonic's 1080/30p (or even 720/60p) format. Especially given the extreme camera movement and complex in-frame movement some of the wind and sea conditions will cause. I need clean pictures with the ability to do slo-mo at times.
The big problem with Panasonic seems to be archiving the copious amounts of material I will be shooting. Don't forget that I will be on a small boat with limited space and a most hostile environment for electronics. From this perspective Sony and Canon definitely have the edge by being able to record onto tape, but how does the HDV codec stack up to DVCPro HD?
If I was able to select (pre-edit) only usable the usable shots and dump the NFG takes, this should reduce the amount of storage space I would require. Would I need an editing platform, or is there some way to do this without going through an editor?
Any input or suggestions or ideas will be happily received!
Cheers,
Bruce
Bruce Quayle is offline   Reply With Quote
Old September 23rd, 2005, 04:34 PM   #2
Regular Crew
 
Join Date: Jul 2005
Location: Uruguay
Posts: 57
Got electricity in the boat?
Federico Martini Crotti is offline   Reply With Quote
Old September 23rd, 2005, 04:50 PM   #3
Major Player
 
Join Date: May 2003
Location: Phoenix, AZ
Posts: 327
Until Blu-Ray DVD's become a commodity and fall in price like DVD-R's have, the best bang for the buck in archiving has to be external firewire or USB2 hard drives.

With clients like Discovery and National Geographic, I think you're right in leaning towards the HVX-200. The HDV codec is going to be much less forgiving in post, when you try to color-correct all the footage to give it that highly polished look that both those networks demand. And the HVX is the ONLY handycam-size camcorder that will allow you to do TRUE slo-mo in the camera.

Have you considered shooting 720/24P, then using 720/48P or /60P for slo-mo? That's the combination that will most closely emulate film, while also allowing slo-mo.

It sounds like what you'll be wanting is to use the 200 with a Firestore recorder, then once or twice a day dump footage through a laptop onto an external hard drive. Will you have access to at least 1 A/C outlet on the boat? You could run the laptop off batteries, you'll just need 1 outlet for the external drive, and then only when you're archiving the footage, at filecopy speeds (several X real-time).

Of course, the possibility was also mentioned of dumping footage off P2 cards directly to a USB2 external drive. I'm assuming, being a docu, that you're going to want longer run times than P2 will provide, therefore the Firestore. I wonder if you could dump from a Firestore, through the camera to external USB2 - no laptop needed! Probably not, but it's worth mentioning.

At less than $1/GB, hard drives are where it's at, at least for the next 6-12 months. I might also want to take the extra step of then cloning the contents of each drive to a second backup drive after each days shooting.
Scott Anderson is offline   Reply With Quote
Old September 23rd, 2005, 05:17 PM   #4
Barry Wan Kenobi
 
Join Date: Jul 2003
Location: North Carolina
Posts: 3,863
Quote:
Originally Posted by Bruce Quayle
I was able to select (pre-edit) only usable the usable shots and dump the NFG takes, this should reduce the amount of storage space I would require. Would I need an editing platform, or is there some way to do this without going through an editor?
You can do that in-camera, instantly. The P2 system lets you review clips and delete bad ones, and protect good ones. Using that system you can keep your usage of the cards quite efficient.

Another thing to consider would be the newly-announced FireStore... at 100gb, it should provide for over four hours of continuous 720/24p recording (note: that's not verified yet, but I'm basing that off the hopefully-reasonable assumption that the drive will drop 'duplicate frames' and only store the active frames). I'm not sure, but I'm betting that you'd be able to delete bad clips from the Firestore in the field as well.
Barry Green is offline   Reply With Quote
Old September 23rd, 2005, 05:28 PM   #5
Regular Crew
 
Join Date: Jul 2005
Location: Uruguay
Posts: 57
I agree with the above. May I add that, if you have enough room, it would be real handy to have an editor with you who can take the FireStore or P2s back and forth from camera to laptop/hardrives. And, in the meanwhile, this person can start cutting and throwing away unwanted footage, clearing space in the hardisks.
Federico Martini Crotti is offline   Reply With Quote
Old September 23rd, 2005, 06:19 PM   #6
New Boot
 
Join Date: Sep 2005
Posts: 16
Quote:
Originally Posted by Scott Anderson
Have you considered shooting 720/24P, then using 720/48P or /60P for slo-mo? That's the combination that will most closely emulate film, while also allowing slo-mo.
Just concerned about the loss in quality when the time comes to transfer up to 1080 from 720. Also thought that the occasional frenetic movement of camera and frame content would be better handled by 30P. Could I not mix the two formats? ie: 1080/30P for majority of recording while 720/60P for in-camera slo-mo?

Quote:
Originally Posted by Scott Anderson
It sounds like what you'll be wanting is to use the 200 with a Firestore recorder, then once or twice a day dump footage through a laptop onto an external hard drive. Will you have access to at least 1 A/C outlet on the boat?
Power will be available via an inverter from a large battery bank so laptop and external hard drives are certainly possible. My concern is the number of hard drives I would need to archive +/- 100 hours of HD material! This is a series I'm shooting, not a single one-hour documentary. Sorry if I didn't make that point earlier.

Quote:
Originally Posted by Barry Green
You can do that in-camera, instantly. The P2 system lets you review clips and delete bad ones, and protect good ones. Using that system you can keep your usage of the cards quite efficient......Another thing to consider would be the newly-announced FireStore......I'm not sure, but I'm betting that you'd be able to delete bad clips from the Firestore in the field as well.
That would work well. From an editing point of view it would certainly streamline the process - something we should be looking at with this new type of technology. (I'm an old f@rt - even remember working with 2" videotape!;)

Quote:
Originally Posted by Federico Martini
...if you have enough room, it would be real handy to have an editor with you who can take the FireStore or P2s back and forth from camera to laptop/hardrives.
No space...no budget...no need. I will be "out there" for a year or two and will have the time to do all the grunt work myself. My only issue would be when I'm shooting ashore and have limited storage and carrying capacity - no SUVs I'm afraid, just Shanks' pony!

Thank you all for your input. Fortunately I have a few months before I leave next July.

Cheers,
Bruce
Bruce Quayle is offline   Reply With Quote
Old September 23rd, 2005, 07:36 PM   #7
Major Player
 
Join Date: May 2003
Location: Phoenix, AZ
Posts: 327
It's been discussed many times that folks would shoot 1080 for the bulk of the production, and only use 720 for the off-speed shots. I have two concerns, neither of which will be addressed until we actually see the camera.

1. Since the CCD's are almost certainly not full 1080 native (which is actually a good thing), there is some sort of interpolation going on to get a 1080p image, most probably having to do with pixel shift. How good is it? who knows? Perhaps there is NO discernable benefit to shooting 1080p, as opposed to 720p with up-rezzing.

2. 720 and 1080 may not intercut seamlessly, even from the same camera, because of the above.

Panasonic has been pretty bold in talking up this camera as a b-cam to a Varicam shoot. Have you seen Varicam footage? IMHO, it's plenty of quality for a Discovery HD program.

Also, are there any PAL markets for this program? If so, 24p is the only framerate that can be used in both NTSC and PAL territories. At 30p, you're pretty much locked into NTSC. I guess it all depends on what the broadcaster is looking for.

Also, I don't think that 15-30 external HDD's (by Barry's calc., with overhead - 15 w/o backup, 30 with backup) at a total cost of $7-15,000 is extreme at all for storing 100 hours of HDCam. In fact, that's as cheap as it's been, EVER.
Scott Anderson is offline   Reply With Quote
Old September 23rd, 2005, 11:24 PM   #8
Regular Crew
 
Join Date: Apr 2005
Location: Singapore
Posts: 97
Seriously, go for tape.

If you're going to be out on a boat for a year, do not mess around with hard drives, lap-tops, etc. It's just begging for trouble. The HVX-200 while being a fine camera (on paper, no one's used it yet), doesn't look like a good choice for documentary work where you have hours and hours of footage.

The workflow is not there yet for long form documentary work, requiring the archiving of so much material. However perhaps by your shoot next july there will be a solution that makes sense.

Another good thing about mini-dv tape is that it's ubiquitous. I once bought a 5 pack of sony dv tapes literally ON the mekong river, from a guy in a longtail boat. If you're travelling around from port to port, there's always somewhere you can buy DV tape. An extra USB hard drive may be more difficult to find.

I will agree that the HVX does have a much better codec than HDV cams, but will that alone result in a better image? The HVX isn't out yet, so no one is sure about the lens and CCDs. Codec is not the be-all and end-all factor in the image.

If the choice is a slightly better image, pitted against the chance of running out of HD space, or heaven forbid catastrophic hard drive failure. It's infinitely better to have a slightly more compressed image on tape, than no image at all.

Still by the time you go on your shoot, the situation may have changed. July is still a long way off. So my suggestion is don't buy now, wait till all the cameras (Sony, JVC, Panasonic, and Cannon), have been out for a while and time tested in the field before you make your choice. This board is the perfect place. If there is any little flaw in a camera, rest assured you will hear about it here.
John M Burkhart is offline   Reply With Quote
Old September 24th, 2005, 03:09 AM   #9
Regular Crew
 
Join Date: Jul 2005
Posts: 131
You may find this BBC Guide to HD helpful. It is still in draft, I believe, and contains some errors. It is not yet supposed to be available to the public but someone found a way in. It provides advice for those new to HD production and direction. Link:
http://www.bbc.co.uk/guidelines/deli...Bookv01_00.pdf
David Andrews is offline   Reply With Quote
Old September 24th, 2005, 12:10 PM   #10
Trustee
 
Join Date: May 2003
Location: Tulsa, OK
Posts: 1,689
I agree 100% with John. You cant do selects on a documentary until you are done shooting. Also, you need to have everything archived and backed up. Do you really want ALL your shots on a couple hard drives in an environment that is uncontrolled? Right now there is no solution but tape for docs of this type and the only affordable and portable HD solution is HDV. I am not a fan of HDV but it would work best for you if you have to go HD.

This is the big problem with both the HVX and the XLH1. To get the best HD results you have to spend a lot more money and it becomes less portable. I personally wont invest in HD gear until there is a more solid and affordable collection media.


ash =o)
Ash Greyson is offline   Reply With Quote
Old September 24th, 2005, 09:53 PM   #11
Major Player
 
Join Date: May 2003
Location: Phoenix, AZ
Posts: 327
I would be very careful to ask the prospective networks if they will accept HDV shot material. The choice of networks leads me to the opinion that the highest level of quality is desirable. I would bet the HVX-200 @ 720/24 (or30)p, sight unseen, over the Sony Z-1 @ 1080/30i any day of the week. The places where the HDV codecs break apart are quck movement, like unpredictable documentary moves on the pitching of a boat, and subtle gradients, like wide expanses of sky.

And while it's true that you can get miniDV tapes anywhere in a pinch, I don't think that's going to be an issue here. I also wouldn't just slap any old off-brand miniDV tapes into the Sony Z1 and expect them to be ultra-reliable, especially if you mix tape stocks. I've had a handful of brand-new DV tapes fail on me over the years, and I've only had one IDE hard drive go south (with warning enough to get the data off, BTW). I wouldn't get too dogmatic about tape being the safest and best storage soulution out there.

Let's say you have the ability to store your external drives in a nice, padded Pelican case, you treat them with reasonable care, keeping them away from undue moisture and shock, and you go the extra step to make a redundant backup on a second drive. In that case, I would say that you have an archival system that rivals, if not exceeds the reliability of MiniDV tape.

And, you get the DVCProHD codec, which will look better right off the camera, and hold up MUCH better to the inevitable post-processing. I've never been much of a Panasonic guy, but this camera is going to be unreal. And for everyone that says the workflow isn't there yet, I beg to differ. The workflow is laid out, right in front of you, and it's the same workflow we've all been using for years with our NLE's. You just have to start thinking about your footage as data, not as tapes.

Who knows, there may be a major gotcha that will surface once this camera is shipped. Maybe the Firestore won't even be available by the time you need to gear up, which would put a big crimp in your workflow. I just wouldn't commit to a particular camera until you have to. In the meantime, why not rent a Z-1 and go shoot some test footage? That will tell you pretty quick if the image is suitable for your purposes. That way, when the HVX-200 ships, you'll already have something to compare to.
Scott Anderson is offline   Reply With Quote
Old September 25th, 2005, 02:25 AM   #12
Major Player
 
Join Date: Mar 2003
Location: Portland, Oregon
Posts: 605
I think everyone is way off on the archive solution here. Archiving a year's worth of footage onto HDD's that've been sitting in a boat is nuts. The footage should only be archived onto DLT--this is by far the most reliable solution. If you check out Panasonic's white paper on the HVX, DLT is their recommended archive solution. I'd say shoot with the Firestore (bring a couple backups) and archive to DLT.

Peter
Peter Richardson is offline   Reply With Quote
Old September 25th, 2005, 06:55 PM   #13
New Boot
 
Join Date: Sep 2005
Posts: 16
Quote:
Originally Posted by Scott Anderson
Since the CCD's are almost certainly not full 1080 native (which is actually a good thing)
Why is this a good thing Scott?

Quote:
Originally Posted by Scott Anderson
Also, are there any PAL markets for this program? If so, 24p is the only framerate that can be used in both NTSC and PAL territories. At 30p, you're pretty much locked into NTSC. I guess it all depends on what the broadcaster is looking for.
We are looking at both North American and European broadcasters so this is iindeed a factor. Wouldn't 1080i be another option to 24p? I'm just concerned that at times the movement within frame may be to frenetic for 24p.

Quote:
Originally Posted by John M Burkhart
Seriously, go for tape.
If you're going to be out on a boat for a year, do not mess around with hard drives, lap-tops, etc. It's just begging for trouble.
This is my big concern. I got back from a circumnavigation 4 years back and that environment is extremely hostile to electronics...heck, I was beginning to corrode! But having said that, I still like what the HVX200 offers from a quality standpoint, provided that it delivers.

Quote:
Originally Posted by Ash Greyson
I agree 100% with John. You cant do selects on a documentary until you are done shooting. Also, you need to have everything archived and backed up.
I would never consider selecting my shots at the shooting stage...I plan only to delete those which are unusable. You are certainly correct about backing up any electronically stored data which would certainly increase the cost.

Quote:
Originally Posted by David Andrews
You may find this BBC Guide to HD helpful. It is still in draft, I believe, and contains some errors. It is not yet supposed to be available to the public but someone found a way in. It provides advice for those new to HD production and direction. Link:
http://www.bbc.co.uk/guidelines/deli...Bookv01_00.pdf
Brilliant! Thanks David. This publication was very informative. Used to work for the Beeb many, many years ago and it was interesting to get their take on HD.

Quote:
Originally Posted by Peter Richardson
I think everyone is way off on the archive solution here. Archiving a year's worth of footage onto HDD's that've been sitting in a boat is nuts. The footage should only be archived onto DLT--this is by far the most reliable solution. If you check out Panasonic's white paper on the HVX, DLT is their recommended archive solution. I'd say shoot with the Firestore (bring a couple backups) and archive to DLT.
I've put out a call to DLT Solutions Inc. and am hoping that they will get back to me with an answer. I'll let you all know.

Thanks so much for all the help guys - a great forum.
Bruce
Bruce Quayle is offline   Reply With Quote
Old September 25th, 2005, 08:46 PM   #14
Major Player
 
Join Date: May 2003
Location: Phoenix, AZ
Posts: 327
Quote:
Originally Posted by Bruce Quayle
Why is this a good thing Scott?
Because with 1/3" CCD's having native 1080 pixels, the result would be poor dynamic range and noise. Graeme Nattress has explained this much better than I ever could. In fact, if you want technical explanations, search for every post Graeme has ever made.

I was lucky enough to have met Graeme at NAB this year, and he's one of the most down to earth, self-effacing guys you could ever hope to meet. He also happens to be a codec/compression super genius. His plugins are pretty darn cool, too. Google him - you'll see. I would trust his opinion implicitly.

Here's one of the posts I was thinking of:
www.dvinfo.net/conf/showthread.php?t=48575
Scott Anderson is offline   Reply With Quote
Old September 25th, 2005, 10:01 PM   #15
Major Player
 
Join Date: May 2003
Location: Akron, Ohio
Posts: 496
Quote:
Originally Posted by Scott Anderson
Because with 1/3" CCD's having native 1080 pixels, the result would be poor dynamic range and noise. Graeme Nattress has explained this much better than I ever could. In fact, if you want technical explanations, search for every post Graeme has ever made.

I was lucky enough to have met Graeme at NAB this year, and he's one of the most down to earth, self-effacing guys you could ever hope to meet. He also happens to be a codec/compression super genius. His plugins are pretty darn cool, too. Google him - you'll see. I would trust his opinion implicitly.

Here's one of the posts I was thinking of:
www.dvinfo.net/conf/showthread.php?t=48575


People said the same thing about the smaller Nikon D2X sensor before it came out. Despite what people thought, the camera is very clean. Technology has come along way in the last few years. I expect the HVX200 to have the same sensitivity and dynamic range as the DVX100a(or close to it).

John
John DeLuca is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Special Interest Areas > Documentary Techniques

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 03:16 PM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network