Documentary Archive solution needed!
 

HELP…please!
I have been unsuccessfully thrashing around in the dark for the past few months trying to come up with the right solution to my production problem:
I am scheduled to begin shooting a documentary series in July next year. The entire series will be shot on and around a small 33 foot boat sailing through the northern and southern Pacific.
As you are probably aware, pressure is being put on producers by broadcasters like Discovery, Travel and National Geographic to shoot on film or HD. The conditions under which I will be shooting this series preclude me from using a DVW or other full size type EFP unit and film is out of the question. I need the flexibility and maneuverability of a smaller "handicam. So it's down to Sony or Panasonic…with Canon a possibility.
Here's where it gets difficult… I don't believe that Sony's HDV 1080i format will compare with Panasonic's 1080/30p (or even 720/60p) format. Especially given the extreme camera movement and complex in-frame movement some of the wind and sea conditions will cause. I need clean pictures with the ability to do slo-mo at times.
The big problem with Panasonic seems to be archiving the copious amounts of material I will be shooting. Don't forget that I will be on a small boat with limited space and a most hostile environment for electronics. From this perspective Sony and Canon definitely have the edge by being able to record onto tape, but how does the HDV codec stack up to DVCPro HD?
If I was able to select (pre-edit) only usable the usable shots and dump the NFG takes, this should reduce the amount of storage space I would require. Would I need an editing platform, or is there some way to do this without going through an editor?
Any input or suggestions or ideas will be happily received!
Cheers,
Bruce

Got electricity in the boat?

Until Blu-Ray DVD's become a commodity and fall in price like DVD-R's have, the best bang for the buck in archiving has to be external firewire or USB2 hard drives.

With clients like Discovery and National Geographic, I think you're right in leaning towards the HVX-200. The HDV codec is going to be much less forgiving in post, when you try to color-correct all the footage to give it that highly polished look that both those networks demand. And the HVX is the ONLY handycam-size camcorder that will allow you to do TRUE slo-mo in the camera.

Have you considered shooting 720/24P, then using 720/48P or /60P for slo-mo? That's the combination that will most closely emulate film, while also allowing slo-mo.

It sounds like what you'll be wanting is to use the 200 with a Firestore recorder, then once or twice a day dump footage through a laptop onto an external hard drive. Will you have access to at least 1 A/C outlet on the boat? You could run the laptop off batteries, you'll just need 1 outlet for the external drive, and then only when you're archiving the footage, at filecopy speeds (several X real-time).

Of course, the possibility was also mentioned of dumping footage off P2 cards directly to a USB2 external drive. I'm assuming, being a docu, that you're going to want longer run times than P2 will provide, therefore the Firestore. I wonder if you could dump from a Firestore, through the camera to external USB2 - no laptop needed! Probably not, but it's worth mentioning.

At less than $1/GB, hard drives are where it's at, at least for the next 6-12 months. I might also want to take the extra step of then cloning the contents of each drive to a second backup drive after each days shooting.

You can do that in-camera, instantly. The P2 system lets you review clips and delete bad ones, and protect good ones. Using that system you can keep your usage of the cards quite efficient.

Another thing to consider would be the newly-announced FireStore... at 100gb, it should provide for over four hours of continuous 720/24p recording (note: that's not verified yet, but I'm basing that off the hopefully-reasonable assumption that the drive will drop 'duplicate frames' and only store the active frames). I'm not sure, but I'm betting that you'd be able to delete bad clips from the Firestore in the field as well.

I agree with the above. May I add that, if you have enough room, it would be real handy to have an editor with you who can take the FireStore or P2s back and forth from camera to laptop/hardrives. And, in the meanwhile, this person can start cutting and throwing away unwanted footage, clearing space in the hardisks.

Just concerned about the loss in quality when the time comes to transfer up to 1080 from 720. Also thought that the occasional frenetic movement of camera and frame content would be better handled by 30P. Could I not mix the two formats? ie: 1080/30P for majority of recording while 720/60P for in-camera slo-mo?

Power will be available via an inverter from a large battery bank so laptop and external hard drives are certainly possible. My concern is the number of hard drives I would need to archive +/- 100 hours of HD material! This is a series I'm shooting, not a single one-hour documentary. Sorry if I didn't make that point earlier.

That would work well. From an editing point of view it would certainly streamline the process - something we should be looking at with this new type of technology. (I'm an old f@rt - even remember working with 2" videotape!;)

No space...no budget...no need. I will be "out there" for a year or two and will have the time to do all the grunt work myself. My only issue would be when I'm shooting ashore and have limited storage and carrying capacity - no SUVs I'm afraid, just Shanks' pony!

Thank you all for your input. Fortunately I have a few months before I leave next July.

Cheers,
Bruce

It's been discussed many times that folks would shoot 1080 for the bulk of the production, and only use 720 for the off-speed shots. I have two concerns, neither of which will be addressed until we actually see the camera.

1. Since the CCD's are almost certainly not full 1080 native (which is actually a good thing), there is some sort of interpolation going on to get a 1080p image, most probably having to do with pixel shift. How good is it? who knows? Perhaps there is NO discernable benefit to shooting 1080p, as opposed to 720p with up-rezzing.

2. 720 and 1080 may not intercut seamlessly, even from the same camera, because of the above.

Panasonic has been pretty bold in talking up this camera as a b-cam to a Varicam shoot. Have you seen Varicam footage? IMHO, it's plenty of quality for a Discovery HD program.

Also, are there any PAL markets for this program? If so, 24p is the only framerate that can be used in both NTSC and PAL territories. At 30p, you're pretty much locked into NTSC. I guess it all depends on what the broadcaster is looking for.

Also, I don't think that 15-30 external HDD's (by Barry's calc., with overhead - 15 w/o backup, 30 with backup) at a total cost of $7-15,000 is extreme at all for storing 100 hours of HDCam. In fact, that's as cheap as it's been, EVER.

Seriously, go for tape.

If you're going to be out on a boat for a year, do not mess around with hard drives, lap-tops, etc. It's just begging for trouble. The HVX-200 while being a fine camera (on paper, no one's used it yet), doesn't look like a good choice for documentary work where you have hours and hours of footage.

The workflow is not there yet for long form documentary work, requiring the archiving of so much material. However perhaps by your shoot next july there will be a solution that makes sense.

Another good thing about mini-dv tape is that it's ubiquitous. I once bought a 5 pack of sony dv tapes literally ON the mekong river, from a guy in a longtail boat. If you're travelling around from port to port, there's always somewhere you can buy DV tape. An extra USB hard drive may be more difficult to find.

I will agree that the HVX does have a much better codec than HDV cams, but will that alone result in a better image? The HVX isn't out yet, so no one is sure about the lens and CCDs. Codec is not the be-all and end-all factor in the image.

If the choice is a slightly better image, pitted against the chance of running out of HD space, or heaven forbid catastrophic hard drive failure. It's infinitely better to have a slightly more compressed image on tape, than no image at all.

Still by the time you go on your shoot, the situation may have changed. July is still a long way off. So my suggestion is don't buy now, wait till all the cameras (Sony, JVC, Panasonic, and Cannon), have been out for a while and time tested in the field before you make your choice. This board is the perfect place. If there is any little flaw in a camera, rest assured you will hear about it here.

You may find this BBC Guide to HD helpful. It is still in draft, I believe, and contains some errors. It is not yet supposed to be available to the public but someone found a way in. It provides advice for those new to HD production and direction. Link:
http://www.bbc.co.uk/guidelines/deli...Bookv01_00.pdf

I agree 100% with John. You cant do selects on a documentary until you are done shooting. Also, you need to have everything archived and backed up. Do you really want ALL your shots on a couple hard drives in an environment that is uncontrolled? Right now there is no solution but tape for docs of this type and the only affordable and portable HD solution is HDV. I am not a fan of HDV but it would work best for you if you have to go HD.

This is the big problem with both the HVX and the XLH1. To get the best HD results you have to spend a lot more money and it becomes less portable. I personally wont invest in HD gear until there is a more solid and affordable collection media.


ash =o)

I would be very careful to ask the prospective networks if they will accept HDV shot material. The choice of networks leads me to the opinion that the highest level of quality is desirable. I would bet the HVX-200 @ 720/24 (or30)p, sight unseen, over the Sony Z-1 @ 1080/30i any day of the week. The places where the HDV codecs break apart are quck movement, like unpredictable documentary moves on the pitching of a boat, and subtle gradients, like wide expanses of sky.

And while it's true that you can get miniDV tapes anywhere in a pinch, I don't think that's going to be an issue here. I also wouldn't just slap any old off-brand miniDV tapes into the Sony Z1 and expect them to be ultra-reliable, especially if you mix tape stocks. I've had a handful of brand-new DV tapes fail on me over the years, and I've only had one IDE hard drive go south (with warning enough to get the data off, BTW). I wouldn't get too dogmatic about tape being the safest and best storage soulution out there.

Let's say you have the ability to store your external drives in a nice, padded Pelican case, you treat them with reasonable care, keeping them away from undue moisture and shock, and you go the extra step to make a redundant backup on a second drive. In that case, I would say that you have an archival system that rivals, if not exceeds the reliability of MiniDV tape.

And, you get the DVCProHD codec, which will look better right off the camera, and hold up MUCH better to the inevitable post-processing. I've never been much of a Panasonic guy, but this camera is going to be unreal. And for everyone that says the workflow isn't there yet, I beg to differ. The workflow is laid out, right in front of you, and it's the same workflow we've all been using for years with our NLE's. You just have to start thinking about your footage as data, not as tapes.

Who knows, there may be a major gotcha that will surface once this camera is shipped. Maybe the Firestore won't even be available by the time you need to gear up, which would put a big crimp in your workflow. I just wouldn't commit to a particular camera until you have to. In the meantime, why not rent a Z-1 and go shoot some test footage? That will tell you pretty quick if the image is suitable for your purposes. That way, when the HVX-200 ships, you'll already have something to compare to.

I think everyone is way off on the archive solution here. Archiving a year's worth of footage onto HDD's that've been sitting in a boat is nuts. The footage should only be archived onto DLT--this is by far the most reliable solution. If you check out Panasonic's white paper on the HVX, DLT is their recommended archive solution. I'd say shoot with the Firestore (bring a couple backups) and archive to DLT.

Peter

Why is this a good thing Scott?

We are looking at both North American and European broadcasters so this is iindeed a factor. Wouldn't 1080i be another option to 24p? I'm just concerned that at times the movement within frame may be to frenetic for 24p.

This is my big concern. I got back from a circumnavigation 4 years back and that environment is extremely hostile to electronics...heck, I was beginning to corrode! But having said that, I still like what the HVX200 offers from a quality standpoint, provided that it delivers.

I would never consider selecting my shots at the shooting stage...I plan only to delete those which are unusable. You are certainly correct about backing up any electronically stored data which would certainly increase the cost.

Brilliant! Thanks David. This publication was very informative. Used to work for the Beeb many, many years ago and it was interesting to get their take on HD.

I've put out a call to DLT Solutions Inc. and am hoping that they will get back to me with an answer. I'll let you all know.

Thanks so much for all the help guys - a great forum.
Bruce

Because with 1/3" CCD's having native 1080 pixels, the result would be poor dynamic range and noise. Graeme Nattress has explained this much better than I ever could. In fact, if you want technical explanations, search for every post Graeme has ever made.

I was lucky enough to have met Graeme at NAB this year, and he's one of the most down to earth, self-effacing guys you could ever hope to meet. He also happens to be a codec/compression super genius. His plugins are pretty darn cool, too. Google him - you'll see. I would trust his opinion implicitly.

Here's one of the posts I was thinking of:
www.dvinfo.net/conf/showthread.php?t=48575

People said the same thing about the smaller Nikon D2X sensor before it came out. Despite what people thought, the camera is very clean. Technology has come along way in the last few years. I expect the HVX200 to have the same sensitivity and dynamic range as the DVX100a(or close to it).

John