EX Workflow - do you keep BPAV files
 

My workflow is that on return from a shoot, I copy the cards, using ClipBrowser, to a project directory on an external hard disc. I then review the clips and add logging metadata to the copy. Finally, using XDCam Transfer, I pull the clips I want into FCP.

XDCAM Transfer creates an mp4 (and xml) file for each of the clips imported, so this workflow means I end up with 2 copies of each clip. I am wondering whether to delete, or possibly archive, the older BPAV folders and wondered what anyone else does.

Cheers,

Nick

I would and do keep the BPAVs as this is the only way you can use the material with other systems in the future.

Typically I just burn each stick's BPAV folder onto DVD-DL (as I mostly use 8GB SXS, so it's nice and easy). That's how I archive the BPAVs but have my working files sitting on RAID 0's (and clones on multiple non-raided back-up drives), mainly as .movs (as I've been using a Mac workflow).

You're spot on Bruce - in the last 2 years I've been using a Mac Pro and MBP workflow. Now, as of this week, I ALSO have a highly capable new i7 Windows 7 PC with Vegas 9E 64-Bit workflow for XDCAM EX work as well.... So, my huge EX3 shot archive is fully accessible on both these platforms - which it certainly would not have been (easily) if I'd not kept the BPAVs!

You never know what you'll do in the future, but by keeping at least one clone of your archive backed up as BPAVs keeps your NLE options fully open.

Think of the BPAVs as your negatives.

...think your BPAV folders as your original master "tapes".. if you have valuable material filmed, just keep your BPAVs ("tapes") in some safe place .... I also have Vegas, which do not need any transcoding of the BPAV files to edit them....

good luck

This is good advice - we also treat BPAV folders - regardless of what media they are stored on as the "master field tapes".

We keep all BPAV folders in separate projects folders. For example, 'New York Shoot June 2009" would have folders labeled Card 1, Card 2, Card 3, etc... and inside would be the corresponding BPAV folder.

We are near-chiving to a 40TB RAID-6 array right now but will begin offloading some of the older project folders to LTO-5 data tapes for long term shelf storage. LTO-5 tapes are around $100 with capacity of 1.5TB each. The challenge is less about the storage and more about the organization.

That's my notion too.
If it is unclear what the "perfect" archiving method might be, it's easy enough to just back-up all the BAPVs on cheap external drives. When the "perfect" solution does finally cross your radar, you can transfer at that time.
And... there's always Murphy's Law- if you back them up, you will never need them; the minute you don't back-up, some disaster you've never even dreamed of will happen and you'll be naked.
How many times I've been burned by saying "No way.. I'll never need this file/clip/format/project/whatever again..."

It is better to avoid nested folder structures that aren't conveniently read by lots of software. The BPAVs are only masters if the software to read them continues to exist. Need to transfer a few clips to someone else? BPAV is a pain. My major beef with BPAV and Sony software is that it is much faster to preview and delete clips with XP or OSX with VLC Media Player and delete.

Though, yes. I keep the BPAVs intact. I thought they were large until I started using the Nanoflash for higher bitrate recording.

OK, sorry I am going against the tide here. And this is just my way of dealing with archiving.

I don't keep the BPAV files. I use XDcam Transfer to make .movs. And I archive those on at least 2 HD's in the field on long shoots, or in the studio at the end of the day. I also make BluRay or DVD's of all the clips.

The discs are stored in different buildings and the BluRays/DVD's in a very safe place.

Now I only use Macs in my suites, most of my projects are edited by me. I do give producers who request BPAV's the folders but this is rare for me.

I also shoot a lot with the 5DmkII and 7D as well as using a NanoFlash often on the XDcams. So it only makes sense to treat every clip the same. You cant use clip browser to look at 5DmkII clips.

I use FCP for organizing and viewing the clips. I also have a separate Database program I wrote that keeps track of all my clips including thumbnails, it is keyword searchable and shows what BluRay and discs and folder the clips are in. I can locate and load clips onto my RAID L5's in the edit suites this way very quickly. I don't use FCP for editing much, too slow, we use M100, no rendering, incredible fast NLE. All the clips that have been picked in FCP get transcoded into ProRes422 and dropped into M100. I find this system very fast and efficient.

If I have to transfer footage to a TV station or a producer on a piece about one of our clients. I usually just transcode to the format they need usually H.264. And send a HD or DVD. We do this often for our clients. We do a lot of tourism related shooting here in NH. We even do a fair amount of FTP uploads for news features. Everyone is very happy with this system.

Below is a screen shot of the DB.

Once again I am not preaching everyone adopt this system. This is just my way of dealing with NO-tape footage. We still use a bit of tape as well. This is way faster. And any tape that gets digitized now gets entered into this archive system for easy retrieval and multi location backup.

Another vote for Olof here. I do the same.

For me, BPAVs are 'exposed film' which need 'developing'. I have no use for storing exposed yet undeveloped film. I want it processed, which - for me, and your mileage may vary - is something that other applications in my ecosystem will open up and deal with as single entities.

So, I enter user metadata into XDCAM Transfer like 'interviewee: name, job title, affiliation' or specific location info, so when the BPAV files get 'developed' they hold that metadata in each file that is viewable in FCP and other apps. They also get given a meaningful name so they can be identified by text search tools.

The BPAV files are then deleted. They no longer exist in a meaningful sense. They are merely an instance of the XDCAM Transfer output that are temporary and quickly forgotten.

* * * WAIT A MOMENT * * *

Yes, folks. That's my workflow, not yours, and your situation is not my situation, so before you tell me that I am an unclean sinner for treating BPAV material with such contempt (you know who you are), I have another workflow when working as a cameraman or as a wrangler.

I had a job last year to record some interviews that would be required in ten years time, maybe more. Of course that footage is backed up as BPAV as well as QT, multiple copies vaulted away. It may be contributing to my pension, so I treat it as such.

I have regular shooting work where I hand over footage. In the olden days, I just handed over cassettes, but as we migrate to tapless workflow, it means handing over 'edit ready' stuff. QuickTime movies are not necessarily 'edit ready' to many Avid editors. So I supply BPAV copies plus boilerplate on what to do next - though I've changed that for some clients.

SD clients like AVI files, so even though I have implored them to see the benefits of HD, I'm now happy to shoot SD and give them AVIs - which work in FCP too, BTW. Such clients wouldn't know what to do with BPAVs, and when I try to explain, they go into 'TILT' mode.

All these client situations are NOT 'broadcast' clients, they have their own workflows.

And - distant rumbling of seething resentment - I am not going to be the repository of all my clients' data unless they pay for it, which almost all I work with DON'T. I may do that as an expedited service, but clients want their footage on a disk for themselves like they'd get cassettes, and BPAVs are not something they can easily understand or deal with, look at, hand over to somebody else, or otherwise interact with - which QT movs are - difficult as they are to get to work well on a pC - capable of.

Blogged it here: EX1R SD Workflow – tales from the front line Travelling Matt

Originally, I would just use the Sony transfer tool to convert the BPAV on the flash card to mxf files on the disk. In an early Sony ClipBrowser revision, there a bug with 720p60 conversion and did not include the audio. Luckily, I only lost *one* entire video session. (nauseating chuckle) Since then, I always keep at least two instances of the BPAV file stored on separate devices before formatting the flash cards.

Olof, I just love the screen shot of the DB! I appreciate you sending me the info on getting it, but I'm on PC. Oh that I knew how to do this myself! I tried with MS Access, but no luck (probably user ignorance). Looks perfect though. Just what I've been trying to accomplish.

Back to the topic, I just pull the EX1's MP4 video file ONLY out of each folder and erase the rest. I am using different cameras often and they all play very well together in a Premiere CS5 workflow (Canon A1 on a Sony MRC1K, Canon 7D and the EX).

I have a special naming format for each file that corresponds to a detailed spreadsheet that tracks all the clips. I've never even used the BPAV files.

Just me...

As anyone can see, everyone has a different workflow. However, some of these guys have been doing this for years so they know what works best for them. With the way FCP is changing...I mean not changing and the progress Avid and Adobe have made, I would advise to keep the BPAV for the newer XDCAM users.

Furthermore, your type of work & clients also dictate your workflow. For example, I have been approached by some people who want to pay for some of my material that I am recording for someone else (live events). If I want their business, then I need to make certain to keep the BPAV in case their workflow is different from mine. This is hypothetical because I use Premiere Pro CS5 so I always keep and archive the BPAV.

AFAIK, only the original MP4 in the BPAV folder can have Flash Band removal applied so there is another reason to keep the BPAV.

BPAV is the camera master. Not keeping it is a disaster waiting to happen.
Just wait 'till you need to hand to a client or other post production workflow that needs it even if it's just to rewrap to something else. For those who say "never happen" I'll say the first time it does it will cost you time, money, possibly the client because of the previous two.

Some notes to add here:

FCP will change, there is according to Steve Jobs a new version that will blow our socks off in the pipelines.... just how long that pipeline may be????? This may (or may not) work directly with BPAV folders.

I believe Sony are working on a unified XDCAM viewer/logger/transfer tool that will ultimately replace the separate XDCAM transfer tool, clip browser and various XDCAM HD PDZ viewers etc. This new tool should incorporate all the clip list editing, database, export and logging functions of all these tools in a single package that will work with EX and XDCAM HD. It will be cross platform compatible so Mac and PC. To use this you will need to keep your BPAV folders. It should be a great tool.

The BPAV folders contain metadata that does not get included in the .mov's. It's likely that forward planning metadata and extra metadata will get added to XDCAM EX in the future.

.Mov's containing XDCAM material are a very specific file, tailored for FCP. Even Premiere CS5 on a Mac doesn't like them.

I always keep at least one copy of the BPAV folders. One way of working is to keep one BPAV copy and one .mov copy. Should you loose one or the other you can (with a little work) re-create the other. .mov's are easy to create from a BPAV. BPAV's take more work as you need to first create mp4's from FCP and then import those in to a new folder using clip browser.

My workflow differs again to everyone here...
 

My workflow differs again to everyone here...

When shooting, I give the clips a title relating to the client/agency/production company.

So I have things like PON, DHG, RED, MC1, I have a notepad in the EX1 bag where I keep a 'diary' of every card. This show something like '105 15/07/10 Pennells Garden Centre 4 DHG_1157 - DHG_1241 Card 6'

So it was my 105th card, shot on the 15th of July, the 4th time I've filmed at Pennells, clip numbers 1157 to 1241 and recorded on card 6. I shoot TOD timecode so sometimes a client will already be making a rough log as we shoot. The diary means that the next time I work with a particular client I can set the camera to number the clips from where they left of the last time. (I also keep a log of how many times I've used each card and any camera information like firmware updates, new kit, problems etc).

The BPAV gets copied into a folder named '105' using drag and drop on OSX to two large USB drives.
I use the 'compare' CRC function in Toast to check the integrity of these copies, and the two drives are kept at different offices.

From one of these drives I can import the footage into FCP using log and transfer where shows in the FCP bin as Reel 105. The project folder in Capture Scratch will end up being copied to two more large USB drives at some point, along with the imported Quicktimes any graphic elements, scripts, etc. etc. This means reinstating the media for amends or future work is fairly painless and I still have the BPAVs for future use.

With a new job I'm just starting, the client keeps one of the BPAV backup drives and uses Clip Browser on a PC to transcribe the footage. He brings the drive in a flightcase and I give him new footage as we shoot it. At some point I hope to have another cameraman on the job, I'll continue using MC1 and he will use MC2 with his own 'diary' of clips.

It can take a little bit of work to keep the archives up to date, but the drives are cheap and seem very reliable, I think as long as you power them up once in a while for an hour or so they'll last for some time. If something fails you have another copy to quickly create a backup, and you can always take the physical drive out to try and repair it.

Duncan.