EX Workflow - do you keep BPAV files
 

My workflow is that on return from a shoot, I copy the cards, using ClipBrowser, to a project directory on an external hard disc. I then review the clips and add logging metadata to the copy. Finally, using XDCam Transfer, I pull the clips I want into FCP.

XDCAM Transfer creates an mp4 (and xml) file for each of the clips imported, so this workflow means I end up with 2 copies of each clip. I am wondering whether to delete, or possibly archive, the older BPAV folders and wondered what anyone else does.

Cheers,

Nick

I would and do keep the BPAVs as this is the only way you can use the material with other systems in the future.

Typically I just burn each stick's BPAV folder onto DVD-DL (as I mostly use 8GB SXS, so it's nice and easy). That's how I archive the BPAVs but have my working files sitting on RAID 0's (and clones on multiple non-raided back-up drives), mainly as .movs (as I've been using a Mac workflow).

You're spot on Bruce - in the last 2 years I've been using a Mac Pro and MBP workflow. Now, as of this week, I ALSO have a highly capable new i7 Windows 7 PC with Vegas 9E 64-Bit workflow for XDCAM EX work as well.... So, my huge EX3 shot archive is fully accessible on both these platforms - which it certainly would not have been (easily) if I'd not kept the BPAVs!

You never know what you'll do in the future, but by keeping at least one clone of your archive backed up as BPAVs keeps your NLE options fully open.

Think of the BPAVs as your negatives.

...think your BPAV folders as your original master "tapes".. if you have valuable material filmed, just keep your BPAVs ("tapes") in some safe place .... I also have Vegas, which do not need any transcoding of the BPAV files to edit them....

good luck

This is good advice - we also treat BPAV folders - regardless of what media they are stored on as the "master field tapes".

We keep all BPAV folders in separate projects folders. For example, 'New York Shoot June 2009" would have folders labeled Card 1, Card 2, Card 3, etc... and inside would be the corresponding BPAV folder.

We are near-chiving to a 40TB RAID-6 array right now but will begin offloading some of the older project folders to LTO-5 data tapes for long term shelf storage. LTO-5 tapes are around $100 with capacity of 1.5TB each. The challenge is less about the storage and more about the organization.

That's my notion too.
If it is unclear what the "perfect" archiving method might be, it's easy enough to just back-up all the BAPVs on cheap external drives. When the "perfect" solution does finally cross your radar, you can transfer at that time.
And... there's always Murphy's Law- if you back them up, you will never need them; the minute you don't back-up, some disaster you've never even dreamed of will happen and you'll be naked.
How many times I've been burned by saying "No way.. I'll never need this file/clip/format/project/whatever again..."

It is better to avoid nested folder structures that aren't conveniently read by lots of software. The BPAVs are only masters if the software to read them continues to exist. Need to transfer a few clips to someone else? BPAV is a pain. My major beef with BPAV and Sony software is that it is much faster to preview and delete clips with XP or OSX with VLC Media Player and delete.

Though, yes. I keep the BPAVs intact. I thought they were large until I started using the Nanoflash for higher bitrate recording.

OK, sorry I am going against the tide here. And this is just my way of dealing with archiving.

I don't keep the BPAV files. I use XDcam Transfer to make .movs. And I archive those on at least 2 HD's in the field on long shoots, or in the studio at the end of the day. I also make BluRay or DVD's of all the clips.

The discs are stored in different buildings and the BluRays/DVD's in a very safe place.

Now I only use Macs in my suites, most of my projects are edited by me. I do give producers who request BPAV's the folders but this is rare for me.

I also shoot a lot with the 5DmkII and 7D as well as using a NanoFlash often on the XDcams. So it only makes sense to treat every clip the same. You cant use clip browser to look at 5DmkII clips.

I use FCP for organizing and viewing the clips. I also have a separate Database program I wrote that keeps track of all my clips including thumbnails, it is keyword searchable and shows what BluRay and discs and folder the clips are in. I can locate and load clips onto my RAID L5's in the edit suites this way very quickly. I don't use FCP for editing much, too slow, we use M100, no rendering, incredible fast NLE. All the clips that have been picked in FCP get transcoded into ProRes422 and dropped into M100. I find this system very fast and efficient.

If I have to transfer footage to a TV station or a producer on a piece about one of our clients. I usually just transcode to the format they need usually H.264. And send a HD or DVD. We do this often for our clients. We do a lot of tourism related shooting here in NH. We even do a fair amount of FTP uploads for news features. Everyone is very happy with this system.

Below is a screen shot of the DB.

Once again I am not preaching everyone adopt this system. This is just my way of dealing with NO-tape footage. We still use a bit of tape as well. This is way faster. And any tape that gets digitized now gets entered into this archive system for easy retrieval and multi location backup.

Another vote for Olof here. I do the same.

For me, BPAVs are 'exposed film' which need 'developing'. I have no use for storing exposed yet undeveloped film. I want it processed, which - for me, and your mileage may vary - is something that other applications in my ecosystem will open up and deal with as single entities.

So, I enter user metadata into XDCAM Transfer like 'interviewee: name, job title, affiliation' or specific location info, so when the BPAV files get 'developed' they hold that metadata in each file that is viewable in FCP and other apps. They also get given a meaningful name so they can be identified by text search tools.

The BPAV files are then deleted. They no longer exist in a meaningful sense. They are merely an instance of the XDCAM Transfer output that are temporary and quickly forgotten.

* * * WAIT A MOMENT * * *

Yes, folks. That's my workflow, not yours, and your situation is not my situation, so before you tell me that I am an unclean sinner for treating BPAV material with such contempt (you know who you are), I have another workflow when working as a cameraman or as a wrangler.

I had a job last year to record some interviews that would be required in ten years time, maybe more. Of course that footage is backed up as BPAV as well as QT, multiple copies vaulted away. It may be contributing to my pension, so I treat it as such.

I have regular shooting work where I hand over footage. In the olden days, I just handed over cassettes, but as we migrate to tapless workflow, it means handing over 'edit ready' stuff. QuickTime movies are not necessarily 'edit ready' to many Avid editors. So I supply BPAV copies plus boilerplate on what to do next - though I've changed that for some clients.

SD clients like AVI files, so even though I have implored them to see the benefits of HD, I'm now happy to shoot SD and give them AVIs - which work in FCP too, BTW. Such clients wouldn't know what to do with BPAVs, and when I try to explain, they go into 'TILT' mode.

All these client situations are NOT 'broadcast' clients, they have their own workflows.

And - distant rumbling of seething resentment - I am not going to be the repository of all my clients' data unless they pay for it, which almost all I work with DON'T. I may do that as an expedited service, but clients want their footage on a disk for themselves like they'd get cassettes, and BPAVs are not something they can easily understand or deal with, look at, hand over to somebody else, or otherwise interact with - which QT movs are - difficult as they are to get to work well on a pC - capable of.

Blogged it here: EX1R SD Workflow – tales from the front line Travelling Matt

Originally, I would just use the Sony transfer tool to convert the BPAV on the flash card to mxf files on the disk. In an early Sony ClipBrowser revision, there a bug with 720p60 conversion and did not include the audio. Luckily, I only lost *one* entire video session. (nauseating chuckle) Since then, I always keep at least two instances of the BPAV file stored on separate devices before formatting the flash cards.

Olof, I just love the screen shot of the DB! I appreciate you sending me the info on getting it, but I'm on PC. Oh that I knew how to do this myself! I tried with MS Access, but no luck (probably user ignorance). Looks perfect though. Just what I've been trying to accomplish.

Back to the topic, I just pull the EX1's MP4 video file ONLY out of each folder and erase the rest. I am using different cameras often and they all play very well together in a Premiere CS5 workflow (Canon A1 on a Sony MRC1K, Canon 7D and the EX).

I have a special naming format for each file that corresponds to a detailed spreadsheet that tracks all the clips. I've never even used the BPAV files.

Just me...

As anyone can see, everyone has a different workflow. However, some of these guys have been doing this for years so they know what works best for them. With the way FCP is changing...I mean not changing and the progress Avid and Adobe have made, I would advise to keep the BPAV for the newer XDCAM users.

Furthermore, your type of work & clients also dictate your workflow. For example, I have been approached by some people who want to pay for some of my material that I am recording for someone else (live events). If I want their business, then I need to make certain to keep the BPAV in case their workflow is different from mine. This is hypothetical because I use Premiere Pro CS5 so I always keep and archive the BPAV.

AFAIK, only the original MP4 in the BPAV folder can have Flash Band removal applied so there is another reason to keep the BPAV.

BPAV is the camera master. Not keeping it is a disaster waiting to happen.
Just wait 'till you need to hand to a client or other post production workflow that needs it even if it's just to rewrap to something else. For those who say "never happen" I'll say the first time it does it will cost you time, money, possibly the client because of the previous two.

Some notes to add here:

FCP will change, there is according to Steve Jobs a new version that will blow our socks off in the pipelines.... just how long that pipeline may be????? This may (or may not) work directly with BPAV folders.

I believe Sony are working on a unified XDCAM viewer/logger/transfer tool that will ultimately replace the separate XDCAM transfer tool, clip browser and various XDCAM HD PDZ viewers etc. This new tool should incorporate all the clip list editing, database, export and logging functions of all these tools in a single package that will work with EX and XDCAM HD. It will be cross platform compatible so Mac and PC. To use this you will need to keep your BPAV folders. It should be a great tool.

The BPAV folders contain metadata that does not get included in the .mov's. It's likely that forward planning metadata and extra metadata will get added to XDCAM EX in the future.

.Mov's containing XDCAM material are a very specific file, tailored for FCP. Even Premiere CS5 on a Mac doesn't like them.

I always keep at least one copy of the BPAV folders. One way of working is to keep one BPAV copy and one .mov copy. Should you loose one or the other you can (with a little work) re-create the other. .mov's are easy to create from a BPAV. BPAV's take more work as you need to first create mp4's from FCP and then import those in to a new folder using clip browser.

My workflow differs again to everyone here...
 

My workflow differs again to everyone here...

When shooting, I give the clips a title relating to the client/agency/production company.

So I have things like PON, DHG, RED, MC1, I have a notepad in the EX1 bag where I keep a 'diary' of every card. This show something like '105 15/07/10 Pennells Garden Centre 4 DHG_1157 - DHG_1241 Card 6'

So it was my 105th card, shot on the 15th of July, the 4th time I've filmed at Pennells, clip numbers 1157 to 1241 and recorded on card 6. I shoot TOD timecode so sometimes a client will already be making a rough log as we shoot. The diary means that the next time I work with a particular client I can set the camera to number the clips from where they left of the last time. (I also keep a log of how many times I've used each card and any camera information like firmware updates, new kit, problems etc).

The BPAV gets copied into a folder named '105' using drag and drop on OSX to two large USB drives.
I use the 'compare' CRC function in Toast to check the integrity of these copies, and the two drives are kept at different offices.

From one of these drives I can import the footage into FCP using log and transfer where shows in the FCP bin as Reel 105. The project folder in Capture Scratch will end up being copied to two more large USB drives at some point, along with the imported Quicktimes any graphic elements, scripts, etc. etc. This means reinstating the media for amends or future work is fairly painless and I still have the BPAVs for future use.

With a new job I'm just starting, the client keeps one of the BPAV backup drives and uses Clip Browser on a PC to transcribe the footage. He brings the drive in a flightcase and I give him new footage as we shoot it. At some point I hope to have another cameraman on the job, I'll continue using MC1 and he will use MC2 with his own 'diary' of clips.

It can take a little bit of work to keep the archives up to date, but the drives are cheap and seem very reliable, I think as long as you power them up once in a while for an hour or so they'll last for some time. If something fails you have another copy to quickly create a backup, and you can always take the physical drive out to try and repair it.

Duncan.

I don't keep the BPAV folders. I import to the mac with XDCAM Transfer, trim and name the clips with QT Pro, tweak them in AE and then back up to external disks.

But you'll get burned the moment you have to hand them to a non Final Cut Pro person whether editor or client.

A few things.

1. If you do need the BPAV structure in the future, just format a new card in the EX and export back to XDCAM with FCP. You can recreate a BPAV structure if you really want to if you deleted the original, with no recompression of material.

2. ProRes codec is now available for free for the PC. You could just export for that and hand it over to a client, along with a multitude of other types.

3. Sometimes I find the idea of keeping the BPAV structure 'just in case' to be like keeping junk in your garage just in case. How much milage you get will depend precisely upon how truly realistic it is that you'll need to hand off to someone else on a different system after you have imported and edited it all. Lets be realistic, for most of us that will be never. And on top of that you can always rewrap to MP4 with the XDCAM exporter, or even export/rewrap as the 1920x1080 MXF format.

Look, there is always a way. That's the whole advantage of using files. The BPAV structure is only important if the editing system in question can only take XDCAM footage in using some sort of dedicated importer. But if that NLE can import files and have them dragged natively to the timeline I'm not sure what the issue with deleting the BPAV structure is. Even if the NLE in question is pathetic, you can always find a way to do it.

Writing an XDCAM EX .mov from FCP out as an mp4 is simple, but each clips has to be done individually unless you are prepared to put them all in the timeline and then export in 12 minute chunks. You also have the issue of having to re-split long clips back into 4Gb chunks if your writing to the camera or other Fat32 media. If you are looking at a big project with 100's of clips this would be a real drag.

I've had to walk at least 3 production companies through this whole process as for various reasons they had to move from FCP to other platforms. It's not pretty and they could have saved themselves a lot of agro by keeping copies of the BPAV's.

That's tough though isn't it? I'm the production company, I store the files. If someone else wants the files they can either have them in the format I choose to store them in or pay me to convert them to a format of their choice. If it's my project I make the calls. I archive the complete project (to HD and Blu-Ray or DVD) and also the MOV files (again HD & Blu-Ray or DVD). The movs are instantly playable and editable. The only reason you'd need the BPAV's at any stage would be either future use on another platform or to hand over to someone else. But this is an easy problem to fix with the 3rd party exporters that will rewrap the files for you.

If shooting for a 3rd party then yes, keep the BPAV's, but for me there is no benefit. I can always rewrap back to mp4 if I really need to thanks to the 3rd party exporters available (Calibrated Software isn't it?).

If I ever did move to another NLE (unlikely) then I'd do what I did with Premiere on the PC and keep a legacy system running to edit the older projects.

I've always archived my DV tapes to HD and DVD as well. As a result I have never had to go back to a tape after the original transfer. They all sit there on the shelf looking pretty, but only played the once. And that's what would happen with the BPAV's, they'd never be used.

Marcus, your fundamental mistake, IMHO, is you're trying to predict the future.

XDCAM EX MOV is specific to FCP and to FCP at a specific point in time. The change in workflow ahead may not be of your choosing. The future scenarios are far too broad and can happen more quickly than you can anticipate.

There is metadata in the BPAV that FCP currently doesn't use. That may change. If so, you've locked yourself out of that.

BPAV is "universal" (can be changed to whatever) and to charge clients for something that shouldn't be necessary is a business practice that can come back to bite you.

You are "a" not "the" production company and a client at any future point may decide that they want you to work on "x" part of the project while another, with different gear and expertise, will work on "y" part of the project.

Will you generally find away around most "movement" issues, probably, is that the most time and cost efficient, no. Read Alister's post for example. I could certainly add many more.

There's a Quicktime codec available for both the PC and the Mac by Calibrated Software that allows EX and XDCAM HD and 422 .mov files to be read by Quicktime on either system without having FCP installed.

I'm just not convinced that keeping the BPAV's is essential at all. What do people do with NanoFlash recorded files when they record to native .mov format? You could record to MXF with the nano too, but you can also rewrap your existing EX files to MXF too.

The way I see it is that the NanoFlash lets you record right from the beginning without the BPAV structure. So why not keep your EX files in the same way that the Nano gives them to you from the beginning?

Simon
As I've said there is certainly ways to rewrap. Do you wand to force someone to buy a plugin though? Do you want to spend the time rewrapping files or explaining how to do this to clients?
I tend to take what I believe is the easy path. BPAV is no larger than the rewrapped MOV. BPAV can get you anywhere you need to go quickly. MOV is fraught with obstacles great and small.

I do not want to force clients or other production houses to buy a specific plugin.
I do not want to invest time rewrapping or billing clients for rewraps that wouldn't have to be done otherwise.
I do not want to lose the metadata that is thrown out in the rewrap to MOV as it may server a future purpose to someone.

Not only does NanoFlash offer MOV recording so does JVC HM series. Yes it can be an immediate convenience recording to MOV directly but that's not what the EX cameras do. Use BPAV to one's advantage, I say

Personally I suspect that their are disadvantages in the metadata use from camera file sources that Apple might account for in the next FCS.

File portability should be an advantage and not something one has to tie up resources working around. I think retaining that advantage is important.

Calibrated also makes a plugin that allows one to use the BPAV/MP4 directly in Final Cut Pro. Disclaimer, I was a beta tester for it. I wouldn't be surprised if Apple including such a feature in the future (along with AVCHD direct support). There's a big advantage having ingested files that link directly to camera master files. That fact that we have to debate the use of MOV shows it's disadvantage. The files take some work to make portable to other workflows.

I don't think QT will go away any time soon, and as long as it's around I suspect I'll be able to find an acceptable way to transcode the files for whoever might need them. We don't all have the same needs when it comes to workflow. It's not one size fits all. I'm not suggesting that there's no point in anyone keeping BPAVs, just that I personally have no need for them.

If Quicktime files become obsolete, the camera master issue would become just one of a number of other issues that we would all hit.

What about stock footage you buy? Any animations or screen recordings in your production? How about other footage you may have that only exists as Quicktime files? In the week I have to transfer some VHS and I'll store that footage as Quicktime. The tape will go back so then I'll left with the file and nothing else.

And what about the fact I store finished productions as Quicktime files as the days of dubbing back to tape are long gone?

Believe me, if Quicktime support vanished you are going to have alot more issues than just your camera masters not working.

I'm guessing it's just logging information in which case I have a system in place already for this that works perfectly well and fully integrates with my non XDCAM footage (as I've previously pointed out I've been storing all my footage as files for 5 years now).

It is my personal opinion that BPAV folders will go away long before QT.

Just my guess.

I keep BPAV's for a while but all my main archives are .movs.

I would say anyone starting out should absolutely keep and backup the BPAV's until they decide the best system for handling their particular workflow.

I think we will see many more systems like P2 mp4 etc in the following years. I am personally rooting for RAW video files as media becomes faster and cheaper.

And I also bet that QT will be compatible with all of them, at least in the next decade or two.

For me it makes sense to keep everything as .movs, weather it is digitized BetaSP, DVcam, HDV or HDcam etc. etc. or whatever solid state comes down the pike. For me it is a great standard.

Marcus if you look at the BPAV in ClipBrowser or the actual XML and other files, you'll see information about camera settings and other information that is thrown out when the file is rewrapped to MOV.

EX XDCAM codec is not standard part of Quicktime. It is part of Final Cut Studio install. The only other way to get EX XDCAM MOV is to buy it from CalibratedSoftware.

There is no easy way to wrap EX XDCAM MOV back to .mp4 or .mxf and that's especially so without Final Cut Studio.

If you look at the many NLEs, it's actually the .mp4 which is "the standard" not .mov. Premiere Pro, Vegas and I believe Edius handle the .mp4 directly.I believe Avid MC5 with AMA may handle it as well. I suspect FCS will handle the .mp4 directly as well in the next upgrade.

Basically EX XDCAM .mp4 direct support is spreading as the standard and .mov, as proprietary will be "legacy" if FCP moves to .mp4 direct support.

What I like about QT, is that it is so inclusive. It is a wrapper for lots of different codecs, like mp2, mp4, mpg as well as some really great loss less codecs built in.

I like the fact that if you have the latest version of FCP, Flip for mac and Sorenson Squeeze there is almost no file you can't read or compress to Flash or whatever.

For me it is really convenient to archive in one format. I am on a Mac but it has never been a problem to transcode to Windows file formats. My main editor is M100 and it has its own codecs but they are free downloads both for Macs and PC's, as long as you have QT installed.

I don't just work with XDcam files. So it makes sense for me to use a common file format in my archives.

But I am not trying to say that everyone should use this approach, just that it is my way of dealing with my video project, and I have been doing that digitally since the early 90's, and I can still open QT files from back then, and use them in my current projects, and export them to any platform a client may require.

Olof, I think Apple was banking on the universality of the Quicktime wrapper and I certainly would have liked it too but I think Apple may be rethinking that. I think they are looking for "universal" codec as may be happening with Apple ProRes. Even though "everything" can play ProRes MOV (free decode for windows and certainly on Macs even without Final Cut) that hasn't happened overall.

The key difference is the unique nature of EX XDCAM MOV which is specific to ONE NLE on the Mac only. The codec is not a standard part of Quicktime nor is it plus the MOV wrapper standard to much of anything else.

I think Apple held out a long time staying with the idea that everything in FCS should be wrapped as MOV. Just a hunch but I'm guessing Apple's about to give on that (next year) with FCS4. If anything Calibrated Software proves that a plugin can allow native codecs in their native wrappers to work with FCP.

You can't even quite compare to AVCHD. That AVCHD is likely going to ProRes and these days that ProRes file can probably go to many other NLEs.

You'll certainly still be able to play EX XDCAM MOV but given it's nature it's the "odd bird" and I don't think it's the way forward.

Of course you can always convert the EX XDCAM to ProRes but, as with AVCHD, that process is an Achilles Heal in that it takes time and the files are significantly larger. Hence I suspect Apple will be supporting native codec, native wrapper. Apple is certainly aware of the ProRes file size issue as one of the key changes between FCS2 and FCS3 was the addition of ProResLT and ProResProxy.

When I hear rumor talk of Apple working on a new codec it may be to work out that ProRes file size issue in their quest for their "universal" codec (not to be confused with making MOV a universal wrapper).

Craig,

The way I understand it is that XDcam is really an MPEG2 fileformat.

ProRes is Inra frame therefore the larger file size and easier editing, it is also much less lossy over generations. Am I wrong Craig?

Also HD/BluRay storage is really cheap these days, that is how I archive, I dont mind the increased size.

Right now XDcam footage is less than 30% of the file format I use for acquisition for projects I work on. I am trying to use a uniform format for all kinds of media, and so far QT has been it.

I see the BPAV folder as an added complexity I don't need.

Craig I am not arguing I want to learn, if there is a better way than the one I am using now I want to know.

XDCAM is MPEG2 GOP and the EX variant is wrapped in MP4.
Certainly ProRes is larger but IntraFrame is only part of the picture. After all DV is IntraFrame as well and it's smaller than XDCAM.
ProRes is not lossless like 8 or 10 bit Uncompressed but it's actually smaller (more efficient) and for most practical purposes functions almost like lossless in that in can hold up well to compositing and grading and other mild generational degradation.

XDCAM EX BPAV(MP4) is the same size and XDCAM EX MOV. The former though can easily be used natively in some systems or rewrapped for other systems that use MXF or MOV.

XDCAM EX MOV does not have the same flexibility of the above although one can certainly work around it. Rewrapping XDCAM EX MOV back to BPAV MP4 or to MXF is NOT readily available.

An XDCAM EX BPAV folder is the same size as an XDCAM EX MOV so both can take the same storage space. In fact, Sony ClipBrowser allows BPAV to be split and remain viable for recording to such things as Blu-ray. Key though is that XDCAM EX BPAV is far more portable, easy to port, to something any NLE can use . . . and these days many more NLEs can handle the MP4 contained within directly. That is NOT the case with XDCAM EX MOV. As noted above I think Apple themselves may move in that direction with Final Cut Studio.

A BPAV folder is not more complex than the cassette that holds a DV tape for Beta tape. You don't need to think about or otherwise investigate the contents. Given what ClipBrowser shows, there's metadata in there that does not go into the MOV though. Going forward the BPAV will be even easier to handle if Apple supports it directly (not that that will mean a change in support for XDCAM EX MOV).

Personally, being future thinking in my archiving, I want a container (BPAV) that I can hand to any client, use on a future project on a future NLE, without risking a potential future constraint that may come to XDCAM EX MOV. To me, that's the main point of an "archive."

Often enough I have to pull something from projects over 5 years old. 5 years is a lifetime in Computer and NLE worlds. I am every much an FCS "fanboi" having used it for 10 years, but also having used Avid 10 years before that . . . along with Adobe Premiere, Radius EditDV, CMX 6000 NLE and a couple of others in there. I want to know that 5 years from now I know I can EASILY use the material I've shot 2 and a half years ago when I bought my EX1.

In particular EX in MOV wrapper is such a proprietary combination, so not universal, that some future series of turns or decisions at Apple and Sony can impact that specific combination.

I just don't see any advantages to EX MOV over EX BPAV and I see many current and future disadvantages to EX MOV.

Additionally I think Apple might be looking at something more efficient that ProRes to come up with a more efficient codec so that there may be something to convert things to that will be both space efficient and portable. That MIGHT be an alternative if it happens.

Craig, I am looking at the big picture. Surely you are not suggesting I take all my files from different cameras as well as all my legacy stock, and convert it to XDcam. Remember XDcam is only a part of the files I work with.

As far as meta data is concerned. I don't personally use it. It does not tell me what filters, lighting etc. I was using. OK so it tells me the profile, but that is not very important to me at least. The lighting is far more important than profile in my opinion.

I just like to have all my different shots be the same easy to deal with format, and not to have to launch a special app or plugin just to deal with some of my files. To me the BPAV system is a pain, and I think it would be great if QT could deal directly with the mp4's. Like the files from the Canon 5DmkII etc.

I have a searchable DB I created myself and it contains the pertinent data I want about each shoot, or stock shot I have. It links to the files on a number of different (and redundant) HD's and optical discs. To use my files I just plug the HD in and import the ready to work with file into M100, AE, FCP or whatever I need to work on it. I don't want to have to launch another app just to rewrap every XDcam file. I want all my files to be ready to work on.

No, camera masters should be archived in their original form to maximize portability going into the future. The 2nd choice would be a universal codec (should such exist) that does not result in loss of resolution.

It will tell you camera number, good for tracing issues.
Lens model which can be important if you need to match that in the future.
Acquisition Date
Gamma mode you used as part of Picture Profile.
Gain
Color Temperature setting
Shutter settings
ND Filter setting
CC Filter
F stop
Zoom setting
Focus indicator

But if you ever need to use anything in an non FCS program you may well need plugins or some way to convert that EX XDCAM MOV file.

Example of some of the EX metadata attached.

So basically the meta information is a load of stuff that I can't see a use for? Ergo it isn't worth preserving.

I totally agree with Graig ... if you have some important shot, just keep the original masters (BPAV folder) in a safe place... it is your master "tape" no matter what platform or NLE you use today or in the future...
with DV tapes we used to "keep" valuable original tapes and "capture" or transcode files in order to work (mov in MAC or AVI in PC) .... I feel the same with EX original masters .... my 2 cents...

Personally I find that metadata not important.

I can always tell the camera used from the file name. And I almost always use TOD as my TC, jam synced to all the cams. And I slate all pertinent data on each shoot and then use my database to put in all the data I want. This makes for really easy retrieval years later.

As I mentioned I use the NanoFlash most of the time now when shooting. The really nice thing about this is that you decide in the NanoFlash menu if you want a mxf or .mov wrapper. And it uses the XDcam codec from Sony, or an intraframe codec. I use the 100mb/sec XD codec most of the time and it is a very nice timesaver not to have to rewrap after shooting. Just copy the files.

I don't understand what Craig has against keeping all your archives and projects in one format. In my studio this makes a great deal of sense. I guess if most of your work is passed on to other facilities the BPAV makes sense.

I keep digitized copies of all my old Umatic, BetaSP, DVcam, DVCpro etc. files. Some going back to the 80's. The tapes from then have lost a lot of their quality. But the digital files are in great shape, and I can retrieve them instantly with no need to digitize over again. That is what I love about the digital domain. I am actually considering trashing a lot of old tape as the video has faded so badly. They take up a lot of space, especially the old 3/4 Umatic..

So I am just making the case for a Database system linked to digital files that can be viewed instantly using in my case QT. And it is no trouble at all to hand off footage to other production houses. I do this from time to time, most of the time to TV stations that need some of my clients footage and then I simply put it on tape, DVD, HD or upload to their FTP site in whatever format they want usually h.264 these days. Naturally I charge for this and my clients love this service.

I just don't understand what is wrong with this approach. I sell my footage on Getty and they want ProRes these days, so that is how I deliver to them. The same format I use, there must be a reason they use this format. They don't want me to send BPAVs.

As noted before, Pro Res is the one thing that trumps others as a "universal" codec. I suspect Apple is about to improve on that.

That metadata is important for matching shoot conditions. When a client wants to update a 5 year old project with a few new inserts I can match the shoot conditions. The above metadata tells me stuff I could never match by eye years later. Things such as DOF, color temp and how I got there (3200k will look different with preset vs white balance card). It tells me the gamma setting I used. I can get close to the look of a shot so the insert doesn't look out of place.

When my clients ask for master clips, ProRes is too large, EX MOV is not compatible, BPAV gives them complete flexibility. EX MOV is fine for FCP users but most of my clients are not. Corporate clients are primarily on Windows (as is most of the world FWIW). I can move from BPAV to ProRes just as easily as from EX MOV to ProRes. I see many advantages to BPAV and very few for EX MOV. It's the reason most NLEs now use native files in native wrappers and why I believe Apple/FCP will soon follow suit.

So Craig how would you handle NanoFlash files?

Also I really doubt that I will be using the same cameras 5 years from now, I now use mostly LED and fluorescent lights, I used to use HMI and tungsten. I really have very little problem matching old footage with new. People age too. Many times I update a corporate video and the spokes person has changed as well as the company logo. I update legacy projects all the time mixing BetaSP with XDcam from the NanoFlash and currently XD EXcams.

I think in the near future we will all be using large chip cameras, hopefully with RAW codec, that will blow todays cameras out of the water, much better low light and greater dynamic range, just like the XDcams blew the old formats (DVcam DV HDV etc.) away. I think the 5DmkII started a revolution, it will take a while but it is coming. And I also believe QT will support all these new codecs.