My workflow is that on return from a shoot, I copy the cards, using ClipBrowser, to a project directory on an external hard disc. I then review the clips and add logging metadata to the copy. Finally, using XDCam Transfer, I pull the clips I want into FCP.

XDCAM Transfer creates an mp4 (and xml) file for each of the clips imported, so this workflow means I end up with 2 copies of each clip. I am wondering whether to delete, or possibly archive, the older BPAV folders and wondered what anyone else does.

Cheers,

Nick

I would and do keep the BPAVs as this is the only way you can use the material with other systems in the future.

Typically I just burn each stick's BPAV folder onto DVD-DL (as I mostly use 8GB SXS, so it's nice and easy). That's how I archive the BPAVs but have my working files sitting on RAID 0's (and clones on multiple non-raided back-up drives), mainly as .movs (as I've been using a Mac workflow).

You're spot on Bruce - in the last 2 years I've been using a Mac Pro and MBP workflow. Now, as of this week, I ALSO have a highly capable new i7 Windows 7 PC with Vegas 9E 64-Bit workflow for XDCAM EX work as well.... So, my huge EX3 shot archive is fully accessible on both these platforms - which it certainly would not have been (easily) if I'd not kept the BPAVs!

You never know what you'll do in the future, but by keeping at least one clone of your archive backed up as BPAVs keeps your NLE options fully open.

Think of the BPAVs as your negatives.

...think your BPAV folders as your original master "tapes".. if you have valuable material filmed, just keep your BPAVs ("tapes") in some safe place .... I also have Vegas, which do not need any transcoding of the BPAV files to edit them....

good luck

...think your BPAV folders as your original master "tapes".. if you have valuable material filmed, just keep your BPAVs ("tapes") in some safe place ....This is good advice - we also treat BPAV folders - regardless of what media they are stored on as the "master field tapes".

We keep all BPAV folders in separate projects folders. For example, 'New York Shoot June 2009" would have folders labeled Card 1, Card 2, Card 3, etc... and inside would be the corresponding BPAV folder.

We are near-chiving to a 40TB RAID-6 array right now but will begin offloading some of the older project folders to LTO-5 data tapes for long term shelf storage. LTO-5 tapes are around $100 with capacity of 1.5TB each. The challenge is less about the storage and more about the organization.

...think your BPAV folders as your original master "tapes".. if you have valuable material filmed, just keep your BPAVs ("tapes") in some safe place ...

That's my notion too.
If it is unclear what the "perfect" archiving method might be, it's easy enough to just back-up all the BAPVs on cheap external drives. When the "perfect" solution does finally cross your radar, you can transfer at that time.
And... there's always Murphy's Law- if you back them up, you will never need them; the minute you don't back-up, some disaster you've never even dreamed of will happen and you'll be naked.
How many times I've been burned by saying "No way.. I'll never need this file/clip/format/project/whatever again..."

...think your BPAV folders as your original master "tapes"..
good luck

It is better to avoid nested folder structures that aren't conveniently read by lots of software. The BPAVs are only masters if the software to read them continues to exist. Need to transfer a few clips to someone else? BPAV is a pain. My major beef with BPAV and Sony software is that it is much faster to preview and delete clips with XP or OSX with VLC Media Player and delete.

Though, yes. I keep the BPAVs intact. I thought they were large until I started using the Nanoflash for higher bitrate recording.

OK, sorry I am going against the tide here. And this is just my way of dealing with archiving.

I don't keep the BPAV files. I use XDcam Transfer to make .movs. And I archive those on at least 2 HD's in the field on long shoots, or in the studio at the end of the day. I also make BluRay or DVD's of all the clips.

The discs are stored in different buildings and the BluRays/DVD's in a very safe place.

Now I only use Macs in my suites, most of my projects are edited by me. I do give producers who request BPAV's the folders but this is rare for me.

I also shoot a lot with the 5DmkII and 7D as well as using a NanoFlash often on the XDcams. So it only makes sense to treat every clip the same. You cant use clip browser to look at 5DmkII clips.

I use FCP for organizing and viewing the clips. I also have a separate Database program I wrote that keeps track of all my clips including thumbnails, it is keyword searchable and shows what BluRay and discs and folder the clips are in. I can locate and load clips onto my RAID L5's in the edit suites this way very quickly. I don't use FCP for editing much, too slow, we use M100, no rendering, incredible fast NLE. All the clips that have been picked in FCP get transcoded into ProRes422 and dropped into M100. I find this system very fast and efficient.

If I have to transfer footage to a TV station or a producer on a piece about one of our clients. I usually just transcode to the format they need usually H.264. And send a HD or DVD. We do this often for our clients. We do a lot of tourism related shooting here in NH. We even do a fair amount of FTP uploads for news features. Everyone is very happy with this system.

Below is a screen shot of the DB.

Once again I am not preaching everyone adopt this system. This is just my way of dealing with NO-tape footage. We still use a bit of tape as well. This is way faster. And any tape that gets digitized now gets entered into this archive system for easy retrieval and multi location backup.

Another vote for Olof here. I do the same.

For me, BPAVs are 'exposed film' which need 'developing'. I have no use for storing exposed yet undeveloped film. I want it processed, which - for me, and your mileage may vary - is something that other applications in my ecosystem will open up and deal with as single entities.

So, I enter user metadata into XDCAM Transfer like 'interviewee: name, job title, affiliation' or specific location info, so when the BPAV files get 'developed' they hold that metadata in each file that is viewable in FCP and other apps. They also get given a meaningful name so they can be identified by text search tools.

The BPAV files are then deleted. They no longer exist in a meaningful sense. They are merely an instance of the XDCAM Transfer output that are temporary and quickly forgotten.

* * * WAIT A MOMENT * * *

Yes, folks. That's my workflow, not yours, and your situation is not my situation, so before you tell me that I am an unclean sinner for treating BPAV material with such contempt (you know who you are), I have another workflow when working as a cameraman or as a wrangler.

I had a job last year to record some interviews that would be required in ten years time, maybe more. Of course that footage is backed up as BPAV as well as QT, multiple copies vaulted away. It may be contributing to my pension, so I treat it as such.

I have regular shooting work where I hand over footage. In the olden days, I just handed over cassettes, but as we migrate to tapless workflow, it means handing over 'edit ready' stuff. QuickTime movies are not necessarily 'edit ready' to many Avid editors. So I supply BPAV copies plus boilerplate on what to do next - though I've changed that for some clients.

SD clients like AVI files, so even though I have implored them to see the benefits of HD, I'm now happy to shoot SD and give them AVIs - which work in FCP too, BTW. Such clients wouldn't know what to do with BPAVs, and when I try to explain, they go into 'TILT' mode.

All these client situations are NOT 'broadcast' clients, they have their own workflows.

And - distant rumbling of seething resentment - I am not going to be the repository of all my clients' data unless they pay for it, which almost all I work with DON'T. I may do that as an expedited service, but clients want their footage on a disk for themselves like they'd get cassettes, and BPAVs are not something they can easily understand or deal with, look at, hand over to somebody else, or otherwise interact with - which QT movs are - difficult as they are to get to work well on a pC - capable of.

Blogged it here: EX1R SD Workflow – tales from the front line Travelling Matt (http://mattdavis.wordpress.com/2010/01/12/ex1r-sd-workflow-tales-from-the-front-line/)

Originally, I would just use the Sony transfer tool to convert the BPAV on the flash card to mxf files on the disk. In an early Sony ClipBrowser revision, there a bug with 720p60 conversion and did not include the audio. Luckily, I only lost *one* entire video session. (nauseating chuckle) Since then, I always keep at least two instances of the BPAV file stored on separate devices before formatting the flash cards.

Below is a screen shot of the DB.

Olof, I just love the screen shot of the DB! I appreciate you sending me the info on getting it, but I'm on PC. Oh that I knew how to do this myself! I tried with MS Access, but no luck (probably user ignorance). Looks perfect though. Just what I've been trying to accomplish.

Back to the topic, I just pull the EX1's MP4 video file ONLY out of each folder and erase the rest. I am using different cameras often and they all play very well together in a Premiere CS5 workflow (Canon A1 on a Sony MRC1K, Canon 7D and the EX).

I have a special naming format for each file that corresponds to a detailed spreadsheet that tracks all the clips. I've never even used the BPAV files.

Just me...

As anyone can see, everyone has a different workflow. However, some of these guys have been doing this for years so they know what works best for them. With the way FCP is changing...I mean not changing and the progress Avid and Adobe have made, I would advise to keep the BPAV for the newer XDCAM users.

Furthermore, your type of work & clients also dictate your workflow. For example, I have been approached by some people who want to pay for some of my material that I am recording for someone else (live events). If I want their business, then I need to make certain to keep the BPAV in case their workflow is different from mine. This is hypothetical because I use Premiere Pro CS5 so I always keep and archive the BPAV.

AFAIK, only the original MP4 in the BPAV folder can have Flash Band removal applied so there is another reason to keep the BPAV.

BPAV is the camera master. Not keeping it is a disaster waiting to happen.
Just wait 'till you need to hand to a client or other post production workflow that needs it even if it's just to rewrap to something else. For those who say "never happen" I'll say the first time it does it will cost you time, money, possibly the client because of the previous two.

Some notes to add here:

FCP will change, there is according to Steve Jobs a new version that will blow our socks off in the pipelines.... just how long that pipeline may be????? This may (or may not) work directly with BPAV folders.

I believe Sony are working on a unified XDCAM viewer/logger/transfer tool that will ultimately replace the separate XDCAM transfer tool, clip browser and various XDCAM HD PDZ viewers etc. This new tool should incorporate all the clip list editing, database, export and logging functions of all these tools in a single package that will work with EX and XDCAM HD. It will be cross platform compatible so Mac and PC. To use this you will need to keep your BPAV folders. It should be a great tool.

The BPAV folders contain metadata that does not get included in the .mov's. It's likely that forward planning metadata and extra metadata will get added to XDCAM EX in the future.

.Mov's containing XDCAM material are a very specific file, tailored for FCP. Even Premiere CS5 on a Mac doesn't like them.

I always keep at least one copy of the BPAV folders. One way of working is to keep one BPAV copy and one .mov copy. Should you loose one or the other you can (with a little work) re-create the other. .mov's are easy to create from a BPAV. BPAV's take more work as you need to first create mp4's from FCP and then import those in to a new folder using clip browser.

My workflow differs again to everyone here...

When shooting, I give the clips a title relating to the client/agency/production company.

So I have things like PON, DHG, RED, MC1, I have a notepad in the EX1 bag where I keep a 'diary' of every card. This show something like '105 15/07/10 Pennells Garden Centre 4 DHG_1157 - DHG_1241 Card 6'

So it was my 105th card, shot on the 15th of July, the 4th time I've filmed at Pennells, clip numbers 1157 to 1241 and recorded on card 6. I shoot TOD timecode so sometimes a client will already be making a rough log as we shoot. The diary means that the next time I work with a particular client I can set the camera to number the clips from where they left of the last time. (I also keep a log of how many times I've used each card and any camera information like firmware updates, new kit, problems etc).

The BPAV gets copied into a folder named '105' using drag and drop on OSX to two large USB drives.
I use the 'compare' CRC function in Toast to check the integrity of these copies, and the two drives are kept at different offices.

From one of these drives I can import the footage into FCP using log and transfer where shows in the FCP bin as Reel 105. The project folder in Capture Scratch will end up being copied to two more large USB drives at some point, along with the imported Quicktimes any graphic elements, scripts, etc. etc. This means reinstating the media for amends or future work is fairly painless and I still have the BPAVs for future use.

With a new job I'm just starting, the client keeps one of the BPAV backup drives and uses Clip Browser on a PC to transcribe the footage. He brings the drive in a flightcase and I give him new footage as we shoot it. At some point I hope to have another cameraman on the job, I'll continue using MC1 and he will use MC2 with his own 'diary' of clips.

It can take a little bit of work to keep the archives up to date, but the drives are cheap and seem very reliable, I think as long as you power them up once in a while for an hour or so they'll last for some time. If something fails you have another copy to quickly create a backup, and you can always take the physical drive out to try and repair it.

Duncan.

I don't keep the BPAV folders. I import to the mac with XDCAM Transfer, trim and name the clips with QT Pro, tweak them in AE and then back up to external disks.

I don't keep the BPAV folders. I import to the mac with XDCAM Transfer, trim and name the clips with QT Pro, tweak them in AE and then back up to external disks.

But you'll get burned the moment you have to hand them to a non Final Cut Pro person whether editor or client.

A few things.

1. If you do need the BPAV structure in the future, just format a new card in the EX and export back to XDCAM with FCP. You can recreate a BPAV structure if you really want to if you deleted the original, with no recompression of material.

2. ProRes codec is now available for free for the PC. You could just export for that and hand it over to a client, along with a multitude of other types.

3. Sometimes I find the idea of keeping the BPAV structure 'just in case' to be like keeping junk in your garage just in case. How much milage you get will depend precisely upon how truly realistic it is that you'll need to hand off to someone else on a different system after you have imported and edited it all. Lets be realistic, for most of us that will be never. And on top of that you can always rewrap to MP4 with the XDCAM exporter, or even export/rewrap as the 1920x1080 MXF format.

Look, there is always a way. That's the whole advantage of using files. The BPAV structure is only important if the editing system in question can only take XDCAM footage in using some sort of dedicated importer. But if that NLE can import files and have them dragged natively to the timeline I'm not sure what the issue with deleting the BPAV structure is. Even if the NLE in question is pathetic, you can always find a way to do it.

Writing an XDCAM EX .mov from FCP out as an mp4 is simple, but each clips has to be done individually unless you are prepared to put them all in the timeline and then export in 12 minute chunks. You also have the issue of having to re-split long clips back into 4Gb chunks if your writing to the camera or other Fat32 media. If you are looking at a big project with 100's of clips this would be a real drag.

I've had to walk at least 3 production companies through this whole process as for various reasons they had to move from FCP to other platforms. It's not pretty and they could have saved themselves a lot of agro by keeping copies of the BPAV's.

But you'll get burned the moment you have to hand them to a non Final Cut Pro person whether editor or client.

That's tough though isn't it? I'm the production company, I store the files. If someone else wants the files they can either have them in the format I choose to store them in or pay me to convert them to a format of their choice. If it's my project I make the calls. I archive the complete project (to HD and Blu-Ray or DVD) and also the MOV files (again HD & Blu-Ray or DVD). The movs are instantly playable and editable. The only reason you'd need the BPAV's at any stage would be either future use on another platform or to hand over to someone else. But this is an easy problem to fix with the 3rd party exporters that will rewrap the files for you.

If shooting for a 3rd party then yes, keep the BPAV's, but for me there is no benefit. I can always rewrap back to mp4 if I really need to thanks to the 3rd party exporters available (Calibrated Software isn't it?).

If I ever did move to another NLE (unlikely) then I'd do what I did with Premiere on the PC and keep a legacy system running to edit the older projects.

I've always archived my DV tapes to HD and DVD as well. As a result I have never had to go back to a tape after the original transfer. They all sit there on the shelf looking pretty, but only played the once. And that's what would happen with the BPAV's, they'd never be used.

Marcus, your fundamental mistake, IMHO, is you're trying to predict the future.

XDCAM EX MOV is specific to FCP and to FCP at a specific point in time. The change in workflow ahead may not be of your choosing. The future scenarios are far too broad and can happen more quickly than you can anticipate.

There is metadata in the BPAV that FCP currently doesn't use. That may change. If so, you've locked yourself out of that.

BPAV is "universal" (can be changed to whatever) and to charge clients for something that shouldn't be necessary is a business practice that can come back to bite you.

You are "a" not "the" production company and a client at any future point may decide that they want you to work on "x" part of the project while another, with different gear and expertise, will work on "y" part of the project.

Will you generally find away around most "movement" issues, probably, is that the most time and cost efficient, no. Read Alister's post for example. I could certainly add many more.

There's a Quicktime codec available for both the PC and the Mac by Calibrated Software that allows EX and XDCAM HD and 422 .mov files to be read by Quicktime on either system without having FCP installed.

I'm just not convinced that keeping the BPAV's is essential at all. What do people do with NanoFlash recorded files when they record to native .mov format? You could record to MXF with the nano too, but you can also rewrap your existing EX files to MXF too.

The way I see it is that the NanoFlash lets you record right from the beginning without the BPAV structure. So why not keep your EX files in the same way that the Nano gives them to you from the beginning?

Simon
As I've said there is certainly ways to rewrap. Do you wand to force someone to buy a plugin though? Do you want to spend the time rewrapping files or explaining how to do this to clients?
I tend to take what I believe is the easy path. BPAV is no larger than the rewrapped MOV. BPAV can get you anywhere you need to go quickly. MOV is fraught with obstacles great and small.

I do not want to force clients or other production houses to buy a specific plugin.
I do not want to invest time rewrapping or billing clients for rewraps that wouldn't have to be done otherwise.
I do not want to lose the metadata that is thrown out in the rewrap to MOV as it may server a future purpose to someone.

Not only does NanoFlash offer MOV recording so does JVC HM series. Yes it can be an immediate convenience recording to MOV directly but that's not what the EX cameras do. Use BPAV to one's advantage, I say

Personally I suspect that their are disadvantages in the metadata use from camera file sources that Apple might account for in the next FCS.

File portability should be an advantage and not something one has to tie up resources working around. I think retaining that advantage is important.

Calibrated also makes a plugin that allows one to use the BPAV/MP4 directly in Final Cut Pro. Disclaimer, I was a beta tester for it. I wouldn't be surprised if Apple including such a feature in the future (along with AVCHD direct support). There's a big advantage having ingested files that link directly to camera master files. That fact that we have to debate the use of MOV shows it's disadvantage. The files take some work to make portable to other workflows.

But you'll get burned the moment you have to hand them to a non Final Cut Pro person whether editor or client.

I don't think QT will go away any time soon, and as long as it's around I suspect I'll be able to find an acceptable way to transcode the files for whoever might need them. We don't all have the same needs when it comes to workflow. It's not one size fits all. I'm not suggesting that there's no point in anyone keeping BPAVs, just that I personally have no need for them.

If Quicktime files become obsolete, the camera master issue would become just one of a number of other issues that we would all hit.

What about stock footage you buy? Any animations or screen recordings in your production? How about other footage you may have that only exists as Quicktime files? In the week I have to transfer some VHS and I'll store that footage as Quicktime. The tape will go back so then I'll left with the file and nothing else.

And what about the fact I store finished productions as Quicktime files as the days of dubbing back to tape are long gone?

Believe me, if Quicktime support vanished you are going to have alot more issues than just your camera masters not working.

There is metadata in the BPAV that FCP currently doesn't use. That may change. If so, you've locked yourself out of that.



I'm guessing it's just logging information in which case I have a system in place already for this that works perfectly well and fully integrates with my non XDCAM footage (as I've previously pointed out I've been storing all my footage as files for 5 years now).

It is my personal opinion that BPAV folders will go away long before QT.

Just my guess.

I keep BPAV's for a while but all my main archives are .movs.

I would say anyone starting out should absolutely keep and backup the BPAV's until they decide the best system for handling their particular workflow.

I think we will see many more systems like P2 mp4 etc in the following years. I am personally rooting for RAW video files as media becomes faster and cheaper.

And I also bet that QT will be compatible with all of them, at least in the next decade or two.

For me it makes sense to keep everything as .movs, weather it is digitized BetaSP, DVcam, HDV or HDcam etc. etc. or whatever solid state comes down the pike. For me it is a great standard.

Marcus if you look at the BPAV in ClipBrowser or the actual XML and other files, you'll see information about camera settings and other information that is thrown out when the file is rewrapped to MOV.

EX XDCAM codec is not standard part of Quicktime. It is part of Final Cut Studio install. The only other way to get EX XDCAM MOV is to buy it from CalibratedSoftware.

There is no easy way to wrap EX XDCAM MOV back to .mp4 or .mxf and that's especially so without Final Cut Studio.

If you look at the many NLEs, it's actually the .mp4 which is "the standard" not .mov. Premiere Pro, Vegas and I believe Edius handle the .mp4 directly.I believe Avid MC5 with AMA may handle it as well. I suspect FCS will handle the .mp4 directly as well in the next upgrade.

Basically EX XDCAM .mp4 direct support is spreading as the standard and .mov, as proprietary will be "legacy" if FCP moves to .mp4 direct support.

What I like about QT, is that it is so inclusive. It is a wrapper for lots of different codecs, like mp2, mp4, mpg as well as some really great loss less codecs built in.

I like the fact that if you have the latest version of FCP, Flip for mac and Sorenson Squeeze there is almost no file you can't read or compress to Flash or whatever.

For me it is really convenient to archive in one format. I am on a Mac but it has never been a problem to transcode to Windows file formats. My main editor is M100 and it has its own codecs but they are free downloads both for Macs and PC's, as long as you have QT installed.

I don't just work with XDcam files. So it makes sense for me to use a common file format in my archives.

But I am not trying to say that everyone should use this approach, just that it is my way of dealing with my video project, and I have been doing that digitally since the early 90's, and I can still open QT files from back then, and use them in my current projects, and export them to any platform a client may require.

Olof, I think Apple was banking on the universality of the Quicktime wrapper and I certainly would have liked it too but I think Apple may be rethinking that. I think they are looking for "universal" codec as may be happening with Apple ProRes. Even though "everything" can play ProRes MOV (free decode for windows and certainly on Macs even without Final Cut) that hasn't happened overall.

The key difference is the unique nature of EX XDCAM MOV which is specific to ONE NLE on the Mac only. The codec is not a standard part of Quicktime nor is it plus the MOV wrapper standard to much of anything else.

I think Apple held out a long time staying with the idea that everything in FCS should be wrapped as MOV. Just a hunch but I'm guessing Apple's about to give on that (next year) with FCS4. If anything Calibrated Software proves that a plugin can allow native codecs in their native wrappers to work with FCP.

You can't even quite compare to AVCHD. That AVCHD is likely going to ProRes and these days that ProRes file can probably go to many other NLEs.

You'll certainly still be able to play EX XDCAM MOV but given it's nature it's the "odd bird" and I don't think it's the way forward.

Of course you can always convert the EX XDCAM to ProRes but, as with AVCHD, that process is an Achilles Heal in that it takes time and the files are significantly larger. Hence I suspect Apple will be supporting native codec, native wrapper. Apple is certainly aware of the ProRes file size issue as one of the key changes between FCS2 and FCS3 was the addition of ProResLT and ProResProxy.

When I hear rumor talk of Apple working on a new codec it may be to work out that ProRes file size issue in their quest for their "universal" codec (not to be confused with making MOV a universal wrapper).

Craig,

The way I understand it is that XDcam is really an MPEG2 fileformat.

ProRes is Inra frame therefore the larger file size and easier editing, it is also much less lossy over generations. Am I wrong Craig?

Also HD/BluRay storage is really cheap these days, that is how I archive, I dont mind the increased size.

Right now XDcam footage is less than 30% of the file format I use for acquisition for projects I work on. I am trying to use a uniform format for all kinds of media, and so far QT has been it.

I see the BPAV folder as an added complexity I don't need.

Craig I am not arguing I want to learn, if there is a better way than the one I am using now I want to know.

XDCAM is MPEG2 GOP and the EX variant is wrapped in MP4.
Certainly ProRes is larger but IntraFrame is only part of the picture. After all DV is IntraFrame as well and it's smaller than XDCAM.
ProRes is not lossless like 8 or 10 bit Uncompressed but it's actually smaller (more efficient) and for most practical purposes functions almost like lossless in that in can hold up well to compositing and grading and other mild generational degradation.

XDCAM EX BPAV(MP4) is the same size and XDCAM EX MOV. The former though can easily be used natively in some systems or rewrapped for other systems that use MXF or MOV.

XDCAM EX MOV does not have the same flexibility of the above although one can certainly work around it. Rewrapping XDCAM EX MOV back to BPAV MP4 or to MXF is NOT readily available.

An XDCAM EX BPAV folder is the same size as an XDCAM EX MOV so both can take the same storage space. In fact, Sony ClipBrowser allows BPAV to be split and remain viable for recording to such things as Blu-ray. Key though is that XDCAM EX BPAV is far more portable, easy to port, to something any NLE can use . . . and these days many more NLEs can handle the MP4 contained within directly. That is NOT the case with XDCAM EX MOV. As noted above I think Apple themselves may move in that direction with Final Cut Studio.

A BPAV folder is not more complex than the cassette that holds a DV tape for Beta tape. You don't need to think about or otherwise investigate the contents. Given what ClipBrowser shows, there's metadata in there that does not go into the MOV though. Going forward the BPAV will be even easier to handle if Apple supports it directly (not that that will mean a change in support for XDCAM EX MOV).

Personally, being future thinking in my archiving, I want a container (BPAV) that I can hand to any client, use on a future project on a future NLE, without risking a potential future constraint that may come to XDCAM EX MOV. To me, that's the main point of an "archive."

Often enough I have to pull something from projects over 5 years old. 5 years is a lifetime in Computer and NLE worlds. I am every much an FCS "fanboi" having used it for 10 years, but also having used Avid 10 years before that . . . along with Adobe Premiere, Radius EditDV, CMX 6000 NLE and a couple of others in there. I want to know that 5 years from now I know I can EASILY use the material I've shot 2 and a half years ago when I bought my EX1.

In particular EX in MOV wrapper is such a proprietary combination, so not universal, that some future series of turns or decisions at Apple and Sony can impact that specific combination.

I just don't see any advantages to EX MOV over EX BPAV and I see many current and future disadvantages to EX MOV.

Additionally I think Apple might be looking at something more efficient that ProRes to come up with a more efficient codec so that there may be something to convert things to that will be both space efficient and portable. That MIGHT be an alternative if it happens.

Craig, I am looking at the big picture. Surely you are not suggesting I take all my files from different cameras as well as all my legacy stock, and convert it to XDcam. Remember XDcam is only a part of the files I work with.

As far as meta data is concerned. I don't personally use it. It does not tell me what filters, lighting etc. I was using. OK so it tells me the profile, but that is not very important to me at least. The lighting is far more important than profile in my opinion.

I just like to have all my different shots be the same easy to deal with format, and not to have to launch a special app or plugin just to deal with some of my files. To me the BPAV system is a pain, and I think it would be great if QT could deal directly with the mp4's. Like the files from the Canon 5DmkII etc.

I have a searchable DB I created myself and it contains the pertinent data I want about each shoot, or stock shot I have. It links to the files on a number of different (and redundant) HD's and optical discs. To use my files I just plug the HD in and import the ready to work with file into M100, AE, FCP or whatever I need to work on it. I don't want to have to launch another app just to rewrap every XDcam file. I want all my files to be ready to work on.

Surely you are not suggesting I take all my files from different cameras as well as all my legacy stock, and convert it to XDcam.
No, camera masters should be archived in their original form to maximize portability going into the future. The 2nd choice would be a universal codec (should such exist) that does not result in loss of resolution.

I don't personally use it. It does not tell me what filters, lighting etc. I was using.
It will tell you camera number, good for tracing issues.
Lens model which can be important if you need to match that in the future.
Acquisition Date
Gamma mode you used as part of Picture Profile.
Gain
Color Temperature setting
Shutter settings
ND Filter setting
CC Filter
F stop
Zoom setting
Focus indicator

I just like to have all my different shots be the same easy to deal with format, and not to have to launch a special app or plugin just to deal with some of my files.
But if you ever need to use anything in an non FCS program you may well need plugins or some way to convert that EX XDCAM MOV file.

Example of some of the EX metadata attached.

So basically the meta information is a load of stuff that I can't see a use for? Ergo it isn't worth preserving.

I totally agree with Graig ... if you have some important shot, just keep the original masters (BPAV folder) in a safe place... it is your master "tape" no matter what platform or NLE you use today or in the future...
with DV tapes we used to "keep" valuable original tapes and "capture" or transcode files in order to work (mov in MAC or AVI in PC) .... I feel the same with EX original masters .... my 2 cents...

Personally I find that metadata not important.

I can always tell the camera used from the file name. And I almost always use TOD as my TC, jam synced to all the cams. And I slate all pertinent data on each shoot and then use my database to put in all the data I want. This makes for really easy retrieval years later.

As I mentioned I use the NanoFlash most of the time now when shooting. The really nice thing about this is that you decide in the NanoFlash menu if you want a mxf or .mov wrapper. And it uses the XDcam codec from Sony, or an intraframe codec. I use the 100mb/sec XD codec most of the time and it is a very nice timesaver not to have to rewrap after shooting. Just copy the files.

I don't understand what Craig has against keeping all your archives and projects in one format. In my studio this makes a great deal of sense. I guess if most of your work is passed on to other facilities the BPAV makes sense.

I keep digitized copies of all my old Umatic, BetaSP, DVcam, DVCpro etc. files. Some going back to the 80's. The tapes from then have lost a lot of their quality. But the digital files are in great shape, and I can retrieve them instantly with no need to digitize over again. That is what I love about the digital domain. I am actually considering trashing a lot of old tape as the video has faded so badly. They take up a lot of space, especially the old 3/4 Umatic..

So I am just making the case for a Database system linked to digital files that can be viewed instantly using in my case QT. And it is no trouble at all to hand off footage to other production houses. I do this from time to time, most of the time to TV stations that need some of my clients footage and then I simply put it on tape, DVD, HD or upload to their FTP site in whatever format they want usually h.264 these days. Naturally I charge for this and my clients love this service.

I just don't understand what is wrong with this approach. I sell my footage on Getty and they want ProRes these days, so that is how I deliver to them. The same format I use, there must be a reason they use this format. They don't want me to send BPAVs.

As noted before, Pro Res is the one thing that trumps others as a "universal" codec. I suspect Apple is about to improve on that.

That metadata is important for matching shoot conditions. When a client wants to update a 5 year old project with a few new inserts I can match the shoot conditions. The above metadata tells me stuff I could never match by eye years later. Things such as DOF, color temp and how I got there (3200k will look different with preset vs white balance card). It tells me the gamma setting I used. I can get close to the look of a shot so the insert doesn't look out of place.

When my clients ask for master clips, ProRes is too large, EX MOV is not compatible, BPAV gives them complete flexibility. EX MOV is fine for FCP users but most of my clients are not. Corporate clients are primarily on Windows (as is most of the world FWIW). I can move from BPAV to ProRes just as easily as from EX MOV to ProRes. I see many advantages to BPAV and very few for EX MOV. It's the reason most NLEs now use native files in native wrappers and why I believe Apple/FCP will soon follow suit.

So Craig how would you handle NanoFlash files?

Also I really doubt that I will be using the same cameras 5 years from now, I now use mostly LED and fluorescent lights, I used to use HMI and tungsten. I really have very little problem matching old footage with new. People age too. Many times I update a corporate video and the spokes person has changed as well as the company logo. I update legacy projects all the time mixing BetaSP with XDcam from the NanoFlash and currently XD EXcams.

I think in the near future we will all be using large chip cameras, hopefully with RAW codec, that will blow todays cameras out of the water, much better low light and greater dynamic range, just like the XDcams blew the old formats (DVcam DV HDV etc.) away. I think the 5DmkII started a revolution, it will take a while but it is coming. And I also believe QT will support all these new codecs.

NanoFlash I'd keep in whatever native form you recorded them to. That would probably be .mov extension although I'd love to seem them use the more universal .mp4 rather than .mov or .mxf. It's funny how wrappers change. In world full of non standards .mp4 really is becoming universal.

Even Apple has played games with this. Note that .m4v used in video for iDevices is really just .mp4 but calls iTunes for example on opening. You can actually change .m4v to .mp4 as the metadata internal to the wrapper is the same. That can not be done with .mov (it can't be change to .mp4 or .m4v since some of the "internals" are in different places). This is another indicator that Apple is really looking to support .mp4 even if the wrapper extension may appear different.

Yes cameras are going to go to large chips very soon as Sony indicates with their $2000 interchangeable lens camera . . . which records to AVCHD. I'd expect the "pro" version by NAB next year or sooner. It won't be RAW though as the data rates are still too high for most recording media. I can't guess where Sony is going, whether such camera will be AVCHD, AVC Intra or, MPEG 50 4:2:2 or an MPEG I frame format.

Granted there are many ways to match settings are accept that they can't match but the metadata is definitely helpful to me. Even with different cameras it quantifies a lot about the "look" of the shot. In some respects it could even be more critical since that info can help match things. Knowing the narrow nature of the focus and the focal point of the subject can give me an idea where to go to set up a similar shot. Obviously it won't mean anything if you're not trying to do that . . . but I have and I am.

BTW given the movement in cameras and codecs it's all the more reason I think EX MOV is the "odd bird." It's way to proprietary (tied basically to one NLE) for me to consider that "archive."

As to the new codecs, I'm not sure if they'll be "direct" support in Quicktime as you now think of it. I think Apple will move to native support although it may be through the "tricks" that make the system think it's Quicktime as Calibrated Software does.

Craig and Olof: PLEASE just agree to disagree on this subject :) I know Olof has been doing this for decades and he has been successful; thus, he knows what works for him and his company. Craig has been obviously doing this for years and knows what works best for him and his company. It sounds as though Olof will never ever move from FCP whereas it sounds like Craig might but is waiting for FCP 4; thus. his trepidation about QT/MOV and native support.

I have been doing this for exactly 1 year and I don't even have a solidified workflow yet. So far, all I do is download the cards through my Siig expresscard to PCIe adapter, save all cards from one event to a single folder with a naming convention like this: xx VIDEO/2010/August/08-03_National_Night_Out_Cam1
After all cards are downloaded, I copy the new folder to both of my 2TB e-sata drives. Because I use Premiere, I never transcode and always keep the BPAV. Furthermore, I might add an Avid workstation for a 2nd editing workstation so I need to keep all options open.

For edited audio, stock video/audio and images, I copy and paste to another pair of 2TB drives (1 internal and 1 external). For everything else, I use Acronis Backup & Recovery 2010 and I use a "Grandfather" scheme - it backs up 3 days a week, once every week and once every month. It keeps the last 3 daily backups, last 3 weekly backups and last 3 monthly backups. This works great for going back to a prior version of a Premiere or AE project file, which I have only done with AE so far. Also, I have separate backups for my PPro files, AE files and Cinema 4D files.

Like I said before, another benefit to the BPAV folder is being able to use Flash Band removal plus whatever other features might be added to ClipBrowser down the road.

Olof, now that I think about it more, I think Craig is trying to convince you that FCP 4 will move to native support; therefore, you won't need to keep the MOV files because FCP 4 will be able to use the BPAV or MP4 at the very least.

Steve, you bring up some additional good points. ClipBrowser may introduce or improve features. FlashBand is a good example, Imagine, when the time comes to reuse the source copied pre FlashBand, to improve on that.

Regarding FCP, one doesn't know what changes are in store for it on many levels. Think of Media 100's history for example . . . all the ownership changes and some of the derivative systems and features.

Even though my approach is different than Olof (BTW I go back to 2" tape dub days) I think readers may want to see why we've arrived at our different places so they can evaluate their own directions. We've seen that in other parts of this forum where the veterans have significant reasons for very different workflows. Better to know a lot about all the options than not enough.

BTW I generally think it's no fun switching edit systems but I've had to do that at times in my life for reasons out of my control. There may come a time when the hardship of switching may be easier than the hardship of staying. This is especially so when a system goes away. EditDV and Discreet Edit* come to mind (although I never used the latter the possibility had come up). I certainly was THRILLED to go from CMX (how could you ever switch from a leading company) 6000 to this "start-up" called Avid in 1989. About 12 years later I was eyeing FCP closely although it had too many weaknesses at that point. By FCP 3 is was much easier to pry my hands away from Avid. For me, I have learned that "forever" generally only exists in the present tense.

I've sat on my hands whilst lurking on the thread, but have finally jumped in.

Craig and I have 'agreed to disagree' over on Vimeo - we both agree the other person has valid points, but our workflows differ too widely. For my part, I love the way XDCAM Transfer enables me to batch name movie files and embed metadata that shows up in Final Cut Pro - I can find footage using CDfinder, I can see metadata useful to me (who is this interviewee? What is the tummy tag info I need? Who do they work for?). When other shooters give me BPAVs to download using badly setup FTP sites, CalibratedQ comes to the rescue. When shooters give me BPAVs to download with no log notes, slates or even audio cues, I thank them dearly for letting me know they shot at f4 and at a colour temperature of 2960 Kelvin. So, when making a compilation of Mr X, top executive of whateverco, it's often the time it was shot that helps me, late at night, piece together who is who and what they're called.

Now, IF they had used XDCAM Transfer, AND they had typed in the info, I would get metadata that is USEFUL to... AN EDITOR. Heard of these guys? They get loads of footage flung at them, and as time goes by, the concept of 'logs' and 'slates' seem to have been forgotten because of 'metadata'.

But I will mention this: Craig is right to keep an eye on the future, as the future has this really tricky habit of becoming the past really quickly. I have material from the mid 1990s and earlier, stored as Macromedia Director 4 and (shudder) Accelerator movies that cannot be viewed now. They weren't kept up to date.

I worry slightly less about the movies I have in QuickTime format as I do worry about the edits I have in FCP1 format. Movies I made in the early 1990s are in MPEG1 format and they still play. The various format files (PCTS?) that some were archived in don't work now - and the cost of conversion aren't worth the cost of getting a consultant to work out how to unpick them, sadly.

The trouble is, in 30-50 years time, who will be able to unpick a FCP file, or even a BPAV or MOV for that matter? I foresee having to translate formats every 10-15 years on proper 'archive' stuff. PhotoJPEG QT was good for a while, but what will take over from that?

Meanwhile, I will remain naughty and stick with the 'keep your MOV, bin your BPAV - unless it's worth something in 10-15 years, in which case back it up in lots of formats' camp. And don't forget to make an XML EDL of your FCP edits! :)

Don't mix up MP4 and mp4, they are slightly different things.

.mp4 is an extension normally used on video files encoded using the mpeg4 codec also known as H264

.MP4 is a multimedia container defined by the MPEG group (MPEG 4 Part 14) that can be used to simultaneously contain any number of media streams including video, audio and data such as subtitles and still images. It is actually based on the Apple Quick Time .mov container but includes extra information about what is contained within the wrapper. The video and audio streams it can hold may be compressed using almost any kind of mpeg from mpeg 1 to mpeg 4 plus a few others as well. It's similarity to .mov is one of the reasons why it's easy to convert XDCAM EX files to .mov

This makes the MP4 wrapper very versatile and cross platform compatible.