With MXF export files, is it really necessary to keep BPAV files?
 

I am rather new to the tapeless solid state thing, and I am loving it. But I am having a hard time understanding why I would need to keep the BPAV files after I have exported them to MXF files. It eats up a lot of storage, and I know storage is getting cheaper by the day. I would appreciate some good reasons why I should keep or dump the BPAVs.

BVAP are your original files, and one HD and one optical copy should be made as a backup.
Otherwise if something happens, and one day it will, you will be out of luck.
Doug J. excellant Vortex media DVD training disks are a must, and will answer most of your questions including this one.

Once I verify my MXF file, I delete the BPAV folders. Absolutely no reason to keep them at all, save for maybe some metadata which the NLEs don't seem to support anyway. I do recommend making backups of the MXF file after conversion and verification though.

If someone can present a solid argument to saving the BPAV folders I'd love to hear it.

Perrone,
It's only the mxf files that are useful to me (Avid Media Composer), but the other day someone needed some of my stuff for a Final Cut Pro edit. I was able to help out, because I had the original BPAV folders. If I had sent this person a copy of my mxf folder, would it have worked? I wondered about this but didn't try it.
Cheers, Malcolm

I'm more worried about losing support for proprietary formats (Sony MP4 and BPAV). At least MXF has more support, though I'm surprised that EX1 MXFs from ClipBrowser aren't read universally.

The MXF files seem to be read by a great many of the common NLEs. I know of none that support the native format. Therefore, the MXFs have far higher value to me than the original files (which are only usable with the clipbrowser software).

But again, if someone has good arguments to the contrary, please present them.

The one caveat to not saving BPAV folders is strictly for AVID users. If you export via CB2 to OPAtom(AAF), Avid will not relink or reimport OPAtom files. You' re only backup capability is via those BPAV files.

I don't see why I need to keep them either. I back up the Quicktimes that are made by the XDCAM transfer tool. I figure all I need are the Quicktimes to edit with. Even if I switch platforms, every edit system can handle Quicktimes. Though I'm open to hear why the BPAVs are needed to be backed up! To me, it's easier to back up the QTs and import QTs when needed to edit. I back up to single layer DVDs. I can divy up the QTs in any order from a 16 GB card to fit on 4 DVDs, to back up a 16GB BPAV seems to me to be more difficult.

I'd sure like to hear if I'm doing it wrong though!

Doesn't keeping the BPAV folder retain the best future compatibility rather than QT (I currently archive both, I'm a FCP user)? If a year or three from now someone wants to use my footage in an Avid or Vegas project (or a future version of FCP for that matter), wouldn't the original camera data be best? I'd love to know if not so as to save some archiving work!

I wish ClipBrowser would preserve the dates.

I agree.
Seems that it would be like saving good JPG's instead of saving the original raw files on DSLR's. IMHO

A jpg is a completely different beast than a camera raw file. No comparison to the MXF/MP4 EX1 files at all.

Personally, I convert to MXF, verify all is well then delete the BPAV folder. I see no reason to keep it. Again, if someone has a GOOD argument to why we should keep it, I too would like to hear it.

No, it's more akin to saving raw files to 32 bit PNG. Exact same files, with the exact same information, but in a more universal format. There is no information lost going from the BPAV folder to MXF.

Does FCP support MXF files? I see no reason to go there, as the quicktime files are my workflow on all platforms, and I can never see FCP not able to support QT. Is there a quality loss in sticking with quicktime vs. MXF?

I don't know that FCP supports the native files.

Here comes help:

Native MXF Editing in Final Cut Pro
Calibrated{Q} MXF Import is a QuickTime plug-in for NATIVE Importing and Editing of MXF Files within Final Cut Pro. Calibrated{Q} MXF Import v1.5 supports the following MXF Files:

Panasonic P2 MXF Files (with separate video and audio tracks seen as 1 file*):
AVC-Intra
DVCProHD
DV50
DV25

Sony XDCAM:
XDCAM HD (35Mb and 25Mb)**
XDCAM HD 422**
IMX***
DV25

Select OP1a:
DVCProHD
DV50
DV25
IMX***

Finally you can enjoy the freedom of native editing in Final Cut Pro without the need for Logging and Encoding.

Calibrated Software's unique solution successfully joins the separate video and audio P2 MXF files so that Final Cut Pro sees them all as ONE file - complete with TimeCode and Reel Name. PullDown can also be removed or added for select frame rates.

And you're not just limited to Final Cut Pro, you can also natively import files in Compressor, Motion, Sound Track Pro, QuickTime Player and Shake.

MXF and FCP
 

Sorry, double posting

So if one edits now in FCP, the consensus is that in addition to the QT files, one should should save either the BPAV folder OR MXF versions. Correct? If so, is there any advantage now to archiving as MXF (again, for someone like me using FCP)? Seems like an extra step that could be done later if necessary.

What quicktime files are you guys talking about? On the Mac version of the clipbrowser software, is the primary choice to export quicktime? PC users don't have that choice.

Why would a mac user export mxf files?

My understanding was that mxf is another wrapper just like quicktime, but more common on PC's. I thought both file types contained the same XDCAM-EX codec so both would require the codec to be installed on the host machine anyway making them pretty much equal in terms of future proofing.

Maybe I'm wrong here but I think mxf is to PC users what quicktime is to mac users. In either case I don't think it would be practical to convert entire BPAVS to either QT or mxf, you only need to convert the footage that's being used as complete individual clips or sub-clips.

I store all my BPAVS on HD and only import the parts I need into FCP. This converts the footage I use into quicktime wrapped XDCAM-EX files. Both get backed up by Time machine to an external drive and archived at regular intervals.

QT or MXF?
 

Rob, the question to use QT or MXF can be answered easily - if you need the Metadata that MXF contains and gets from the raw material, then opt for MXF right away.
In that case archive the MXF files, BPAV is not needed any more. To edit MXF files in FCP natively, use the above mentioned Calibrated MXF plugin for FCP.

If you only need video and sound but not the Metadata, then you can use QT files and archive the BPAV folder(s).

However, as it is a more cumbersum to split a large BPAV folder i.e. from a 16GB SDHC card onto 4 single layer DVDs, I'd rather opt to convert the BPAV content into MXF files from the very first moment on (instead of converting the BPAV folder into QT files and to archive those as well as the BPAV folder(s)). MXF is easier to archive, easier to handle, easier to retrieve later.

FWIW, MXF is a wrapper like QT, like AVI, like...
So the content in an MXF file "picture-quality-wise" is identical to the source material or that of the corresponding QT file.

Hope this helps. P.

I didn't (maybe still don't) understand MXF. My goal is to archive the most future-proof format of the orginal video (like having a camera tape). I'm still under the impression that QT is not that format. Even now, if I were to need to turn a project over to an Avid editor, wouldn't my QT files require some transcoding? That is, wouldn't the original camera data be better? I don't know--am asking.

My interest in this thread was whether MXF files were somehow better than the BPAV folder. Aside from being able to more easily split them onto DVD's and the metadata, it seems they're not. Correct?

MXF was built to do exactly what you said you wanted. An attempt to future proof these files including carrying the metadata (which QT cannot). The BPAV folders, at least for now, will require a re-wrap to be useful to most editors. In that regard, the original BPAV folders would be LESS useful to editors because they would not only have to download the Clipbrowswer software, they'd have to wait for the re-wrap for any and all files you submit to them.

If you gave them the MXF files, they could drop them into the NLE with no further effort, and simply go to work. The MXF files are visually identical to the original files that were stored in the BPAV folder structure so nothing is lost.

I save the BPAV folders because you can't use clipbrowser to export XDCAM 50 mbps 4:2:2 or downconvert to SD from the mxf files.

It's the mxf files I toss. I can always recreate them.

Huh?

How are you getting XDCam 50? And why wouldn't you convert to SD in you NLE?? I really don't get where you're going here...

Clipbrowser 2.0 exports 4:2:2 50 mbps, right? (You have to pay for the license to lose the watermark.)

As for your second question, clipbrowser converts to SD faster than most NLEs.

Mine won't. Maybe yours does. In any event, even if you export that way the video will still be the same as if you exported "MXF for NLE", so I don't see any advantage.

I'll have to try it sometime.

No advantage qualitatively, but someone could be asking for delivery in that format.

Somewhat moot for me since my Clipbrowser won't do it, but I'd probably render in my NLE anyway where I'd have more control.

There are many ways to skin this cat. Exporting to Sony mxf is absolutely identical to the original mp4, no reason to save one or the other, as long as you archive one. You can also export directly to Avid AAF, but, AAF files lose their link to the source material in Avid, so you have to be really careful to backup your data with the BPAV info.

IMHO, the absolute best way is to do a Cineform conversion directly from the BPAV folder, then archive the CFHD.AVI file. Sony .mxf will not import into all NLE's. CFHD will. Within Avid, a CFHD file does not need to be transcoded to DNxHD. It can be imported to whatever DNx resolution you select...very convenient. CFHD also imports as 601/709, every time. Certain NLE's, like Vegas will play games with the superwhites that you have to be very careful about wrangling in order to keep your images undistorted.

Long form GOP(eg Sony mxf) is designed to be a camera storage medium. It is NOT an edit medium. Sure, you can edit with it, but, that comes with its own set of problems. There's lots of folks who will claim they have no problems editing mxf, and I won't argue. But, at a minimum, they are incurring multi-generational losses on a grand scale.

I just can't imagine the logistic nightmare of saving BPAV folders. Totally unnecessary and cumbersome.

Bill, so far (as we've discussed) I haven't edited in HD and rendered to HD (haven't gotten that far yet), just SD.

Do I understand you to say that editing with the Cineform files eliminates recompression when working totally in HD?

Hi Jay...
Using CFHD doesn't totally eliminate recompression losses, but, if you visit their site and look at their examples, 10 generations and the image still holds up. CFHD is a very dependable and predictable codec...no surprises, no failures. Cineform has frequent updates. They seem to be a wee bit sloppy, making software mistakes as they go thru all these evolutions, but, all in all, well worth working with Cineform as a routine part of my workflow. One of their options is to convert to SD when you transcode. For my money, it's too early (i.e. before editting) to downconvert, but, I've heard positive results from their downrez algorithms.

Jerry's initial question was: With MXF, do I have to archive BPAV?
Your second sentence is the answer: NO. Period.
And BTW, the original is MP2 (in an MP4 wrapper), not MP4. Don't confuse BPAV with AVCHD.

AVI is just another wrapper like Quicktime and MXF but with lots of probs coz Microcrap did not get it for years how to write a good wrapper. Why this additional wrapper without additional benefit and why another codec?
Plus: Avids do read MXF.
And concerning Vegas: It does not make sense to run in circles just because some software authors play games. Better quit them and choose a product that fulfills essential requirements. Let your purse vote. Professional results require professional tools. Nothing less. If you count the time to run circles, it costs you lots of coins more than what the app did cost you less.

That used to be so - in the early days of MP2. With the advent of far more powerful PCs, lots of RAM, things have changed fundamentally and MP2 Long GOP has grown to a full scale edit format. Take HDV or XDCAM on Avids, Liquids, Macs to name but a few.
On the other hand, MXF is a wrapper, no format (or codec), don't mix it. MXF is worldwide THE interchange format for video between broadcast organizations. So the question is, can the NLE read a certain MXF file AND does it understand the content in that MXF file, can it de-cipher the data in that file and is the required codec available?

I can therefore not understand why you talk about generation losses. A digital copy is as good as the original. If you change the codec and have to re-render, than you get a generation loss. Your above mentioned CFHD.AVI incurs a generation loss, not MXF with MP2 Long GOP content.

I could not agree more. But IMHO MXF is THE way to skin the cat.

With all due respect. P.

Peter...

Thanx so much for expressing your point of view, however, you have some misinformation.
First of all, if you want to playback or decode the mp4 file in the BPAV folder, you need an AVC codec, wrapper not withstanding.

Secondly, mxf is not a universal format. there are several versions which are incompatible with one another, depending on the OEM. Sony mxf, for example, is not the same as Avid mxf. As you said, mxf is just a wrapper. What's contained in the wrapper is as variable as avi or mov.

Lastly, CPU processor speed has nothing to do with generational losses of a video codec. Long form GOP is still a interframe compressed mpeg2 codec. It will suffer a loss every time you go thru the decompress/recompress process, and it doesn't matter how fast the processor does this. CFHD is an intra-frame comression scheme. If I can quote cineform: " None of these formats were designed for post-production – they were either designed for capturing video on tape (DVCPRO HD), distributing video (MPEG), or still images (JPEG). Their application in a post-production environment often introduces unwanted characteristics including: i) generally poor multi-generational quality, ii) 8-bit arithmetic processing limitations, iii) specialized hardware, and iv) lack of a compatible forward path to higher resolutions (Cinema 2K/4K) or arithmetic depth (10-bit, 12-bit). In most cases these codecs are not worthy of use in a Digital Intermediate workflow, and is the reason many have felt that "compression" within DI is a "bad word". " While the penalty you pay is larger file sizes, the silver lining is that you don't suffer compression/decompression losses. A digital copy is NOT an exact duplicate if it has been decompressed/recompressed. May I refer you to this URL: http://www.cineform.com/technology/C...termediate.htm

I still read about people having problems playing back long form GOP codecs. They see a bit of stuttering, simply due to pipeline roadblocks, I/O limitations, whatever is taking up CPU cycles. This doesn't happen with CFHD.

With all due respect.

Bill, sorry... Wrong. The video format of the EX1 is MPEG2. See picture below, EX1 OpsManual Page 124. MP2 in an MP4 wrapper. No AVC codec needed, but an MPEG2 codec.
SP Mode files are basically HDV files which after rewrapping for example to QT run natively in Final Cut Pro or any other app with the proper (Calibrated) codec.

I did not want to say that MXF were a universal format but it is a worldwide acclaimed and proven container that fulfills what professionals in the video and broadcast industry need and require. Mainly for cross plattform file exchange. Notwithstanding the fact that MXF comes in several "flavours".
As a matter of fact, when re-wrapping a BPAV file to MXF it keeps its entire MP2 content, therefore I made intentionally the distinction between being able to read a file (container) and to understand or decode its content (codec).
So the Sony MXF is as good as the corresponding BPAV file.

Where did I say that?

Again: The question was not which codec is best for post-pro but does Jerry have to archive the complete BPAV folder structure or is MXF as good as BPAV. And I repeat:
MXF is identical to BPAV, not need to bother with both.

And again: CFHD is one generation behind the BPAV original or its identical twin Sony MXF.

If you transcode from MXF for post pro, take DVCProHD, CFHD, ProRes as whichever other interframe codec you prefer. But that is a different topic.

Best P.

huh????
must be a language problem 'cuz, I don't understand

I've saved all mine for almost a year, and I wouldn't call it a nightmare. I just drag the foler to two external drives, and then import QT from one of them (I back up the QT to DVD).

Based on the disagreements in this thread, I don't plan on changing my workflow any time soon...

You guys have totally left me in the dust here.

Can anybody recommend a site that will explain MFX, BPAV, wrappers, AVI, AVC, Cineform , CFHD etc.

You don't need a site. Let's try it this way:

Most cameras record a type (format of video). In the case of the EX1 it records a file using mpeg2 compression. This kind of compression is also used for DVDs. Sony's implementation of the mpeg2 codec is newer than most. This could present a problem for some trying to place that file into their editors. So the mpeg2 file is "wrapped" into a more standardized format called MXF or "Media Exchange Format". This is supposed to make it easier to share that file from place to place and make it more recognized by more programs.

When the EX1 writes the files to the SxS card, it writes them in a folder structure. That structure is called BPAV. If you open up a BPAV folder on your computer you'll find that it contains a specific folder structure with files in them. The Sony Clipbrowser software takes this proprietary file structure, and extracts an mpeg2 file wrapped up in an MXF file container. So the filename has the structure xxxx.MXF.

There are three fairly common file "wrappers" out there and some smaller known ones. On Macs the most common is the Quicktime wrapper. It has the extension .MOV. On the PC, the most common wrapper is the .AVI. Inside of those wrappers you can have all kinds of files. When you open your NLE and tell it to open a file, it looks at the wrapper to see if this is a wrapper it knows about. If so, it allows you to put the file on the timeline. If the compression method used to create the file inside that wrapper is known to your NLE, then it displays the video and audio. If it doesn't know the compression method (codec) then the file opens on the timeline but may be missing the video or audio portion, or maybe both.

In some cases, the codec is so specific that it doesn't need a wrapper. An example of this is Windows Media Files. These are created by Microsoft encoders and always have a .WMV extension. It is it's own file type inside it's own wrapper. No other wrapper is needed. Another is .mp3 audio. It's a very specific type of audio that needs no wrapper.

Now let's talk about codecs. There are generally 3 types.

1. Codecs used to capture video. These codecs need to take a lot of raw data being thrown at them by the camera, and write them quickly out to tape or media card. They are far less concerned with how easy it is to DECODE the information they ENCODE. These are also known as acquisition codecs. The Sony EX1 uses Mpeg2, the Panasonic HVX uses DVCProHD.

2. Editing or intermediate codecs. Once we get the video back to the editor, we need something that will hold up while editing. It would be nice if they were easy on the computer CPU so that they would play in real time even as we make changes to them. Very few codecs are truly great at this. An exception to this is the Cineform codec. It holds up great while editing, and is very fast. When I convert my HD files to this codec for editing, they move like miniDV files on the timeline.

3. Mastering or finishing codecs. These codecs are used to store all the information we want saved once we are done editing. In some cases they are very specific like mpeg2 for writing to DVDs, or AVCHD/Mpeg4 for writing to BluRay. In some cases, they are more generic like when we want a lossless codec to make sure that we have an EXACT representation of our finished work in case we want to produce something from it later.

In some cases, the codecs can pull double or triple duty. In the case of the Panasonic cameras, their DVCProHD codec is often used in all three phases. It's not necessarily ideal for all 3, but the time saved in not having to transcode from one thing to another is a real help when speed is of the essence. This is one reason these cameras have taken such a stranglehold of broadcast news. You can shoot, edit, and master without ever having to change codecs and it saves a lot of time.

Cineform works great as an editing codec,and it's not bad as a mastering one either. I've mastered some of my HDV projects this way. It produces fairly small files for as high a quality as they are. CFHD is shorthand for CineformHD.

Other times, I'll want to really master to something that is totally lossless. and when I want that, I may render with the PNG Lossless codec inside a quicktime container. Or use the Blackmagic lossless codec inside the AVI container.

Does this make things clearer for you?

Thx Perrone for your clear and very explanatory words.
Wish I could write like you do. P.