What's the big scare about copying BPAV? at DVinfo.net

Go Back   DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Sony XDCAM EX Pro Handhelds
Sony PMW-300, PXW-X200, PXW-X180 (back to EX3 & EX1) recording to SxS flash memory.


Reply
 
Thread Tools Search this Thread
Old December 19th, 2010, 12:22 PM   #1
Major Player
 
Join Date: Oct 2006
Location: Burbank CA
Posts: 466
What's the big scare about copying BPAV?

Relatively new to the Ex-1r and the file workflow. Would like some feedback regarding copying, archiving and the somewhat confusing file designations.

Seems that some of us are casual about using the Mac OS finder or Windows to simply copy the BPAV folders from the memory cards to a hard drive. I've seen warnings from Sony and some editors that the initial copy process should be done with either Sony's clip browser software or an application like Shot Put pro. Shot Put has the ability to copy the cards to multiple locations simultaneously.
So far, I've separated (in my mind anyway), the archive and copy process from the editing. Once the archive is done, then there are multiple ways to import the clips into whatever NLE your using.
I see references to file types such as MP4, MXF etc.... Part of my confusion is how the clips get, or if and when they get transcoded. I thought the XDCAM format was mpeg 2 anyway.

I am very familiar with tape based workflow. I have a very uneasy feeling about this whole file workflow process as I feel lucky when the clips somehow manage to import in my NLE (final cut pro). However, I'm looking for a deeper understanding as to what's going on at the various stages of the process.
Anyone have a link to a better understanding? I've already purchased the Vortex DVD's.

thx
David Morgan is offline   Reply With Quote
Old December 19th, 2010, 01:43 PM   #2
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
Just a few things.

The Sony camera places it's XDCamEX codec (which you rightly stated is Mpeg2 based) inside a file with an MP4 extention. That file structure does not have the ability to carry metadata. So the metadata is stored separately inside the BPAV along with an index and other information.

Doing an OS file copy is usually fine. But there is no verification on that process and if any small glitch happens, it can make the video file corrupt, it could show up in the video or audio, or you may see no effect at all. Programs like ClipBrowser do a verification process on the files that are copied to make certain they are bit for bit accurate and can be relied on.

Some NLEs cannot use the BPAV structure and so clipbrowser allows you to re-wrap the video/audio files into a .MOV file for use on the Mac, or a .MXF for use on the PC. Not all NLEs need this done. And if it is done, you lose the metadata. There is no transcode happening here.

I am unfamiliar with how Final Cut does what it does, but Sony Vegas can use either the BPAV structure or the MXF files. It does little with the metadata though. Avid on the other hand can use either the BPAV or the MXF fles, and if using the BPAV has access to ALL the metadata it seems. Very handy.

There really is a lot less to go wrong here than there is with tape. There is not verification process other than with your own two eyes when coming off tape. Tape also cannot carry metadata. Well at least not at our level. I can't even begin to tell you how nice it is to import clips into the NLE and sort by good takes automatically.
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old December 19th, 2010, 02:02 PM   #3
Major Player
 
Join Date: Oct 2006
Location: Burbank CA
Posts: 466
thx, explanation is pretty clear. How about renaming? Can you rename clips at any point? Or, should you rename after injest into the NLE? If you do that, it seems that going back to the original archive files with the renaming would make finding clips a small nightmare?
David Morgan is offline   Reply With Quote
Old December 19th, 2010, 02:56 PM   #4
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
DO NOT RENAME THE CLIPS...

If you want to create a mess, this is about the fastest way to do it. Again, Metadata is your friend. I don't know what FCP gives you, but you should be able to do anything you need with the metadata and not touch the actual clip names.

This is actually one thing I love about Avid. You CAN rename the clips inside Avid to anything you like and Avid just doesn't care at all.
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old December 19th, 2010, 03:34 PM   #5
Major Player
 
Join Date: Jan 2010
Location: Maassluis, The Netherlands
Posts: 294
Quote:
Originally Posted by Perrone Ford View Post

This is actually one thing I love about Avid. You CAN rename the clips inside Avid to anything you like and Avid just doesn't care at all.

Just like Premiere... It's a great way to organize inside your project.
__________________
Brainstormnavigator searching for the hole in the sky.....
Audiovisual Designer (NL) - http://www.brokxmedia.nl
Walter Brokx is offline   Reply With Quote
Old December 19th, 2010, 03:34 PM   #6
Sponsor: Westside AV
 
Join Date: Dec 2008
Location: Mount Washington Valley, NH, USA
Posts: 1,365
If you want you can rename the clips in FCP as well, it does not change the actual filenames. I tend not to do this but use comment fields for notes and folders/bins for organizing in FCP.

FCP is actually a nice database and quick search program as well as an editor, like I am sure Avid is as well.

I usually drop all the clips from a project into one FCP project and keep it in the same folder as the archived clips. It is much better IMHO than XDcamTrans for making notes and viewing clips, it will play back through my AJA Kona and Matrox MX02, so you can see what you really have.

Personally I do not keep my BPAV's once I have rewrapped to .movs. Because most of my projects are shot with a lot of different cameras and formats, so keeping everything as .movs is easiest for me. I use the nanoFlash a lot and it does not make BPAV's, it uses a higher bitrate EXcodec.

So personally I don't see much use for the BPAV file structure.

But before you have established your system of archiving and working, I would definitely keep the BPAV's. It also makes it easier to send clips to someone not using a Mac. Although that is easy by just trans coding the .movs in any number of different programs like Squeeze or Compressor.

I have personally never had a problem just drag copying BPAV's, over 800 hrs worth I think. But it probably is safer to use Clipbrowser or Shotput with verification. I always copy to two HD's and check a couple of the files by playing them back, before wiping the SxS cards.
__________________
Olof Ekbergh olof@WestsideAV.com
Westside A V Studios http://www.WestsideAVstore.com/
Olof Ekbergh is offline   Reply With Quote
Old December 19th, 2010, 04:37 PM   #7
Regular Crew
 
Join Date: Mar 2008
Location: Bridgenorth
Posts: 116
For what it's worth...

We've been doing straight Windows OS file copies of BPAV folders from SxS cards to network archive storage since the PMW-EX1 came out and we have never had a corrupt file problem or anything similar; it's proven to be very reliable.
Steve Gibbons is offline   Reply With Quote
Old December 19th, 2010, 05:29 PM   #8
Trustee
 
Join Date: Feb 2006
Location: USA
Posts: 1,684
I've been told by a friend who is a a major computer geek that if you use the finder to drag & drop and check that the size of the file is exactly the same by checking with get info then you are safe. He says if the bits are the same it can't be corrupted.

I have had Clip Browser not see some files that were fine in the past but supposedly that's fixed tho it loeft me wary. I personally found the shot Put interface not simple enough and easy fro someone I gave the job to to screw up. Nice thing about Clip browser is that it will put many cards in one BPAV folder this connecting spanned cards. I always use it for spanned cards.

Someone else told me that Carbon Copy Cloner was also reliable because t double checked copies.
Leonard Levy is offline   Reply With Quote
Old December 20th, 2010, 05:06 AM   #9
Major Player
 
Join Date: Jul 2003
Location: Leeds, UK
Posts: 660
Images: 10
I use Toast to 'compare' the BPAV master to the two copies I make after I've done a drag and drop copy using the OS. It takes twice as long but ensures your copies are perfect. This is CRC checking.

Using any other system like Shotput or Clip Transfer will take just as long, the only way to make it faster is to use faster hard drives and a faster card reader.
__________________
FCPX/LS300/EX1/FS100/GoPro/Vinten/HotHead/Jib/Track/Dedos/Lightstorm/Coollights
Duncan Craig is offline   Reply With Quote
Old December 20th, 2010, 05:06 AM   #10
Inner Circle
 
Join Date: Oct 2001
Location: Honolulu, HI
Posts: 2,053
Computers have error checking as a routine part of the file copy process. That goes for the Mac or PC.

If it weren't for that, you'd have all sorts of problems arising from making backups, installing programs, etc.

So you can use the clip browser for duplication and transfer, or you can just drag-and-drop, and save yourself a lot of additional effort.
__________________
Dean Sensui
Exec Producer, Hawaii Goes Fishing
Dean Sensui is offline   Reply With Quote
Old December 20th, 2010, 05:52 AM   #11
Regular Crew
 
Join Date: May 2009
Location: Cambridge, UK
Posts: 39
Dean's right. There is no issue using drag and drop to copy files otherwise data would always be getting corrupted - it never does - as long as the files copy across completely the'll be fine.

Of course you must NEVER mess around with the file structure or rename files in the BPAV folder.
__________________
Andy Taplin
www.andytaplin.co.uk
Andy Taplin is offline   Reply With Quote
Old December 20th, 2010, 05:57 AM   #12
Vortex Media
 
Join Date: Jan 2009
Location: New England, USA
Posts: 2,480
I also move BPAV folders around all the time with the Finder and have never had a problem. Clip Browser and XDCAM Transfer are important parts of may workflow, but that doesn't mean BPAV folders can't simply copied from one location to another without causing problems.
__________________
Vortex Media http://www.vortexmedia.com/
Sony FS7, F55, and XDCAM training videos, field guides, and other production tools
Doug Jensen is offline   Reply With Quote
Old December 21st, 2010, 04:59 AM   #13
Major Player
 
Join Date: Jun 2008
Location: Monroe, NY
Posts: 686
Ditto,

John
John Peterson is offline   Reply With Quote
Old December 22nd, 2010, 12:20 PM   #14
Major Player
 
Join Date: Nov 2006
Location: Brisbane, California
Posts: 530
I have FCP and an EX1.

I use Calibrated Software's Quicktime Plugin "EX XDCAM MP4 Import" with native EX1 Files. This excellent software makes Quicktime think that the EX1 files are Quicktime and allows you to play them without transcoding anywhere Quicktime is used, including Final Cut Pro. I am really grateful that Calibrated created this plugin. It's saved me terabytes of uselessly wasted hard drive space and weeks of wrapping / transcoding time.

Finder Copies vs another tool:

Although the Finder or any OS based copy is quite reliable in normal copying, if one bit is off or corrupted in transit or if the source flash card destination hard drive is on the edge, you can get errors that are not reported in a Finder copy. It's also possible that a 'get info' would not report a difference in size. This is very rare but in fact I have experienced it a few times. Using a more fail safe method to copy, then compare the copy with the original, is more reliable. For EX1 I use the freely-supplied Sony EX Clip Browser software to perform the copies. It's easy, reliable, creates a destination directory with a unique name based on the date, and if you set the preferences correctly you can get what is called a CRC copy which ensures that the destination is the same as the original. You can then browse and playback the resultant copied EX video with it and rename the top level directory if you wish. Alternative methods, such as Shotput, or Chronosync with verification (which I use to transfer non EX flash media) is also possible, but not necessary because the Clip Browser is fast and reliable and has other browsing features.

Backing up entire directory vs other methods:

I would backup the entire flash drive to the hard drive, preserving the structure and meta data. Then you can always have the media in it's original form if you want to use it and other apps need that meta data to interpret the EX1 files properly, such as the FCP plugin or Calibrated's plugin. You can then import the files to FCP and perform the 'selects' there. I like to use 'subclips' to organize my FCP clips rather than renaming the clips. The file info is stored in the 'clip' and not altered but the subclip mechanism works well for me.

Alternatively, you can use Premiere CS5, which works natively with EX files, is faster than FCP. I am using that increasingly over FCP because it can natively work with pretty much every file type you throw at it.

Hope this helps.
Keith Moreau is offline   Reply With Quote
Old December 30th, 2010, 08:01 AM   #15
Regular Crew
 
Join Date: Jan 2008
Location: Central Coast, NSW Australia
Posts: 119
I use Finder to copy my BPAV's all the time. Clip Browser is too slow for my purposes. However, I have had several instances where only the video files have copied and not the two metadata folders. Luckily I always check the files after copying and if I have time also open up the folder in XDCAM Transfer and check that they copied ok, so disaster was averted. I would recommend you always check the copied files before reformatting your cards.
Shaun R Walker is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 08:18 PM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network