With MXF export files, is it really necessary to keep BPAV files? - Page 4 at DVinfo.net

Go Back   DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Sony XDCAM EX Pro Handhelds
Sony PMW-300, PXW-X200, PXW-X180 (back to EX3 & EX1) recording to SxS flash memory.


Reply
 
Thread Tools Search this Thread
Old December 11th, 2008, 09:52 AM   #46
Regular Crew
 
Join Date: Apr 2007
Location: Johns Creek, GA
Posts: 42
Thanks all for the spirited discussion! I have purchased some inexpensive 1TB external drives and backed up my MFX files.
Jerry Wiese is offline   Reply With Quote
Old December 11th, 2008, 04:34 PM   #47
Inner Circle
 
Join Date: Jun 2003
Location: San Jose, CA
Posts: 2,222
Quote:
Originally Posted by Peter Kraft View Post
Plus: Avids do read MXF.
AVID Media Composer 2.7.7 and earlier wouldn't handle EX1 MXf. 3.0 does.
Gints Klimanis is offline   Reply With Quote
Old December 11th, 2008, 06:35 PM   #48
Trustee
 
Join Date: Feb 2006
Location: USA
Posts: 1,684
Perrone,

Thanks, that was incredibly clear and helpful.

Where does Apple Pro Res fit into this schema. Is it similar to Cineform?

Can you add a touch more that brings "Long GOP" into the picture as well?

- Lenny
Leonard Levy is offline   Reply With Quote
Old December 11th, 2008, 11:54 PM   #49
Trustee
 
Join Date: Jun 2004
Location: Denver, Colorado
Posts: 1,866
Quote:
Originally Posted by Rob Collins View Post
I've saved all mine for almost a year, and I wouldn't call it a nightmare. I just drag the folder to two external drives, and then import QT from one of them (I back up the QT to DVD).

Based on the disagreements in this thread, I don't plan on changing my workflow any time soon...
I agree. With Clipbrowser, just drag the thumbnail to a folder. The BPAV structure goes with.
Tom Roper is offline   Reply With Quote
Old December 12th, 2008, 01:33 AM   #50
New Boot
 
Join Date: Mar 2008
Location: Johannesburg, South Africa
Posts: 6
Thanks Perrone, incredibly clear and enlightening!

I work with broadcast news and archive the BPAV to RAID (one BPAV folder per story). My reason basically beeing I get these wonderful thumbnails, and all the metadata easy to read. Saves me a lot of time when I quickly need archive footage.
Fredrik Sperling is offline   Reply With Quote
Old December 12th, 2008, 02:32 AM   #51
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
And in THIS case, I COMPLETELY agree that saving the BPAV folder makes total sense. Most people won't need the thumbnails in archiving their master files, but broadcast is a totally different animal.

Quote:
Originally Posted by Fredrik Sperling View Post
I work with broadcast news and archive the BPAV to RAID (one BPAV folder per story). My reason basically beeing I get these wonderful thumbnails, and all the metadata easy to read. Saves me a lot of time when I quickly need archive footage.
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old December 12th, 2008, 02:54 AM   #52
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
Quote:
Originally Posted by Leonard Levy View Post
Perrone,

Thanks, that was incredibly clear and helpful.

Where does Apple Pro Res fit into this schema. Is it similar to Cineform?

Can you add a touch more that brings "Long GOP" into the picture as well?

- Lenny

Sure Leonard.

Apple's ProRes fits into phase 2 and phase 3. It's a robust codec that can stand up to editing (it was built for it) and is good enough to master in. There is also ProResHQ which is another step up, but is a bit "heavy" for most machines to use for editing. If you are cutting footage off RED or other "film" based footage it might make sense. ProRes is a bit lossy, but perfectly acceptable for a few generations of rendering.

The idea of long GOP is simple in theory.

Let's say we are shooting 30 progressive frames per second on our camera. It's on a tripod, and it's on a sidewalk aimed at a building. Every half second we record 15 frames of video. An intraframe camera like the HVX or like true film, will record 15 frames that are complete. It's like taking 15 rapid pictures with a digital still camera and that gets recorded to the media. In pictures that don't change much, this is an incredibly wasteful way to store data.

In the same scenario, Long GOP takes a full frame on frame 1. Then on frame 2 it takes another full frame and compares it to the first. It then only records the difference between the two and writes that to the media. It does the same for the next 13 frames. Well at least the Sony does because it uses a 15 frame GOP structure. Other cameras like the JVC use a 7 frame GOP structure which would handle rapid motion better.

Where Long GOP falls down is if there is a large amount of change between the frames in it's structure. At some point it can't keep up with how quickly things are changing. The software is usually pretty smart. It's making predictions of what will change from one frame to the next to speed up things. When that begins to fail, the picture quality begins to show artifacts. You can see exactly the same thing in your DVDs even if you start with intraframe sources because Mpeg2 which is used for DVDs (and the EX1) use the same technology.

Newer Long GOP codecs like the improved EX1 mpeg2 codec do a better job at this stuff than the older (DVD and HDV) mpeg2 based recorders. Better still are the new mpeg4/AVCHD recorders like the new Panasonic HMC150. The problem is that their efficiency at writing the data better comes at the cost of your computer working harder to DECODE the same stuff later. So you either need a faster computer for the new stuff, or you need to transcode to something easier to handle like Cineform or ProRes.

The internet community usually rails against Long GOP because it is harder on computers in editing, and the codec can be overwhelmed with fast action. What is nice is that the EX1/EX3 allows professional users and soon amateur users to bypass the long GOP codec. The SDI port on the side allows you to take the information off the sensors uncompressed. Just like in the $100K+ cameras. And you can record it any way you want. The downside is that you have to pay extra for that recorder. But in cameras without an SDI port, this option simply does not exist. Why this is exciting to pro users of the EX1/EX3 or amateurs with deeper pockets is that the EX1/EX3 give you the image sensors of a far more expensive camera, and with SDI based recording, you can now write that data stream. And you can do it for tens of thousands of dollars less than you could to do it 2 years ago.

Make sense?
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old December 12th, 2008, 04:02 AM   #53
Major Player
 
Join Date: Nov 2007
Location: Europe
Posts: 245
Perrone,
Chapeau :-)
Peter Kraft is offline   Reply With Quote
Old December 12th, 2008, 07:21 AM   #54
New Boot
 
Join Date: Mar 2008
Location: Johannesburg, South Africa
Posts: 6
Thank's again Perrone, learning a lot here.

I also take my finished story cut in FCP and export it as XDcam MP4, import it to ClipBrowser and put it on my RAID. This way I can easily put it on the SxS-card in the camera, and play it out via the SDI.

If time is short I also use this way (SDI out) to feed a story via satellite instead of compressing and ftp.
Fredrik Sperling is offline   Reply With Quote
Old December 12th, 2008, 07:51 AM   #55
 
Join Date: Nov 2001
Location: santa fe, nm
Posts: 3,264
Images: 10
As you point out, Perrone, Sony's long GOP codec is an interesting animal. At 35mbps(ala EX1), images are crystal clear, until you get full frame motion, like in a pan. Then the image breaks down, due to the compression artifacting at 35mbps.

While I still don't have my hands on a Convergent Design SDI to CF recorder(have a nanoflash on order)all the reports I've seen show a much reduced motion artifacting. The C-D CF recorders also use the Sony codec, storing data at a selectable 50 or 100mbps. This is a good compromise, since there's not a lot of portable equipment that can capture an HD-SDI signal.
Bill Ravens is offline   Reply With Quote
Old December 13th, 2008, 12:46 AM   #56
Trustee
 
Join Date: Feb 2006
Location: USA
Posts: 1,684
Thanks again Perrone,

Your replies are now saved on my desktop. I sort of know most of this but it helps to see it all at once.
Now last questions -

1 - what's the diff between using Cineform or ProRes.

2 - I'm on a Macbook Pro - should I bypass ProRes HQ and just stick with regular ProRes?

3 - I know a lot of people who edit HD using Panasonic DVCPro100 codec. Often these are guys who are used to shooting P2 or with varicam etc. I've been trying to convince them to move over to an Ex-1 but they are freaked out by long GOP and the fact that its a different codec.

Can they convert Ex-1 XDCAM footage to DVCPro100?

I guess its kind of a pain for them because they need to convert all of it up front otherwise they are constantly rendering.

I've been told that the easiest way to cut EX-1 footage is to use and XDCam timeline but set for renders to be in ProRes. Can you do the same for DCPro100 and would that solve the problem? Just render once when dropping the stuff in and then it is always ProRes?

Sorry if this a bit afield of the rest of the thread here.

Lenny
Leonard Levy is offline   Reply With Quote
Old December 13th, 2008, 02:45 AM   #57
Inner Circle
 
Join Date: Dec 2005
Location: Bracknell, Berkshire, UK
Posts: 4,957
Quote:
Originally Posted by Bill Ravens View Post
As you point out, Perrone, Sony's long GOP codec is an interesting animal. At 35mbps(ala EX1), images are crystal clear, until you get full frame motion, like in a pan. Then the image breaks down, due to the compression artifacting at 35mbps.
Must be something wrong with your camera as my pictures don't break down on pans.

Perrone's statement that AVCHD is better than MPEG2 is not one I agree with. Certainly long GOP AVCHD can produce smaller files. AVCHD is a member of the Mpeg4 family and was really designed for very low bandwidth use such as web streaming and in that application Mpeg4 is very good, but I have yet to see any long GOP AVCHD or Mpeg 4 that has artifacts down at the level of XDCAM HD.

Perrone's description of the GOP sequence is simplified .Mpeg 2 not only forward predicts the motion in the picture but also backwards predicts, so your only ever 7 frames from your full reference frame, the Bi-directional predicted frames are call B frames, forward Predicted frames are P frames and the reference full frames are I frames. The frame sequence looks like this:
IBBPBBPBBPBBPBBIBBPBBPBBPBBPBBP
__________________
Alister Chapman, Film-Maker/Stormchaser http://www.xdcam-user.com/alisters-blog/ My XDCAM site and blog. http://www.hurricane-rig.com
Alister Chapman is offline   Reply With Quote
Old December 13th, 2008, 03:40 AM   #58
Major Player
 
Join Date: Nov 2007
Location: Europe
Posts: 245
Quote:
Originally Posted by Alister Chapman View Post
Must be something wrong with your camera as my pictures don't break down on pans.
It could be the camera... or the operator. In my early days of film camera operating, we were told not to do pans of more then 30 degrees per second with the camera with a 90 degree angle to the picture pane. A 180 degree pan therefore took 6 seconds. To not to pan too fast we counted loudly... and it worked. Faster pans always resulted in visibly stuttering pictures on the screen.
That may be old craftmanship many an operator now smiles mildly upon, nevertheless video follows the same psychovisual laws as old perforated film (especially with progressive frame rates).

If I really need a fast pan, I either do it verrry slowly and speed that take up during post pro or I open the shutter as far as it gets with that specific camera, sometimes even adding a light horizontal motion blur in post pro before speeding the pan up to the desired value.
Peter Kraft is offline   Reply With Quote
Old December 13th, 2008, 03:51 AM   #59
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
Quote:
Originally Posted by Leonard Levy View Post
Thanks again Perrone,

Your replies are now saved on my desktop. I sort of know most of this but it helps to see it all at once.
Now last questions -

1 - what's the diff between using Cineform or ProRes.

2 - I'm on a Macbook Pro - should I bypass ProRes HQ and just stick with regular ProRes?

3 - I know a lot of people who edit HD using Panasonic DVCPro100 codec. Often these are guys who are used to shooting P2 or with varicam etc. I've been trying to convince them to move over to an Ex-1 but they are freaked out by long GOP and the fact that its a different codec.

Can they convert Ex-1 XDCAM footage to DVCPro100?

I guess its kind of a pain for them because they need to convert all of it up front otherwise they are constantly rendering.

I've been told that the easiest way to cut EX-1 footage is to use and XDCam timeline but set for renders to be in ProRes. Can you do the same for DCPro100 and would that solve the problem? Just render once when dropping the stuff in and then it is always ProRes?

Sorry if this a bit afield of the rest of the thread here.

Lenny
Leonard, I can't fake it. I know very little about editing on FCP. However, something you have pointed out here is one of the major reasons I never will. On to your questions:

1. Both Cineform and ProRes are terrific codecs. ProRes is apple specific and you cannot share data for editing with anyone on a PC using any program. Period. I am not certain if Cineform is PC only, but I've only seen it used on PCs.

2. Unless you are originating with very high quality footage (2k or higher), I'd just stick with ProRes.

3. DVCProHD is 1280x1080. Taking footage from a full raster camera like the EX1 to DVCProHD throws away a bunch of information. Your 1920x1080 becomes 1280x1080. The Panasonic guys edit in this codec because their cameras SHOOT in this codec. They don't record anything better in the first place, so no need to shift to a 1920x1080 codec in post.

The DVCPro HD codec is superior to the EX1 codec in a couple of ways. First it is intraframe and not long GOP. This has clear advantages during phase 1. Second, it captures marginally more color information per frame of footage. The big loss of the DVCPro HD codec is that it's horizontal resolution isn't nearly as good as XDCAM EX. And the XDCAM EX is terrible for phase 2 or 3. It's a phase 1 codec only (or primarily).

My beef with FCP is that you need to have all of your sequences match. I am SO used to Vegas where I can drop ANYTHING on the timeline, cut it, and render out. No need to convert anything at all. I have dropped EX1 footage, miniDV, and RED stuff (2k .mov) onto a timeline, cut it like it was native, and rendered out to Windows Media for streaming. It's all seamless. FCP really needs to get on the ball with this. For a pro level NLE, this is a really stifling limitation.
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old December 13th, 2008, 03:57 AM   #60
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
Quote:
Originally Posted by Alister Chapman View Post
Perrone's statement that AVCHD is better than MPEG2 is not one I agree with.
Fair enough. We can agree to disagree.

Quote:
Originally Posted by Alister Chapman View Post
Certainly long GOP AVCHD can produce smaller files. AVCHD is a member of the Mpeg4 family and was really designed for very low bandwidth use such as web streaming and in that application Mpeg4 is very good, but I have yet to see any long GOP AVCHD or Mpeg 4 that has artifacts down at the level of XDCAM HD.
Watching mp4 encodes of 2k or 3k footage, with 40+ Mbps rates, I am inclined to disagree. I cannot speak to the footage quality of the HMC-150, but I have seen high bitrate AVCHD on BluRay, and it looks DARN good. Even with very high motion content.

Quote:
Originally Posted by Alister Chapman View Post
Perrone's description of the GOP sequence is simplified .Mpeg 2 not only forward predicts the motion in the picture but also backwards predicts, so your only ever 7 frames from your full reference frame, the Bi-directional predicted frames are call B frames, forward Predicted frames are P frames and the reference full frames are I frames. The frame sequence looks like this:
IBBPBBPBBPBBPBBIBBPBBPBBPBBPBBP
Yes, I simplified. Thanks for filling in the blanks for a more full description.
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 05:36 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network