STORAGE / WORKFLOW Help - Page 3 at DVinfo.net

Go Back   DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Sony XDCAM EX Pro Handhelds
Sony PMW-300, PXW-X200, PXW-X180 (back to EX3 & EX1) recording to SxS flash memory.


Reply
 
Thread Tools Search this Thread
Old August 26th, 2009, 08:06 AM   #31
Major Player
 
Join Date: May 2008
Location: Moab, UT
Posts: 264
Similar to Olof's. I use two sxs 16's, two sxs 8's, and the phu-60, then two g-tech 500gb raid mini's to put identical copies of the BPAV folders on, dump, and then erase cards/phu.

It would be better to not have to dump during a shoot, as it's an accident waiting to happen, especially after a 12 hour day shooting, but I'm not yet trusting enuf of the mxr solution.

After shooting, dump the two field drives onto two identical 2 TB FW raid 0's (0+1) for editing, plus put original BPAV's on a raid 1 archive drive, plus backup to Blu-ray. Erase the field drives.

When the two edit drives get filled, I'll switch to a sata raid 5 unit.
__________________
www.packcreekproductions.com
EX3, Mac Pro, FCP
Mike Chandler is offline   Reply With Quote
Old August 26th, 2009, 10:08 AM   #32
Vortex Media
 
Join Date: Jan 2009
Location: New England, USA
Posts: 2,483
Quote:
Originally Posted by Steve Gibbons View Post
What are people typically doing "in the field" with their EX footage?
Keep an eye out for this coming out this fall. Most people should be able to get by with having only two SxS cards in the field.

PXU-MS240 Mobile Storage Unit From Sony

You'll be able to off-load cards 10x faster than you can shoot them.
__________________
Vortex Media http://www.vortexmedia.com/
Sony FS7, F55, and XDCAM training videos, field guides, and other production tools
Doug Jensen is online now   Reply With Quote
Old November 5th, 2009, 05:38 AM   #33
New Boot
 
Join Date: Jul 2004
Location: Sydney, NSW Australia
Posts: 24
Hard Drives for Archiving?

I've been doing quite a lot of research recently on archiving.

For the cost and convenience hard drives are definitely the go, but there is a big issue of drives seizing up after sitting on a shelf for a couple of years. They seem to need to be spun regularly to keep the lubricants from solidifying.

Maybe the Drobo would help with this, but it will fill up eventually and you might have the same problem. Incidentally, I haven't figured out how the data is spread across the drives, and what the full drives actually contain.

I prefer solid state media for archiving, and I tried going down the Gold Archival DVD road (through ProDisc), but not only is it bloody expensive, it's also painfully slow and labour intensive (relatively speaking). Not a good trait for maintaining an archive system - pity because it seemed like the most stable option. Unfortunately, Blu Ray doesn't seem to much of an archival medium, as the pits are really small and therefore more error prone for storage.

Maybe Solid State Drives could be an option for "the future".

By the way, good idea about giving the clients a hard drive, that makes a lot of sense.
Tyler Smith is offline   Reply With Quote
Old November 5th, 2009, 02:51 PM   #34
Regular Crew
 
Join Date: Nov 2008
Location: San Francisco, CA
Posts: 121
I've recently done quite a bit of research on the question of flash for "archiving" and the clear message I found was that flash memory is the worst possible choice for long-term or long-ish-term storage. Well, maybe it's better than rice paper.

Most of the conversations about flash focus on the longevity of the devices - i.e. 10000 erase-write cycles for MLC and 100000 cycles for SLC - but people don't often go into the retention of data on flash devices. The basic fundamental principle of how flash works is that each cell contains a floating gate that either holds a tiny charge or it doesn't. As that charge leaks away over time, the cell's contents become questionable.

The short of it is that retention is interactive with usage, so the more a device is written to, the shorter the retention period will be. This can sometimes end up being months and not years. And some consumer devices use really bad flash controller chips and allow "silent errors" to creep in undetected with the reasoning being that they're just consumer devices and just being used for silly photos and music and nobody will notice or care if a bit gets flipped now and again. To extend the retention of flash devices, you need to periodically (every few months or so) refresh/rewrite the data. This, of course, shortens the retention period of the device even further. Not a good way to go.

I know they're nice and small and easy to store and cheap and many here have a strong attachment to the idea of using consumer grade flash devices for long-term storage, but knowing what I now know, I won't trust any consumer MLC device to retain data beyond several weeks.
__________________
EX3, Z1U, FX1, D1X / NeXTstation Turbo Color / FCS3
Brooks Graham is offline   Reply With Quote
Old November 5th, 2009, 03:01 PM   #35
Major Player
 
Join Date: Jul 2009
Location: Hampshire, UK
Posts: 693
Hard drive and blu-ray/DVD for the projects inc footage and separate blu-rays just for the footage as a separate backup. Works for me. The raw footage also goes onto another HD which is stored on a separate site.

Make sure you spin the drives up from time to time though.

The mistake some people make is to assume archiving is "forever". Storage technology is moving so fast that in 5 years time they'll be a better, larger more reliable and cheaper solution. So just work on the basis that you are only storing the data on the medium term because you'll be moving it anyway in the future.

Media deterioration is an issue pretty much whatever digital format you use. I have premium Gold CD-R's from 10 years ago that won't play, but the cheap unbranded turquoise ones are just fine. Go figure! I would take all claims about media life with a pinch of salt. After all, if you look at most of the guarantees you'll find all you are entitled to is replacement media. Not much comfort if you've lost your only copy of a project.
__________________
Marcus Durham
Media2u, Corporate Video Production For Your Business - http://www.media2u.co.uk
Marcus Durham is offline   Reply With Quote
Old November 8th, 2009, 09:11 AM   #36
Major Player
 
Join Date: Mar 2006
Location: Mount Rainier, MD
Posts: 428
Quote:
Originally Posted by Jason Leigh View Post
To keep all the raw files of all jobs is insane in regards to space.
I don't agree at all with this assessment. This is exactly what I do. Drives are so cheap now, that there is no reason not to keep all your full projects. I don't use RAIDs - don't trust them. They do nothing to protect against file system failures. Instead I use a mirrored drive. Basically I use a trayless SATA drive tower and have sets of drives, for example "Media 05" and "Media 05 Backup". I use Sync Backup to synchronize the drives nightly. When the drives are full I stick both on the shelf and get 2 new drives "Media 06" and "Media 06 Backup" I use Western Digital Green drives because they are fast enough and run a lot cooler (to my mind this makes them less likely to fail, but I don't have empirical evidence of that).

Additionally, when I'm done with a project a render out a ProRes self-contained Quicktime master, a version without music and titles, and individual audio tracks stored on an Archive drive with it's own mirrored copy.

There are a million ways to archive and backup. I've come up with a system that works great for me.
Brett Sherman is offline   Reply With Quote
Old November 8th, 2009, 02:09 PM   #37
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
"I don't use RAIDs - don't trust them."

You do realize that mirrored drives = RAID 1?
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old November 9th, 2009, 08:50 AM   #38
Major Player
 
Join Date: Mar 2006
Location: Mount Rainier, MD
Posts: 428
I use the term "mirrored" loosely. It's not a real-time mirror, but rather a nightly synchronization. This eliminates the risk of a file system failure affecting both drives.
Brett Sherman is offline   Reply With Quote
Old November 9th, 2009, 08:58 AM   #39
Inner Circle
 
Join Date: Nov 2006
Location: Tallahassee, FL
Posts: 4,100
Ah ok. Gotcha. The terminology threw me.
__________________
DVX100, PMW-EX1, Canon 550D, FigRig, Dell Octocore, Avid MC4/5, MB Looks, RedCineX, Matrox MX02 mini, GTech RAID, Edirol R-4, Senn. G2 Evo, Countryman, Moles and Lowels.
Perrone Ford is offline   Reply With Quote
Old February 18th, 2010, 05:30 PM   #40
Regular Crew
 
Join Date: Sep 2009
Location: Ogden
Posts: 161
As far as continuing the workflow into FCP, would you next:?

1) Open FCP
2) Import media (recently converted .mov's)
3) Set timeline settings to ProRes422
4) Drag clips to timeline and edit away

??



Quote:
Originally Posted by Mitchell Lewis View Post
Okay I thought it might be fun to share our workflow. Maybe it will help someone. We produce television commercials and corporate videos so 99% of our footage/projects must never be lost as someone has paid us to produce them.

Here's what I'm calling THE LAZY BUT CAUTIOUS WORKFLOW. It's been working really well for us since January of this year. We haven't lost a single piece of footage. (knocks on wood)

INGEST
1) Connect the SxS card to my MacBookPro
2) Connect a Western Digital 500GB buss powered FW800 external hard drive to MacBookPro
3) Launch XD CAM Clip Browser
4) Using XD CAM Clip Browser create a new folder on the WD 500GB external hard drive
5) Using XD CAM Clip Browser drag all clips from the SxS window into the WD 500GB hard drive window
6) If there is a second SxS card, insert it, select all the clips and drag them into the same folder. This is important because any clips that might have spanned the two cards will be automatically joined together into one clip.
NOTE: This is my BPAV archive drive. Once it fills up, I put it on the shelf and purchase a new one.

EDIT
1) Move the WD 500GB hard drive to our MacPro tower
2) Launch XDCAM Transfer
3) Click the ADD button and navigate to the new BPAV folder on the WD 500GB hard drive
4) In XD CAM Transfer name all the clips (sometimes just selecting all of them and giving them all the same name)
5) In XD CAM Transfer sort through the clips and label the good clips with OK
6) Change the Status View so that only clips labeled OK are shown
7) Go to XDCAM Transfer>Preferences>Import Tab and choose a location to save your MOV files to. We save them to our fiber channel RAID.
8) Select all the clips and click the IMPORT button

BACKUP
1) We have a FW800 drive that we use to back up our RAID. We use Apple Time Machine because it's easy, and we don't forget do back up on a regular basis.
2) Our RAID is striped RAID-5 so if one of the drives suddenly goes bad, the RAID switches to RAID-0 mode and you can continue editing until you replace the bad drive.

ARCHIVE
1) When the RAID gets full, we use EMC Retrospect Backup to archive our project files (MOV videos, graphics, audio, everything except the BPAV folders) off to Western Digital 500GB portable FW800 hard drives. Retrospect automatically flows the data from one drive to another as the fill up. But the best part of using this software for archiving is when it comes time to restore. You simply launch the program, find the data or project you want to restore, select RESTORE and the program tells you want drive(s) to install. Once you install the drive it automatically starts the restore process.
2) As discussed in the INGEST description, we save all our BPAV folders on to separate portable hard drives. Worst case if we were to loose all our project files, we would at least still have the original footage. Also, we'll sometimes re-import footage from the BPAV files for use in other projects.

Let me know if anyone finds this useful. (or is pissed off that I took the time to post it) :)
Annen James is offline   Reply With Quote
Old February 18th, 2010, 07:02 PM   #41
Trustee
 
Join Date: Dec 2008
Location: Kennewick, WA
Posts: 1,124
Quote:
Originally Posted by Annen James View Post
As far as continuing the workflow into FCP, would you next:?

1) Open FCP
2) Import media (recently converted .mov's)
3) Set timeline settings to ProRes422
4) Drag clips to timeline and edit away

??
Close.... Here's what I would do:

1) Launch FCP
2) Close any open projects (not even an Untitled project should be open)
3) Go to Final Cut Pro>Easy Setup
4) Choose the Easy Setup that matches the format you recorded the majority of your footage. I normally shoot in XDCAM EX 1080p 30 VBR (VBR = HQ and CBR = SQ)
5) Import media (your recently converted .mov's)
6) IMPORTANT: Go to Final Cut Pro>User Preferences - Render Control tab and change the Render Codec to: Apple Pro Res 422 [HDV,XDCAM HD/EX/HD422 only]

So in summary, what you've done here is set up a project where all new sequences will be created with the XDCAM EX codec, but anytime you RENDER, the render file will be ProRes codec (which yields better results than rendering in XDCAM EX). Buy doing this, you can drop your clips into the timeline and they will play without rendering (assuming your hardware is fast enough....which it normally is).

Hope that helps! :)
__________________
Sony EX3, Canon 5D MkII, Chrosziel Matte Box, Sachtler tripod, Steadicam Flyer, Mac Pro, Apple/Adobe software - 20 years as a local videographer/editor
Mitchell Lewis is offline   Reply
Reply

DV Info Net refers all where-to-buy and where-to-rent questions exclusively to these trusted full line dealers and rental houses...

Professional Video
(800) 833-4801
Portland, OR

B&H Photo Video
(866) 521-7381
New York, NY

Z.G.C.
(973) 335-4460
Mountain Lakes, NJ

Abel Cine Tech
(888) 700-4416
N.Y. NY & L.A. CA

Precision Camera
(800) 677-1023
Austin, TX

DV Info Net also encourages you to support local businesses and buy from an authorized dealer in your neighborhood.
  You are here: DV Info Net > Sony XAVC / XDCAM / NXCAM / AVCHD / HDV / DV Camera Systems > Sony XDCAM EX Pro Handhelds

Thread Tools Search this Thread
Search this Thread:

Advanced Search

 



Google
 

All times are GMT -6. The time now is 11:14 AM.


DV Info Net -- Real Names, Real People, Real Info!
1998-2017 The Digital Video Information Network