View Full Version : CF card 15 frame drop outs between 4 gig segments


Keith Forman
May 11th, 2008, 07:45 PM
I've noticed that when using the CF cards that there is about a 15 frame loss between the 4 gig segments (60i HDV). This occurs with the Transend card and the Sandisk CF card that I have. This does not appear to be a speed of the card issue becuase there are no other drop outs.

FAT32 limits the card segments to 4 gigs which makes it understandable why there is this size limit.

This half second drop out is very bothersome because when I shoot with two camera, I lose sync every 22 minutes and have to resync--not to mention the other issues related to this problem.

Anyone else have this problem or have any information on this?

kdbf

Steve Mullen
May 14th, 2008, 01:52 AM
I've noticed that when using the CF cards that there is about a 15 frame loss between the 4 gig segments (60i HDV).

If others are finding this -- it certainly argues for using 4GB cards so at least you avoid this problem. (And, can be burned to DVD.) On the other hand, with a larger card, one might get lucky several. :)

What I don't get is given the overwheming, IMHO, advantages of tape, why are folks even bothering with SD cards?

I might be more wowed by cards if there wasn't a extra box to attach to the camcorder. Or, if the camcorder had two built-in slots. Seems like the whole SD cards deal was an after-thought.

Modern NLE suck in a whole tape in an hour. Is anyone in that big of a hurry to start editing? I mean hard news used to be shot on film! Decades of breaking news was shot on tape!

Until I can shoot an hour on a card that costs $15 -- I can't see why I'd bother to get burn an achieve DVD -- all before I erased a card.

What am I missing?

Zach Love
May 14th, 2008, 03:23 AM
CF & tape record. tape back up on the shelf & you don't put more hours on your tape heads when you transfer (aka capture) CF footage to your NLE.

personally I think this is the way to go if you need to save footage. shooting an hour of footage & then waiting an hour to edit really slows me down.

I've worked in news & while I love working on NLEs, many things that I edited tape to tape would have taken at least 50%-100% longer if I had to capture all my footage to a hard drive

as too the 15 frame drop, that really stinks & should ("should" as in they 'should,' not that they 100% 'will') be fixed by Sony, but for now that is what the tape is for

if you really want to be shooing only on cards, well easy... sell your Z7U & buy an EX1 or HVX-200

Giroud Francois
May 14th, 2008, 04:16 AM
usually i take the time of capture to look at the picture, so when capture is over, i got a pretty nice idea of what i got.
blindly transfer files on a computer and spending hours to derush them is pretty the same time.

Chris Medico
May 14th, 2008, 04:32 AM
Have you read what is happening with the Canon HG cameras?

I don't have a Z7 but I wonder if the problem similar to what is happening with the Canon HG cameras when they span multiple 2gig files. They have found if you do a binary copy and join of the files from the command line in Windows that the glitches disappear.

It might be worth a try to see if this is a work-around for the Z7.

Read the first post here

http://www.dvinfo.net/conf/showthread.php?t=113083

In post #16 they start talking about joining the files through a binary copy in windows.

I hope it helps.

Laurence Kingston
May 14th, 2008, 06:13 AM
Are you using the Sony transfer utility to rejoin the clips? This utility transfers the clips, rejoining any that go over the maximum fat32 four gig limit while it transfers:

http://www.sonybiz.net/biz/view/ShowContent.action?site=biz_fr_EU&contentId=1198162909729

Keith Forman
May 14th, 2008, 06:21 AM
Have you read what is happening with the Canon HG cameras?

I don't have a Z7 but I wonder if the problem similar to what is happening with the Canon HG cameras when they span multiple 2gig files. They have found if you do a binary copy and join of the files from the command line in Windows that the glitches disappear.

It might be worth a try to see if this is a work-around for the Z7.

Read the first post here

http://www.dvinfo.net/conf/showthread.php?t=113083

In post #16 they start talking about joining the files through a binary copy in windows.

I hope it helps.

Sony has suggested that I use the "M2T connection tool" but I can not get it to work. It does not see any m2t files at all (no idea why). However Sony no longer supports this utility and in fact does not make it available to the public anymore.

Has anyone else had this problem with the drop outs between 4 gig segments?

KDBF

Keith Forman
May 14th, 2008, 07:12 AM
Are you using the Sony transfer utility to rejoin the clips? This utility transfers the clips, rejoining any that go over the maximum fat32 four gig limit while it transfers:

http://www.sonybiz.net/biz/view/ShowContent.action?site=biz_fr_EU&contentId=1198162909729

That software is for the HVR-DR60 not for the HVR-MRC1 unit and does not work with the later.

kdbf

Laurence Kingston
May 14th, 2008, 09:01 AM
That software is for the HVR-DR60 not for the HVR-MRC1 unit and does not work with the later.

kdbf

It's for both. I use it myself so I'm really sure of this. It stitches any split clips back together while it transfers and should fix your problem.

Keith Forman
May 14th, 2008, 10:46 AM
It's for both. I use it myself so I'm really sure of this. It stitches any split clips back together while it transfers and should fix your problem.

You might be correct but I have not had any ability to see anything other than the HVR-DR60 unit with the Recording Unit Utility Software software.

kdbf

Chris Medico
May 14th, 2008, 01:55 PM
Sony has suggested that I use the "M2T connection tool" but I can not get it to work. It does not see any m2t files at all (no idea why). However Sony no longer supports this utility and in fact does not make it available to the public anymore.

Has anyone else had this problem with the drop outs between 4 gig segments?

KDBF


Yea, I know. Please try the binary copy and join trick. Let us know if it works. I suspect it will solve your problem.

Chris

Laurence Kingston
May 14th, 2008, 02:04 PM
You might be correct but I have not had any ability to see anything other than the HVR-DR60 unit with the Recording Unit Utility Software software.

kdbf

The utility should be able to see the files off of any fat32 device. For instance, it will work off a CF card reader with the Z7 memory card. When you scan the drive you will get a list of clips and you can select HDV, DV or both. Any video section that is long enough to be split across a couple of clips will be shown as a single video with two or more parts. I can't see why it wouldn't work for you.

Keith Forman
May 14th, 2008, 03:33 PM
The utility should be able to see the files off of any fat32 device. For instance, it will work off a CF card reader with the Z7 memory card. When you scan the drive you will get a list of clips and you can select HDV, DV or both. Any video section that is long enough to be split across a couple of clips will be shown as a single video with two or more parts. I can't see why it wouldn't work for you.

I still can't get either Sony utility to see any files.

PROBLEM WITH THE DROP OUTS UPDATE: Sony thinks the problem has to do with the GOFs. I tried it in Adobe Premier and the 15 frames seem to be there. Vegas for some reason has the drop outs (which is strange since it should be the most compatible NLE with the Z7).

Laurence Kingston
May 14th, 2008, 04:13 PM
The Sony transfer utility should see any FAT32 device you connect to your system. It shouldn't matter what it is beyond being FAT32 and having the clips in the appropriately named folders (as of the Sony naming conventions). It is really strange that you can't use this.

Dave Blackhurst
May 14th, 2008, 11:25 PM
FWIW, I've had enormous trouble with finding anything that reads 8G cards from a CX7 other than using the camera - only one of my computers has a reader that will work correctly, and I can attest you absolutely need the Sony utility to restitch the long clips back together correctly, although I suspect the binary method will work but haven't tried it.

Sounds rather similar as the files are unreadable unless stitched (the Canon files had a short gap like you describe), even though they are 100% fine when in playback on the camcorder.

You may want to look into uninstalling the utility and reinstalling, and see if there are any updates as well - both were needed for me to get my CX7 working (yeah, I know, different cam, but the tapeless workflow is similar)

And for the poster wondering about tapeless convenience - shooting multicam...

30 minute event = hour and a half + to dump tape, 30 minutes tops with MS/card dump - you're editing before the first tape would be ingested. I think you may be trading off in render time, but I like to let my machine render while I sleep or do somethig else, and if I start editing an hour earlier...

Steve Mullen
May 15th, 2008, 01:25 AM
30 minute event = hour and a half + to dump tape, 30 minutes tops with MS/card dump -- you're editing before the first tape would be ingested.
In the entire history of film and video no one could do this -- and yet all the great work of a Century of motion pictures was created. Do you really believe that saving a few hours is going to actually make for better productions?

Video -- which is fast compared to film -- can hardly be said to have greatly advanced the art of moving pictures. In fact, I can't think of anything done with video that is "better" than what was done using film in the last century.

Unless one is shooting news -- speed is really not critical. I can't imagine wanting to start editing after shooting. I want good food and drinks, a night's rest, and time to let my mind wonder back over what I shot.

After using a harddisk camcorder for a year -- I still agree my initial feeling that a BUILT-IN harddisk is the way to go if SPEED is critical. And BD is even better.

SD cards, IMHO, are just as premature as P2. Obviously, some day EVERYTHING will be solid-state. But, just like laptops with solid-state storage -- that day is not now. Witness the problems you are having.

When solid-state wins it will not be FAT32! FAT32 is totally wrong for video!

Laurence Kingston
May 15th, 2008, 06:29 AM
I hear what you're saying Steve but I disagree.

Let's say I come back from a gig with two tapes of video and I need to transfer the footage before I go to bed. With tape capture I have an extra two hours lobbed onto the end of my day. With a 32GB Compact Flash, I just start the process and go take a shower and let it transfer the two hours plus footage by itself.

Some people may be having problems with this process but for me it is working quite well. Longer clips stitch back together with no problems and the fact that they are split up on the original card makes them easier to back up on DVD-R where I would have problems with the larger files not fitting.

There are still times when I like to use tape (hence the Z7 instead of an EX1) but when I go tapeless, the whole process seems to work quite well.

Keith Forman
May 15th, 2008, 07:43 AM
[QUOTE=
You may want to look into uninstalling the utility and reinstalling, and see if there are any updates as well
[/QUOTE]

It is not an installable utility. It runs without the need to install it.

kdbf

Dan Keaton
May 15th, 2008, 09:07 AM
When solid-state wins it will not be FAT32! FAT32 is totally wrong for video!

Dear Steve,

When one wants to develop a device which records to removable media, such as Compact Flash or SD, one has to choose a protocol for formatting / storing the data on the media. This protocol provides for the directory structure and the structure for storing the data.

It would be great to just develop a new protocol, something better than Fat32 for video and audio, but then this would be proprietary and it would cause problems for users wanting to read the data on their computers. Of course, with the proper software, this new proprietary format could work flawlessly.

FAT16 (File Allocation Table - 16 bits) and FAT32 are available for use and work both on PC's and Macintoshes. FAT32 is licensed by Microsoft for use by others.

NTFS is commonly used on computers, but Microsoft does not license others to use this formatting/storage protocol.

So, we are really stuck with using a proprietary format, or using FAT32.

FAT32 works, if the proper precautions are taken:

1. The device writing the data has to be very smart when closing one file and opening another, so that no data or frames are lost in the process.

2. The user can not just read a portion of the files independently, such as placing one segment of the file on the timeline, then placing the next segment of the file next to it.

If this is done, then it will appear that audio or video or both are lost. This is especially true if the data uses a Long-GOP format.

3. What is required is that the various segments have to be stiched back together so that the original data is now seamless. This can be accomplished as the data is read from a media card to the computer.

So, a device developer could easily develop a new format for storing video/audio files on removable media. But would it be accepted by the users?

Laurence Kingston
May 15th, 2008, 09:48 AM
So, a device developer could easily develop a new format for storing video/audio files on removable media. But would it be accepted by the users?

That's exactly what the Sony transfer utility does. Will it be accepted by users? Well, I've accepted it, and so have quite a few others...

Dave Blackhurst
May 15th, 2008, 10:50 AM
That's exactly what the Sony transfer utility does. Will it be accepted by users? Well, I've accepted it, and so have quite a few others...

Exactly! Once I got PMB installed and working correctly (and I blame my machine which is long overdue for a fresh Win-ders install...), it's been incredibly easy to work with the files/video. Sure, it's a little challenge to get going, but that's life on the bleeding edge.

I know that speed isn't everything, but I'll take every advantage to get to the edit faster (event/wedding), the longer it takes to get started editing, the more things get in the way... at least that's how it seems to work for me! Whether ingesting from a flash card or a HDD (and doesn't the HDD have the same formatting issues??), dumping the video in around 1/3 the time is worth it in my book, and maybe gives you that time to relax, have some drinks, etc! I like to see what I've got "in the can" ASAP, but then again I'm not making the "next great movie" except in the eyes of the guy/gal wanting to see it ASAP...

Keith Forman
May 15th, 2008, 11:31 AM
Exactly! Once I got PMB installed and working correctly (and I blame my machine which is long overdue for a fresh Win-ders install...), it's been incredibly easy to work with the files/video. Sure, it's a little challenge to get going, but that's life on the bleeding edge.

I know that speed isn't everything, but I'll take every advantage to get to the edit faster (event/wedding), the longer it takes to get started editing, the more things get in the way... at least that's how it seems to work for me! Whether ingesting from a flash card or a HDD (and doesn't the HDD have the same formatting issues??), dumping the video in around 1/3 the time is worth it in my book, and maybe gives you that time to relax, have some drinks, etc! I like to see what I've got "in the can" ASAP, but then again I'm not making the "next great movie" except in the eyes of the guy/gal wanting to see it ASAP...


How did you get the transfer program (utility) to work? I can't seem to get either of the Sony utilities to work properly.

John Knight
May 15th, 2008, 04:28 PM
Video -- which is fast compared to film -- can hardly be said to have greatly advanced the art of moving pictures. In fact, I can't think of anything done with video that is "better" than what was done using film in the last century.

Unless one is shooting news -- speed is really not critical. I can't imagine wanting to start editing after shooting.

FAT32 is totally wrong for video!

What's the weather like on your planet Steve?

Keith - have a read of this
http://www.sony.com.au/objects/Other/HVRDR60/html/soft.html

Specifically the M2T File connection section.

SOME NLE's will automatically stitch for you - others don't but probably will in future patches.

Apple Final Cut Pro 6.02 and Edius 4.6 - DOES support stitching
Adobe Premiere CS3 - Needs stitching tool
Sony Vegas Pro 8.0 - Needs stitching tool for Z7 unit (but support stitching from DR60 Hard disk

Hope this helps.

Chris Medico
May 15th, 2008, 04:37 PM
Hello?? (knock)(knock)(knock).. Is this thing on???

Please try the suggestion for doing a binary join of the files.

It does work with footage from my DR60 when it spans multiple files. Works MUCH better than stitching it in your editing software.

Steve Mullen
May 15th, 2008, 08:10 PM
What's the weather like on your planet Steve?

No need to get nasty.

As I said, I've got a HDD camcorder and, compared to tape, I really don't like it. I've got my tapes going back to Beta. I really like having everything. I can't imagine speed ever being more important TO ME.

Until 12-months ago -- weddings were done with no problems using tape. Why, suddenly, is the world better because you can capture the night after shooting? What's wrong with starting the edit at noon on Monday? Capture the tapes while you have coffee and read your email and the newspaper. Do you get a huge bonus for getting done a few hours faster? If you don't, what's the big hurry?

Anyway -- it's just my opinion base on my year with non-tape camcorders. I'm not coming to take you baby away. Relax.

PS: Trust me -- FAT32 is not going to be used in a world of terrabytes. (In fact, I don't think it can go over 132GB.) And, it won't be NTFS either. There will be an Open standard.

Brian Standing
May 15th, 2008, 08:31 PM
I can't imagine speed ever being more important TO ME.

Well, speaking as someone who tries to juggle a full-time non-video related job, a 7-year-old son, a wife, and still try to find time to edit, anything that shaves any time off the logging/capture process is a good thing in my book!

PS: Trust me -- FAT32 is not going to be used in a world of terrabytes. (In fact, I don't think it can go over 132GB.) And, it won't be NTFS either. There will be an Open standard.

The Linux Ext2 or Ext3 system would be a good candidate.

Chris Medico
May 15th, 2008, 08:35 PM
Hi Steve,

I have say that going tapeless has been a godsend for me. What I do would fit more into the news type category even though its not news. I have to produce a product and hand it to the customer on a DVD within an hour of the end of the event. This wasn't realistically possible before I started using DVRs. It took a while to get my workflow efficient enough to make the desired timeframe but now I can take a 20 minute video do a motion graphic overlay and render it out to a DVD within an hour.

When I shoot weddings I use a mix of tape and DVRs. I've already paid for the DVRs so might as well use them right? They do make ingest much nicer but my HC3 tends to be hidden in a bush somewhere with a tape in it. Overall its a nice to have but doesn't affect timeline or how I put the product together.

This post up to here is a bit OT but the following is On Topic.

When I shoot video that spans multiple files onto the DR60 I join the files together using the method described in the Canon thread and I don't get any audio stutter or dropped frames at the splice point. Since my DR60 is a Sony product and using FAT32 I have high confidence file handling is done the same as it is on your CF recorder. I also suspect the same method to join the files together will take care of the problem. Best of all its a ZERO cost solution.