Canon Vixia HG10 2GB File Limit Headache
 

I am starting this thread for a couple of reasons. First, it's nice to have a central repository where everyone can dump issues and experiences related to the topic. Second, I sincerely hope that everyone who owns an HG10 (and this problem will probably be present in the HF10 & HF100 too since Canon has included the same software) puts serious pressure on Canon to fix this problem immediately. As far as I'm concerned, as it stands today, this camera is useless for recording clips longer than ~17 minutes in HXP mode. I'm guessing that most people who have purchased this camera will not have the array of outboard audio equipment at their disposal that I have assembled over the past 15 years, so this should be a serious issue for every owner of this camera.

I purchased the HG10 because (1) it is basically the AVCHD equivalent to the venerable HV20, and (2) I am fully onboard with the AVCHD workflow. Honestly, I knew about this problem before purchasing this cam but figured it was isolated to the included editing software, which would not have affected me.

Here's the issue in a nutshell. The file system on the HDD (I am guessing) used in the HG10 imposes a 2GB file size limit. Apparently, this 2GB limit is not part of the AVCHD spec so it is NOT a problem with the format. This brickwall limit is easy to prove. Set the recording mode to HXP, place the cam on a table, press record, and walk away. After 20 minutes, press the record/stop button, plug in the power adapter and USB cable from your PC. The cam's HDD will be mapped to a logical drive (G:\ on my PC) and you can simply browse the contents of the cam's HDD, which is essentially the same structure as a blu ray disc. The video clips (stored as .mts files) are all located in the "Streams" subdirectory. They are incrementally numbered (e.g., 00000001.mts, 00000002.mts, etc.).

If you started with zero footage on the cam and shot 20 minutes of HXP footage, you will find exactly two .mts files in the Streams subdirectory. If you watch the 20-minute clip prior to connecting the cam to your PC, you will see and hear 20 minutes of seamless video/audio. If you drag and drop the two .mts files and open them in your NLE, you will notice 418 samples of audio (originally sampled in cam @ 48K) missing just prior to the splice point and exactly two dropped frames just after the splice point. Doesn't matter if you shoot 24PFor 60i.

If you transfer the 20-minute footage using the included Guide Menu app, you will have two <datetimestamp>.M2TS files. If you open those files in your NLE, you will see the exact same result.

I also converted the .mts files into a Cineform intermediate .avi file using NEO HDV to see what would happen. Interestingly, after converting two .mts files shot in 24PF (resulting in two Cineform .avi files) and opening those two files in Vegas Pro 8.0b (build 217), the two dropped frames were gone from the second file but the audio was exactly as it was in the first raw .mts and Guide Menu converted .m2ts files.

original email to Canon Tech Support
 

Here's my original complaint sent via the web...

"After thorough testing, it appears that the software Canon has provided to transfer video from the HG10 to a PC (running WinXP) irreparably damages both the video and the audio streams when transferring videos shot in HXP mode exceeding ~15-17 minutes in length. The exact damage is in the form of audio and video dropouts at the splice point(s).

The only caveat regarding this issue listed in the documentation accompanying the HG10 is the following, "Longer scenes will be divided into files smaller than 2 GB. They will be saved as separate scenes when transferred to a computer. After these separate scenes are transferred back to the camcorder, you may notice audio drop-outs during playback at the points where the original scene was divided."

<EDIT: the above quote is referring to the included Corel DVD MovieFactory SE editing software>

The above statement is simply incorrect. The audio drop-outs do NOT occur as a result of transferring the video from the PC to the camcorder. Just the opposite. They are caused by transferring the video from the CAMCORDER TO THE PC.

I have no intention to transfer edited videos from the PC back to the camcorder. So the above quote from Canon's documentation should not apply to me. However, the problem is clearly apparent as I discovered while evaluating both the audio and video streams in my NLE. What is extremely frustrating is the fact that playing back the spliced video files while they remain on the camera yields absolutely flawless audio and video at the splice points.

Therefore, I am reporting this as a non-documented bug and request that it be corrected. If it cannot be corrected, then I will be forced to return the camera to Canon as defective and expect a full refund of my purchase price."


I really don't intend to return the camera at this point. So I merely wanted to convey the extent of how serious a problem I consider this to be. And I wanted to catch the attention of competent help right away. Unfortunately, that did not happen. To Canon's credit, though, I did get an email response in less than one hour <-- seriously impressed me considering it was ~11pm.

First response:

Subject: Re: Response from Canon - Technical Support
(KMM8279231V63919L0KM)
Dear Mr. Courtney:

Thank you for your inquiry. We value you as a Canon customer and appreciate the opportunity to assist you. I sincerely apologize for any inconvenience you may be experiencing due to the video and audio dropouts that you are receiving while trying to print with the HG10 camcorder.

Canon camcorders generally record at a framerate of 29.97 fps. Verify that the video editing program you are using is set to this, and not 30 fps. This is a difficult issue to address, as several different video capturing programs have their own specific settings that could effect recorded video. If the video can be viewed properly on a TV using the STV-250N Stereo/Video cable, without the drop outs, then it would be necessary to contact the manufacturers of the video editing program that is being used. If the drop outs do appear on the TV then the camcorder heads may need to be cleaned.

We hope this information is helpful to you. Please let us know if we can be of any further assistance with the HG10 camcorder.

Thank you for choosing Canon.

Sincerely,
Stephen
Technical Support Representative


*extremely frustrating*

follow up email to Canon
 

Second email...

'If the drop outs do appear on the TV then the camcorder heads may need to be cleaned.'

This is a HDD based capture camcorder, not a DV20. THERE ARE NO HEADS TO BE CLEANED.

'If the video can be viewed properly on a TV using the STV-250N Stereo/Video cable, without the drop outs, then it would be necessary to contact the manufacturers of the video editing program that is being used.'

Wrong. I've already stated exactly how and why the dropouts occur. Playback of recorded footage in the camcorder connected to ANY display yields perfect footage. It's the process of transferring the footage outside of the camera that destroys data at the splice point(s). When I open spliced files from a Sony CX-7 after transferring using the accompanying PMB software, data at any splice point is presented perfectly and contiguously. <thanks here to Dave Blackhurst for his verification> And yes, my video editing software is set to the correct frame rate for the footage to be edited.

I'm seriously impressed with the response time to my inquiry. However, I clearly know more about this camcorder than you. I suggest you either escalate this issue to a higher support level so you don't continue to insult my intelligence, or shoot 20 minutes worth of HXP footage on an HG10, walk over to a PC, transfer the files using the accompanying Media Guide software, open the two files in whatever editing software you want, look at the video frames on the timeline and the resulting audio wave, and then come back and tell me what you've discovered..."

Some people may think some of my reply was a bit over the top, but I've spent the last 15 years of my life in IT consulting dealing with uninformed initial responses such as these. After a while, it tends to get to ya, LOL.

Another reply from Canon
 

Second response:

Subject: Re: RE: Response from Canon - Technical Support (KMM8283462V77146L0KM)

Dear Mr. Courtney:

Thank you for writing to us. I apologize for the confusion regarding your inquiry. We do our best to answer all customer inquiries as accurately and efficiently as possible. I am sorry that incorrect information was sent in response to your message. We do regret any inconvenience or difficulty that resulted from our miscommunication.

While Canon does provide a copy of Corel DVD MovieFactory SE, we do not offer support for this application. For support concerning the Corel Application Disc, please contact Corel directly, at:

1-510-979-7118 or 1-800-772-6735, Monday - Friday, 09:30 a.m. - 05:30 p.m. (PST)

http://www.ulead.com.tw/Ulead/Survey...?pg=1&fs=us#cs

Please note, Canon does not produce video editing software, therefore, we can offer no support on specific editing programs. Because there are so many editing programs available, it is not possible for us to offer support or to catalog those programs. We recommend that you contact the software manufacturers, or perhaps visit their websites, to find the information you seek. We apologize for any inconvenience this may cause.

Have you tried using the Canon Backup utility to drop the files from the camcorder directly to your hard drive, then manually rename the ".M2T"
files that the Backup Utility copied to ".M2TS" for opening in your NLE software?

We would suggest trying that workflow to see if cutting out the Corel Software usage altogether will work for you. Nothing in the Canon Backup Utility, says clips have a 2 GB limit like the Corel software, so maybe the timeline and audio won't be affected.

We hope this information is helpful to you. Please let us know if we can be of any further assistance with your HG10.

Thank you for choosing Canon.

Sincerely,

Chris
Technical Support Representative


*finally, some degree of competence* But still, they're just not getting it. My reply today:


"Chris, finally, great response. Thanks for the suggestion, but it still doesn't fix the problem. First, I need to clarify that I am not using Corel/Ulead DVD MovieFactory SE. It's garbage. I am using Sony Vegas Pro 8.0b (build 217). I have no problems whatsoever opening either the raw (dragged & dropped) .mts files or the converted (using Guide Menu) .m2ts files.

Again, here's the problem. Apparently you guys are not using these AVC cameras otherwise you should know this. The HG10 (or perhaps the file system used in the camera) imposes a 2GB file limit on video clips. This is very simple to prove. And really, I want someone on your support team to do this - shoot 20 minutes of continuous HXP footage, push stop, and then connect the camera to a PC using your USB cable. When you browse the removable media drive (HG10) in Windows, you will see that the single 20 minute clip is split into two .mts files ON THE CAMERA.

If you play the 20-minute clip ON THE CAMERA, it plays back perfectly. So clearly, there is something inside the camera that enables correct splicing of the two files to provide a seamless transition between the two files. As soon as you remove those .mts files from the camera, either by simply dragging and dropping or via the included Guide Menu software, ~400 samples (sample rate = 48K) of audio are dropped from the first file and exactly two video frames are dropped from the second file at the splice point. Curiously, the audio in the second file is perfect, while the video stream in the first file is also perfect. Makes no sense to me.

What appears to be happening is that some form of metadata present on the HG10 which enables proper splicing of the two files is not being transferred to the PC when the raw .mts files are transferred. This KILLS the chance of using this camera for any type of shooting requiring >17 minutes of HXP capture (and seriously, no one is going to use the low bitrate settings anyway - let's be real here). I can handle a dropped video frame or two, but dropped audio is an entirely different story. Imagine shooting your kid's Christmas play and your audio drops out at a key moment in his/her dialogue. There are going to be a lot of pissed off parents as soon as they transfer that video from the HG10 to the PC for high def disc burning.

So, shoot 20 minutes worth of continuous HXP footage on an HG10, transfer the files to a PC using whatever method you'd like, then open those files in whatever NLE you prefer and then come back and tell me that the audio isn't butchered at the splice point. As far as I'm concerned, the transfer ustility Canon provides with the HG10 irrepairably damages the video footage upon transfer. Seriously, what good is a HDD video camera when you can't reliably extract the footage from it?"


This is where it stands as of right now. I'll post Canon's reply. Anyone who cares to, please jump into this discussion (so I'm not talking to myself, LOL!). To be fair to Canon, I haven't actually tried the backup utility route yet. I will do this when I get home tonight and really hope it does work. If it does work, then Canon needs to clearly note this workflow workaround. If it doesn't, perhaps there will be a good chance that someone will fix the issue with this provided software since it is a Canon-specific app.

Hi Aaron -

A couple observations... the Sony PMB software creates a single long file from the component .mts files, and it is seamless, just like in the camera.

There must be something in the AVCHD spec that is dictating the arbitrary 2G file sizes and stitching... and clearly there are some implementation issues.

In another thread I started I observed that the first .mts file from a Sony CX7 is absolutely unreadable unless processed through the PMB software - a fact confirmed by Sony... it appears the first file with the Canon IS readable... but then the "stitching" is flawed.

Don't know how good you are at taking the file code apart (it's just a bit over my head), but there may be clues in there.

I've accepted that I have to use the PMB software with the CX7, it's not an inconvenience, but what you've discovered scratches Canon off my list until they figure it out.

I think that CANON has it right if playback on the camera is correct - IOW the camera correctly "stitches" clips together. BUT, the way the files are tagged prevents just sticking them together with sucess, and apparently the Corel app is not putting the pieces back together properly either. Have you tried using hte Corel app JUST for importing, and seeing if it will create a single file from the bits? It looks like you did, but figured I'd ask.

FWIW, tech support is often pretty clueless at the first level - they deal with people using the DVD tray as a cupholder... but even at the higher levels, sometimes they are stumped by an engineering decision - NO one at Sony can figure out why the first .mts of a long clip can't be played without being processed through the PMB software, but it can't, no matter what you throw at it - camera playback is fine, and once run through PMB, everything is hunky dory!

Sounds to me like Canon has a similar issue, but their "outsourced" software is at fault, from your description.

Hope you figure out the "fix", I know I was gald once I solved the questions with the CX!

Thanks Dave for providing me some anecdotal ammunition to throw at Canon! There was just no way I was not going to use it in this case, LOL!

There are three primary Corel/Ulead app's included with the cam - (1)Guide Menu - the "approved" cam ->PC & PC -> cam transfer utility, (2) the InterVideo Win AVC player, and (3) Corel Movie Factory DVD SE.

As I said, I both dragged & dropped the .mts files and used the approved Guide Menu app to get the clips onto the PC. The result was the same, as indicated. I also tried the cheesy DVD authoring software which actually provided a dedicated button to link clips together into a single file. Playback of the resulting file yielded the same result as editing the .mts and Guide Menu software converted .m2ts files in Vegas Pro.

I then converted the Corel Movie Factory "stitched" clip into one of the default AVCHD formats provided in the editor. Since that was going to take 3+ hours to render, I decided to go to bed. I'll look at the resulting file tonight but don't expect anything different than what I've already experienced.

Right now, it's the Canon backup utility or bust.

What we need is for someone with a flash mem AVCHD cam to state whether or not their cam splits long continuous footage into files <2GB. That should clear up this question...

CX7 is a flash memory camera.

and BTW, I tried recording in different bitrates - they still are split at the 2g point (s), just takes longer at the lower resolutions.

Duh
 

Thanks Dave for that clarification. If it truly is part of the AVCHD "camcorder" spec - even if unwritten - then this could prove to be quite a problem for mass adoptation of AVCHD unless every manufacturer really dials this in and gets it right out of the gate.

Canon's backup utility simply copies the entire contents of the HG10's HDD into a specified directory on the PC. The .mts footage files are identical whether they are "backed up" via the software or dragged and dropped individually by the user.

I don't think this is a Vegas issue because I converted via Cineform NEO HDV a 2GB clip into an .avi, opened that in Vegas, and the audio was chopped identically to the .mts (and .m2ts) AVCHD file. To make sure, my next step is to download and install a trial version of a different NLE and see what happens...

Another Canon response
 

By the language, it seems that I'm getting closer to talking to the right people here. At least now they're admitting to the 2GB limit being imposed by the HG10...

Dear Aaron J. Courtney:

Thank you for writing, I am sorry to hear of any difficulties with the video footage from your HG10 camcorder and our email responses.

Regrettably, the HG10 does not have any settings to change the 1.9GB size that your videos and MTS files download.

I have forwarded your comments to Canon USA through our Customer Feedback process. This process allows us to capture important feedback from our valued customers. As we constantly strive to improve our products and services, your comments are vital to our continued success.

I will need to research this question in more detail, and will write back within 48 hours.

I am sorry for any inconvenience this may have caused you. Please let me know if we can be of any further assistance with your Canon HG10.

Thank you for choosing Canon.

Sincerely,

Deva
Technical Support Representative

today's follow-up from Canon

Dear Aaron J. Courtney:

Thank you for your inquiry. I am sorry for the delay in my response
regarding your HG10 camcorder.

I completely understand your frustration regarding the dropped audio and video once downloaded!

I have confirmed with one of our Senior Level Technician that it appears that the camcorder is working correctly.

This would be an issue with the software, we would recommend contacting Sony for support regarding the assembling of the 2GB clips.

I would also recommend trying this out with the Corel software that was shipped with your camcorder.

If this doesn't answer all your questions, please contact me again with the more details.

I am sincerely sorry for the inconvenience this may have caused you.
Please let me know if we can be of any further assistance with your Canon HG10.

Thank you for choosing Canon.

Sincerely,

Deva
Technical Support Representative

Today's reply from me
 

Ok, you guys are still not getting it. Yes, the camcorder is functioning perfectly. As it stands right now, however, NO video footage exceeding 2GB worth of file size captured by the camcorder can be removed from the camcorder without ruining the audio stream in the process.

I would like you to ask that same SR. level tech how he/she would playback 20 minutes of continuous video shot @ HXP that has been transferred to a PC.

The provided Corel software cannot reassemble the .mts files from the HG10's HDD into one seamless video and audio stream if that combined file size >2GB. I'm not talking about stringing multiple clips together here. I'm talking about reassembling a 20-minute clip after it has been split into multiple .mts files <-- and how stupid is this anyway! Seriously, there is no reason on Earth to limit video files to this stupid 2GB limit anyway. That was a file size limit imposed under FAT16, not NTFS. If I shoot for 8 hours continuous @ HXP on this camcorder, I should get ONE 40GB CLIP, NOT TWENTY 2GB FILES that have to reassembled into the original clip - which cannot be done seamlessly outside of the camera with the software Canon has provided with this camera (or any software in the world for that matter).

Neither the included editing software nor the player can playback HG10 footage without dropping the audio (and two video frames) at the splice point.

My question is INCREDIBLY simple:
I would like to know how to seamlessly playback spliced .mts files originating from one, lengthy (>17 minutes @ HXP) captured video on ANYTHING OTHER THAN THE HG10.

Additional thoughts
 

So far, I have come up with two PERMANENT solutions to this problem...

(1) Use outboard audio and shoot 24PF.
(2) Capture footage via HDMI out on HG10.

The outboard audio is pretty much self explanatory. Using pro gear will always yield better results. But it's added expense and inconvenient too. After converting 24PF footage (pulldown removal) into Cineform's intermediate .avi, I found no dropped frames. Now that could have been purely by chance (cadence).

The HDMI capture would also work because the cam correctly reassembles the .mts files on the fly and will push that out over the HDMI port. BUT, then you're back to a realtime (ala HDV) workflow process again. YUCK! Not why I bought into the whole AVCHD deal in the first place...

HMMMM -
I think you need to eliminate one possibility - and it's probably hard to do on the camera screen... if you playback the original long clip (and I presume it is shown as a single clip in the cam, as on the CX?), is there any dropout at the splice point - probably nearly impossible to spot without having the whole deal on the timeline, where the small dropout is hard enough to spot...

This has me wondering about how clean the splice is on the CX - going to have to set up a test with a moving object and something for audio, but I'm pretty sure the PMB software assembles gap free. I've got my tests saved, maybe I can review those and see if anything is there - the one "live" clip I shot seemed perfect, but the gap you're describing is sooooo small, I'll want to double check it!

IF you can play back the clip glitch free either in camera or played back to a big TV and audio system, then it's not the camera per se, but with the software, but I'm having trouble seeing that as a possibility - I *think* you will find the glitch is there on playback direct from in camera... if it is, maybe a firmware update chould fix it? If it isn't, then they need to jump up and down on Corel for a fix.

Just my thoughts, will have to set up a test out of curiousity for the CX, but I'm feeling better about my Sony all the time <wink>!

Hi Dave, the piped in audio via the Mic in jack makes it really easy to locate exact points in time on both the cam and in any NLE. That's why I chose to dub in a live show I recorded. You simply locate the exact splice spot in your NLE, take note of the song and the location in that song, and then dig through the clip on the cam until you hit the exact same spot. I even had both PC and cam playback streams semi-sync'd within a couple 1/10's of a second.

The in-cam playback is perfect, both audio and video. I reviewed the exact splice spot at least 10 times and I can definitively say it is flawless. As I said earlier EVERYONE will hear the audio drop when playing back audio of a "continuous" nature - music, rhythmic, etc. You can't miss it.

The clip in the preview/playback window in the cam does appear as a single clip (looking at a video frame, not the actual file structure). However, if you browse the HDD contents of the HG10, that single clip is comprised of two separate .mts files (assuming <~34 minutes @ HXP).

OK, just wanted to confirm that. That's how the CX does it as well - shows a single clip, but when you look at the file structure, it's mulitple files, 2G limit...

SO, I think you've eliminated one issue... sort of... the data is there SOMEWHERE, or you'd have a dropout on cam playback. THE BAD news is this means there's a software glitch that seems to be causing the problem when reading back the .mts files... and it seems to be on everything you've tried.

Perhaps this is why Sony has it's own software (PMB) - so as to avoid the problem on import. Obviously the camera internal firmware gets it right somehow, so that's a "start", but theres' still some issues to unravel.

Juse wondering. Have you tried copying the files to your PC with HGBackup, formatting the camera and then copying back to the camera with HGBackup? Does the camera still play seamlessly?

I will have to try with Procoder and with Elecard AVCHD Converter Studio to see what they do. I will be recording meetings in April which will run over the 2GB limit several times over, so the break over the splice point will be annoying (but I can probably live with it).

Hi,

I might try this myself sometime, but since you already have the files.

Have you tried from the command prompt:

copy /b 000001.mts+000002.mts joined.mts

(change the filenames as appropriate)

Maybe the files are a straight binary join. When you treat the second one individually it may just start at the next keyframe or something, thus losing samples.

Regards,
Andreas

Anticlimactic ending
 

I have not done this, but it should not affect in-cam playback because the backup utility effectively transfers the entire filesystem contents from the HG10 onto your PC. So moving data back and forth in this manner should not affect the viability of that data. I will verify this, if I remember, when I need to dump the contents of the HDD due to lack of capacity. But....

It appears Andreas has provided the solution to this problem. I performed a cursory review in Vegas after using the good 'ole DOS syntax and I think this problem is solved. Great job Andreas! I am running an extended test using the entire ~90-minute concert to make absolutely sure this issue can be put to rest...

I hope it does solve the problem.

Incidentally, what is the difference between the .mts files straight from the camera, and the m2ts files from Guidemenu? Is it just a case of renaming? I notice that the file sizes are identical.

-DONE- & Additional thoughts
 

Andreas' fix is golden. Once again, I DI'd 80 minutes of a live show I recorded into the HG10, placed the cam on a table, pressed record, and walked away. Five .mts files were created as the cam split the 80-minute clip according to its spec. I dragged & dropped the five files from the cam to my PC, ran the binary copy command, and waited. Took about 20 minutes total to recombine the five files into one .mts file. I then opened that file in Vegas. Next, I successively opened each of the component .mts files on the same timeline so I could easily see the "snap to" splice points. Muted the Vegas' spliced file's audio track and pressed play. The result was perfectly continuous audio at every splice point. LOL, I still can't believe how simple of a fix this turned out to be!

I was beginning to think this actually was a Vegas or CoreAVC PRO problem, so I set up a second PC with the various Elecard AVC offerings. Didn't matter - audio and vid drops at the joint.

It seems that Sony clearly has the more complete solution here. I'd guess that their cameras tag additional information onto to the beginning .mts file about how to correctly reassemble the clips via their software. That is much nicer than the Canon "fix" because you don't have to hunt around for all the separate clips and take care of the reassembly on your own <--which could lead to mistakes and additional time to redo it correctly.

One thing is for sure - it's really nice to have an NLE that can natively open these .mts files. It greatly reduced the time overhead in dealing with this manual assembly process.

All things are "simple" once you figure them out, but htis was a big BUG, any way you slice it, glad Andreas had a fix!! I think he's on to the cause of it too, probably a cadence thing of some sort.

It's absolutely silly that Canon couldn't have figured out the fix, but I guess that's what make DVinfo THE place - we can combine our expertise and "fix it in the field"!

MODERATOR - highly suggest this be made a STICKY, with the fix noted in the title... so anyone can find it fast! Since I might try the new Canon AVCHD cams, I'll know to look here for the fix!

Followup with Canon
 

Kudos to Aaron for characterizing the problem and to Andreas for theorizing the solution!

In addition to making this thread sticky, as a new owner of an HG10 myself, I would love to see Canon/Corel fix this issue as a patch (either camera firmware or as an update to the Guide Menu software provided by Corel).

I would kindly ask Aaron to followup his open ticket with Canon and provide them with the solution from this thread. It seems to my that it would not be all that difficult for the Guide Menu software to re-assemble the mts files while copying them from the camera to the PC. That way, we would end up with one m2ts clip for each start/stop during recording on the camera -- seems natural.

Dave.

P.S. It would also not hurt if other members of this forum also "discover" this problem and raise it with Canon -- may increase the priority of providing an official fix.

Friday's Email Response from Canon
 

they're still not getting it, LOL...

Dear Aaron J. Courtney:

Thank you for your reply. My name is Raymond and I am the Online Support Coordinator here at Canon ITS. Your message was forwarded to me for a reply.

In testing the HG10 camcorder mentioned in your email inquires it appears your issue may be computer resource related. Many things will factor into how your system interacts with any software and devices connected such as drive fragmentation, the speed of the hard drives, ATA versus SATA, processor and bus speed, and most importantly, RAM.

In testing on a 1.6GHZ Windows XP 1 GB RAM computer, we were able to reproduce the same problem using the supplied Corel applications.

On a 3.6GHZ Hyper threaded Windows XP 2GB RAM, no problems when using supplied Corel applications.

Also, for comparison, when using iMovie08 on a 2GHZ duel core iMac with 1GB RAM, there were no problems.

If you have not already defragmented your primary hard drive, you may want to consider running disk utilities to free any virtual memory that is being accessed.

Also if you are working with the files directly on the camcorder you may want to copy the AVCHD folder to a hard drive and work with it from there. This will eliminate any bottleneck that may be caused from the data transfer through the USB connection.

The issue you are encountering is not related to the data files being broken into sectors; rather, it appears to be a result of how your computer is handling those large files.

Please let us know if we can be of any further assistance with your HG10.

Thank you for choosing Canon.

Sincerely,

Raymond
Online Support Coordinator

As a new member of DVi and a new owner of the Canon HG10, I have been following this thread closely. With only a few days with the HG10, I have yet to run into this problem since I haven't recorded long enough to have split MTS files. But I know at some point I will, so finding this information will certainly prove to be invaluable. Thanks to all the hard work of you guys!

Working with AVCHD has certainly been a learning experience for me. I know a little bit about video formats and codecs, but finding the right software for this new video camera has been arduous at best.

First, I had to find out how to actually view the raw MTS files on my PC because I wanted to be able to quickly copy the files from the HG10 to my PC via Explorer and then open then up for immediate viewing. Purchasing the COREAVC product was the route I took, which required re-installing Haali Media Splitter because it was still trying to use FFDShow instead of CoreAVC at first. I really didn't want to install any of the included software that came with the HG10.

Now then, I use my Xbox 360 as a media center via a nifty program called TVersity. However, it won't work with the raw MTS files (not much does, it seems). So my current workflow is to use Windows Media Encoder to convert my MTS files to WMV-HD and they play flawlessly on my Xbox 360 and look great! So I can keep all of my actual video files on the PC and just stream them to the 360 when I want to view them on my HDTV.

So far I really like the HG10. My only main gripe with it is the lack of manual Gain adjustment. I don't like the high level of video noise when the camera boosts the gain in order to increase brightness. I did find that little trick that locks the exposure with no gain, but that's still not as good as having a manual gain adjustment. Other than this, however, I really enjoy using the HG10.

My email reply to Canon
 

Thanks Raymond for continuing this discussion with me. I appreciate it because I firmly believe this is going to soon become a much greater problem that Canon will be forced to deal with. Again, I don't believe we're talking about the same problem here.

I am not referring to stuttering issues due to insufficient PC resources in reference to the hardware demands the AVC codec imposes. Actually, my editing platform is more powerful than all three computers you list. Specifically, I have an Intel Core 2 Duo running @ 2.2GHz, 2GB of RAM, a 250GB SATA HDD for the O/S and apps and a separate hardware accelerated (AMCC 3Ware 9650SE Controller) RAID 0 array using three 250GB Seagate 7200.10 SATA-2 3.0Gb/s NCQ HDD's. This machine was built specifically for AVCHD editing and HDM authoring.

The problem with Canon's AVCHD implementation is quite simple. Canon does not provide a utility to reassemble multiple .mts files resulting from an extended video shoot (e.g., recording >~17 minutes in HXP mode). To be absolutely clear, here's what I did, which can be duplicated by ANYONE with a Canon AVCHD cam. I mic'd in an audio feed of a 90-minute rock concert that I had my audio mastering software play back from the beginning. The audio track gives you continuous, rhythmic sound which makes it incredibly easy to hear audio dropouts.

I placed my HG10 on a table, pressed play in the audio package to start the concert, simultaneously pressed record on the HG10, and walked away. Eighty minutes later, I returned and stopped the HG10 from further recording. I then connected the cam to my PC and browsed the HG10's HDD where I noticed that five new .mts files were created from this single 80-minute recording. Of course, a single new "clip" was presented for review in the cam's LCD window.

Using the provided Guide Menu software, I then transferred all five .mts files to my PC. And herein lies the real problem with Canon's AVCHD implementation. Instead of recombining the five .mts files back into a single continguous .m2ts file, the software simply copies the five .mts files into your chosen directory and renames each of the five files as <datetimestamp>.m2ts. When you playback these five files successively in ANY AVC player, NLE, converting utility, etc., you will experience a loss of audio and exactly two dropped video frames at each and every splice point where the previous .mts (or .m2ts - doesn't matter) file stops and the next successive file (in correct order) begins. Personally, I believe that the HG10's filesystem halts the audio recording in the first clip before it halts the video. This is why there are two dropped video frames in the second clip. Actually, I don't believe any video is actually lost, simply the audio and video are not sync'd correctly at the splice point. I think the audio preceeds the video in the successive .mts files and the AVCHD players and NLE's add two blank video frames to match the beginning of the audio.

What this means is that you cannot playback or edit any clip that exceeds 2GB of disk space without experiencing audio and video dropouts. The ONLY way anyone can playback a video of this length is IN THE CAMERA because the cam alone possesses the logic to splice back together the .mts files in the streams subdirectory.

Canon's backup software is of no help either because the software (correctly IMO) copies the entire filesystem of the HG10 to your PC.

So, I'm sorry, you did not successfully playback spliced footage from an HG10 on either the PC or the Mac without experiencing dropped audio and video around the 17-minute mark, assuming the footage was shot in HXP mode, because Canon does not provide a utility to reassemble these multiple .mts files back into the original seamless clip.

Fortunately for Canon, some bright individuals at dvinfo.net put their heads together where a work-around to the problem came from a user in New Zealand. As it stands right now, the only way to seamlessly splice these .mts files back into the orginal clip and avoid dropped audio and video is to use the copy /b command at a Windows command prompt.

I used this command on the five .mts files created from the 80-minute recording, opened the resulting 9GB .mts file in Vegas Pro and both the audio and video played back perfectly, without any drops at any of the original splice points. To make absolutely sure, I then opened in another video and audio track (on the same timeline) each of the five .mts files successively which identified exactly where each splice point originally existed. I carefully reviewed all four splice points along the timeline in the copy /b command spliced 9GB .mts file and confirmed that I could not find ANY dropped audio samples or video frames.

Although this is a workaround to this serious flaw, it is definitely not an ideal solution. Sony clearly has the best solution to this 2GB limit AVCHD problem because their software, upon transferring the clips to the PC, reassembles on the fly the multiple .mts files back into one contiguous file (representative of the point in time where the user pressed the button to start and stop recording) that seamlessly plays back in any decoding playback or NLE AVCHD compatible software.

Therefore, I, on behalf of all dvinfo.net HG10 owners, am requesting that additional software or firmware be provided by Canon that will correctly (i.e., seamlessly as the original recording was seamless) reassemble the multiple .mts files back into the single clip as it appears in the cam's playback menu window. If you care to, you can view the original thread documenting this issue from start to current work-around here http://dvinfo.net/conf/showthread.php?t=113083

Please respond with any further questions. And I would seriously appreciate your forwarding of this entire message to the team responsible for the HG10's (or Canon's AVCHD product) development.

Thanks.

Canon support
 

Bravo, Aaron! One suggestion for dealing with tech support: you might try producing a pair of "before/after" videos, one with and without the dropouts and posting the result online for them.

The following site has a channel devoted to videos created on the HG10:

http://www.vimeo.com/hg10

Dave.

Today's Canon Support E-mail
 

Looks like the information in this thread (and the thread itself, LOL) will make it to the right people...

Dear Aaron J. Courtney:

Thank you for the very detailed feedback on this issue. I apologize for the inconvenience this has caused you.

I have forwarded your inquiry to our product engineers for evaluation and testing. Once they have done this, we hope to have additional information for you.

Please feel free to contact us again if you have any additional questions or concerns.

Sincerely,

Ron
Sr. Contact Center Supervisor On-line Support Canon ITS


Boy, they sure are polite, LOL! But give me a fix over politeness any day. I'll follow-up to this thread when I receive more info...

Hi everyone,

If you playback the stitched-up file in your preferred media player, does it report the correct/new video time length (time stamp)? Can you seek through the file properly?

The reason I'm asking this is because some times I require joining few short clips together without the need of editing or re-encoding. In order to get the joined up file 100% compatible in all media players, I need to do the following:

1. Join the clips using either copy /b command or TSSplitter
2. Run the joined up file through tsremux and output as TS format (not M2TS)
3. Run the file again through tsMuxeR and output as TS format (not M2TS)

My HG10 is nearly full, and cannot record a long clip and test it at the moment.

Thank you all,
Riad

You can see it on this test clip I made a while back. I recorded an hour of my cell phone stop watch and then made a video clip. You can see the drop outs. (I spliced out the in between parts..the rest is losing frames between those times I list below).


Test here:

http://www.stage6.com/user/bas_vp/vi...le-Splice-Test

in HXP mode

file1: finish 18:03.2
file2: start 18:03.2 finish 36:01.7
file3: start 36:01.8 finish 54:00.7
file4: start 54:00.9

The unfortunate thing here is that Canon will probably not get too many calls on this for the simple reason that most people won't be shooting continuously for that long. The problem will therefore elude them. I'm not even sure that the casual shooter will feel compelled to call if he does see 1 glitch in his recording.

The more people that report this issue the more likely it is to get fixed. This is a significant issue and deserves immediate attention by Canon. It is not reasonable to assume people will be using DOS prompts to workaround this.

^^^Agree 100% and that's why I wrote what I did in my final follow up message to Canon. Hopefully, the people with the authority to actually do something about this issue will choose to properly remedy it.

I did a quick check using the included WinDVD junk media player with the stitched .m2ts file and it plays contiguously without dropped a/v at the original splice points. The player also displays the correct aggregated time. I would assume you could seek through the file also, except this lame software doesn't allow you to do that with any .m2ts file.

It's such a simple fix to implement, I think once called to their attention they should have no excuse for not fixing it - even a guy who hasn't programmed in decades should be able to implement the fix in about 2 minutes...

I agree that "most" people won't shoot longer clips, but I'll be the first soccer mom who records juniors first play and it hiccups should be able to convince them <wink> it's an important thing to have WORK RIGHT!

It'd be a deal killer for me...

Lucky for the big manufacturers they have such talented "beta testers" <VBG> who cna figure out the tricks and tell them how to fix the little bugs!

Pixela software update
 

As promised, I am following up with this thread with new information. Since Chris' evaluation of the HF10 and the accompanying Pixela transfer software appears to indicate that the discontiguous file transfer problem has been fixed, I decided to follow-up with Canon and request a copy of the software for evaluation with the HG10.

Their email response seems to indicate that the Pixela software is now included with the HG10 because the person who responded to my request apologized for my not having received the software with the camera. I'm still skeptical, however, because without the cam tagging successive files that belong to one contiguous clip, how is the software going to determine which files comprise the original >2GB clip. Perhaps the cam is already doing this and Guide Menu is simply ignoring that data. Don't know, but hopefully will know very soon...

Well I have some very disappointing news. Canon has decided to hang the HG10 users out to dry on this issue. They are not addressing it and refuse to accept responsibility for the problem, instead passing the buck to Corel. When I pressed the issue stating that the cam is useless for recording anything >17 minutes as it stands right now, the response effectively was "find different software."

This led to further discussion of the Pixela software, which the Canon support rep's official response was, "It's not supported with the HG10." Fine, I can accept that. When I asked for a copy, he told me Canon was not licensed to distribute the software with the HG10. He gave me a telephone number for contacting Pixela, which I did. I requested an eval copy of their software to try with the HG10. The Pixela support person then responded that they legally can no longer distribute their own software because Canon purchased the rights to the software, LOL! He recommended that I contact Canon's parts department in order to obtain a copy.

This prompted additional discussion referring to the ability of the Pixela software to correctly reassemble broken .mts clips back into the orginal contiguous clip presented in the cam's lcd. At this point he openly admitted to the software's 1 hour limitation, apparently meaning that if the original contguous clip exceeds 60 minutes in length, all you're going to get is 60 contiguous minutes. I don't know what is done with the remaining .mts files that, say, comprise a 90-minute shoot. What a joke. I told him this was completely unacceptable, and he mentioned he would submit the complaint to the software development team <-- which I'm sure he did because he appeared very sincere as opposed to the Canon support rep who continued to semantically dance around the whole issue.

So, where does that leave us with Canon's AVCHD cams? Well, we know the copy /b command does work with the HG10. I would assume it would also work with the HF10/100, but this remains unconfirmed. The Pixela software, according to their rep, will correctly reassemble the clips as long as the entire shoot is less than one hour in length. Who knows what happens at that point. Their software may totally mangle that splice point beyond repair; in which case you'd be better off not using their software and simply relying on the copy /b command if that's even possible with the HF10/100 since I'm wondering if Canon had to implement some form of data tagging to these .mts clips in the HF10/100 in order to get the Pixela software to correctly tag the .mts files that belong to a larger, contiguous file, thus avoiding choosing individual clips that don't belong to the longer file and tacking those on as well. I mean, you have to wonder how the software knows which clips belong together and which ones are individual takes.

This could indicate a difference between the HG10 and the HF10/100 in how the .mts clips are laid to storage since Canon apparently discovered this problem after the HG10 was released yet before releasing the HF10/100, and they may have implemented a firmware revision to the HF10/100 knowing they would be using the Pixela software instead of the Corel software.

I'd really wish someone with a Sony AVCHD cam would record a 90-120 minute clip and see how the PMB software handles that length because I was under the impression that somehow Pixela was involved in the Sony software as well. HEY, and ANYONE WITH A PANNY AVCHD PLEASE JUMP IN HERE and let us know how their current AVCHD cam's handle long recordings and the subsequent transfer to a PC...

Hi Aaron -
Ahhh, the ol' "corporate two step" one of the funniest dances on the face of the planet, unless you're the dance partner!

I'll switch the quality setting and let a CX7 run out of curiousity - never contemplated more than an hour (TAPE based mental limitation!!), but I have run an hour on a 4G stick, and it stitches fine. With an 8G I think I can get 2 hours at decent res, so I'll have to test.

Still want to try that copy/b hack, as it appears simple enough.

I think sometimes the end user has to accept that the designers have a specific viewpoint and simply never contemplate the creative possibilities (TWO hours on a single piece of media, who would think of such a thing <wink>!).

THIS is how technology moves forward. Someone creates an artificial barrier, and people with creative minds and stout craniums bust through it at some point and new and cool things happen... it's just tough being on the "bleeding edge" of the revolution. Patience is a virtue, obviously, but playing with new toys is fun with the right attitude! Pain reliever of your choice also sometimes helps....

My take would be if the copy/b hack works, who cares if the software guys didn't get enough Jolt cola and pizza to make it work right the first time? Wouldn't be the first time a "workaround" was required! That said, don't give up just yet, keep squeaking, and look out for that gigantic gooey glob of slippery stuff in case it comes your way!

Dave

PS - may be a few days before I get to run a test, and I'm going to guess that Panasonic owners just don't take long clips for some strange resason! I'm sure the same issues are there with file size and need to stitch...

LOL! Thanks Dave, I appreciate your help.

hello everyone, i am new to this forum. i have the hg10 camcorder for just a few weeks now. overall i love the camera, however, i have a few questions that, after reading this thread, i am sure you guys can help.

1st- i am not sure how to implement the copy/b workaround to fix the seaming issue after 20 minutes of recording. could you explain a little more in detail on how to that application.

2nd- is there anyway to convert the .mts or .m2ts files into something that can be emailed or posted online. i cant seem to do anything with these files. i have a ton of clips from a recent rock climbing trip and i want to get some clips to my friends. i only have the coral software that came with the camera and it seems that making a dvd of just raw footage is time consuming and very slow and even too big for a dual layer, as the software won't allow me to burn more than 100 clips.

being new at this i am not sure what type of info you need. it seems you guys understand this camera better then canon. any suggestions would be greatly appreciated

thanks
michael

Hey Michael, first, assuming you're on a Windoze PC, simply open a command prompt, cd (change directory) to the path where the .mts files exist, and type at the cursor: copy /b <first>.mts+<second>.mts+<third>.mts <originalclip>.mts where <first>, <second>, and <third> represent the original filenames and <originalclip> represents the name of the contiguous clip that you desire.

It helps tremendously to have an editing software package or playback software that can view the original .mts files; otherwise, you're guessing about which files belong to which clip.

If you're going to want to do any degree of serious video editing, then you'll probably want to pick up a third party software package. Otherwise, you may want to look at/wait for Canon's DW-100.

hello Aaron,
thanks for the quick response, i will try it soon.
any info. on how to convert single clips to something more usable with the rest of the world. ie. email, youtube, facebook uploading.

this would make it so much easier to share videos with other people, rather then having to edit, convert and transfer to dvd and then give it to someone. i am sure you understand. it requires less work and isn't that what it's all about... more for less

I agree that the 2GB limit is stupid. But after spending a number of years in the computer industry, I think I understand why they're doing it. It's a Cover Your Ass thing. They're taking the path of least resistance and do not assume any risks, no matter how remote. "It ain't broken, so why fix it?"
Except, of course, it is broken, but in a more subtle fashion.

Also, in a crooked way, it's a "good thing" that they split the files the way they do. A binary copy is something one can do on any computer, on any OS, no matter what's the software. It's also a trivial operation. It could have been much worse.

It also explains the corruption at the beginning and end of the files - probably the last frame, or the last GOP or so, gets truncated, so either the video or the audio track, or both, get corrupted.

Yeah, they also don't assume any responsibility for their decision to have to rely on faulty third-party software. At this point in time, the only advantage AVCHD has over HDV is convenience in the acquisition phase. On every other point, it's second rate, IMO. With Canon's turnkey-less AVCHD implementation (unlike Sony's), HDV is arguably still the better option and by a wide margin at that.