Combining many nano clips
 

What's the best way to turn a whole bunch of 3.5GB nano files into one file.

QT .mov 100Mb/s long-GOP nano files transfered from a nexto to a MacPro's (OSX 10.6.1) attached fast drive(s) with filesystems that can handle huge files. I don't want to edit anything, I just want turn all the little files into a bigger file. I have FCP7, CS4, and pretty much anything I need. A lossless combining of course, no re-encoding; the fastest, easiest mindless method. (I know I can't just concatenate the files in UNIX due to the container headers, etc).

I can load them all into FC7, and output a single file, but is that the only way? (And I don't want a reference QT movie).

Thanks!

Billy

cat *.mov > large_file.mov

seems to work but I'd need to do more testing to be sure. FCP can also do this without re-encoding. Let me see if I can remember how. I have no need to do this so I can't remember exactly..

I just tested with mpegstreamclip too. It works just as well as cat does from the command line, but you also get a nifty progress bar to stare at. cat gives you no progress.

just select all the files in finder, right click, choose open with mpegstreamclip. save as new big file.

Also if you are doing cat, I would probably recommend only concatenating clips from a single continuous shot. I think if you combine clips from multiple shots, the GOP pattern might be interrupted since the the run out on the last clip of one shot won't match the beginning of the pattern on the new shot. This is that testing I talked about.

Now if only there was a PC method...

PC's don't have cat? That's just crazy to me. It's like the most basic command line utility of any platform.

looks like copy can do the same thing as cat.

copy /B file1.mov+file2.mov+file3.mov large_file.mov

/B is for binary.

someone should test it.

Hi Aaron-
We use a closed GOP structure on the XDR/nano, so I can't imagine any problems contatenating any clips together, even from different shots. Also our files always end on a GOP boundary, so no issues butting files together.

Best-

Ok, since Mike cleared up the GOP pattern issue, I see no problem at all with concatenating files (regardless of OS).

Can anyone else see a problem with this? I've done about a dozen tests this morning and I can't find any issue with it.

It's cheap (free), requires no re-encode, and works on any platform.

This thread is very interesting, since the "subfiles" that may accompany a clip are clumsy at best, error prone for sure.

I'm looking into this little app:

Peretek, Inc. - Simple File Joiner

The key is to make it idiot proof. Shouldn't need to enter the file name(s) since that would introduce a great error potential.

If this utility doesn't work, I might have to break out my Visual Basic kit.

Combining sems to work for QT files. What about MFX?

And as it seems QT can be combined, is this true for MXF?

Combining is an issue for me as I know most editors won't like the "sub clips."

And right now, QuickTime from the nanoFLASH for PC consumption ain't happening. So MXF, which is a wrapper...might not like being combined.

If you don't like entering the file names, use automator to make a droplet on your desktop. Just select all the files and drop it on the droplet and a few moments later a joined file will be made.

Ummm. Let me hazard... you're talking mac, I believe.

Yes, automating something like this is easy as pie on a mac. No scripting needed, drag and drop simplicity.

Just drop your clips in a conformed sequence and with "Two Mono channels" and export without re-compress.
In case there is any GOP to be rebuild (cuts) will be at 50Mbps, but only these GPOs.
rafael

On a related note, a Panasonic tech gave me some DOS commands to stitch together AVCHD clips from the HMC-150.

This did not re-encode anything, but was not instant as it seemingly had to go through the files bit by bit.

This is for the PC btw.

Here is what he sent:

<Procedure to merge files>
1. Copy all divided files in a specific folder on the NTFS drive(ex.
C:\AVCHD Temp)
2. If the divided files are 1000.m2ts, 1001.m2ts and 1002.m2ts, Run
"copy /b 1000.m2ts+1001.m2ts+1002.m2ts AVCHDfile.m2ts" via [Command
Prompt].
3. Then divided files are merged into a single file (AVCHDfile.m2ts in
the example).
4. Import the merged file into EDIUS.


I then showed this and a member wrote a little utility with a windows interface that let one pick all of the files to merge and the choose a destination and filename.

This is really useful for live events where the record is running for a while. One clip in the editor means one filter and adjustment application before cutting.

Hope this helps.

Tim -

Can you provide a link to the fellow's program? I'd like to test it.

Ah. Did a little detective work and found it.

Testing time.

Thanks for all the info; I'll be testing it out later this week and see how it goes.

I could have sworn that all QT .mov files had header info, and so thought that a simple concatenation from the command line wouldn't work, but I'll be delighted to see that I was wrong. :)

Or could it just be that QT can play the files and deal with a bunch of headers transparently to us... (Though running the files through MPEG Streamclip will take care of that, as will running them through FCP). I'll try combining both ways (command line and streamclip) and see if the output files are the same.

Billy

My initial testing shows the timecode is in the new file exactly how it should be. Which makes me question once again, if you would want to do this with files there were not a continuous shot.

Some software can not handle timecode breaks in the mpeg stream properly and if it could, would you want it to?

I'll be combining files from continuous shots only.

I have no intention of combining non-sequential files (not sure why you thought that was what I had in mind). What I was talking about was whether QT .mov files had headers, (I thought they did) and if so, how come concatenating smaller (sequential) files worked. If they do have headers, then QT is smart enought to be able to deal with multiple headers in a single file. More likely, QT doesn't have/need headers for this type of file. In any case:

I'll know more (maybe...) later this week.

Billy

Hi,

perhaps I'm missing something, but I tried concatenating (gluing together) 2 Quicktime files with the terminal command line cat function, the result simply plays the 1st file and that's it, in Qt Player.
Kind of what I expected, as each file has indexing info and a simple concatenation of files wouldn't account for this.

I concatenated 2 mxf files and it wouldn't even open in XDCAM Viewer.

I can believe that m2ts files (as mentioned above by Tim Polster) would concatenate successfully, since they don't have any indexing info in the file.
The same is probably true of nanoFlash .MPG files.

Tommy Schell

stitching together MXF would probably not work. Quicktime files from XDR work fine for me though.

So is there any hope of an MXF clip-join utility?

Bear in mind that this would only be useful if it is quick without needing to render anything.

I'm with Tommy here. There are definitely headers on the QT .mov files, and simply concatenating multiple files together does NOT work. Quicktime simply plays back the first segment and doesn't know about the subsequent ones. I recorded a bunch of clips yesterday and tested them out last night.

I also ran the files through MPEG Streamclip, and it was able to losslessly process the multiple clips into a single clip that Quicktime was happy with. (Use "Save As"). Examining the combined file with a hex editor revealed that MPEG Streamclip had done the right thing, with a single header defining the combined file properly, and no extra headers buried throughout the file.

So I don't know exactly what Aaron is seeing, but all my testing shows that simply concatenating nano QT .mov files fdoes NOT work.

Billy