AE50 - Do you convert to intermediate?
 

I'm doing a batch script to copy files from card to pc (joining spanned files) and convert to intermediate format (and/or create "down-sized" SD version as well)

how you guys edit the videos from ae50? native editing? convert to some intermediate format?

i use Canopus HQ for a long time for intermediate/export purposes, and a while back the codecs became free, so now is my solid choice.

also, if wanted, i pretend to release the script after testing and making a proper installer.

i started this because i like to came home from the event and go to sleep. so the script do "everything" unattended. (detect memory card > ask for output folder > copy and join files to disk > verify file count > eject memory device > convert to intermediate (HD) convert to intermediate (SD). its also "multi-threaded" (multiple encoding's at same time)

Re: AE50 - Do you convert to intermediate?
 

I edit native in premeire pro. I see absolutely no point in using an Intermediate codec. It is a unnecessary time wasting step for me..... obviously if you are a film maker you may need to for color correction purposes. But i find h264 stays together quite well with slight tweaks....

Re: AE50 - Do you convert to intermediate?
 

I do work with Edius and edit natively which is a decission mainly because of space on the harddrive I"m trying to save (Canopus hq files can be 3 to 4 times the size) and that works out well, everything I do is realtime, including colorcorrecting BUT if space wasn't an issue I would convert every project to canopus HQ, the workflow is much faster in that codec, everything, including scrubbing the timeline, is butter smooth. You have to experience it to see the benefits.

Marlon: That is very cool, are you releasing the batchscript for free? Too bad it can only do one card unattended, would be great if you could attach several cardreaders at once and it would convert one by one.

Re: AE50 - Do you convert to intermediate?
 

i use vegas, editing AVCHD is good, not amazing, but good. the main problems is with long projects, start to become sluggish. with intermediate files everything is ultra-fast. i also use external encoder, so exporting is also very fast.

yes, i will release for free. just need more testing and testing. i think i can "implement" multiple card-readers also ;)

Re: AE50 - Do you convert to intermediate?
 

Marlon, would it be easy to modify the script to perform some more basic copy paste functions? I was thinking of attaching several cardreaders at once (as after a wedding there are always several cf cards to be copied) so that each cardreader gets a drive nr and then let the batch software run to copy all the data from the cards to 2 different drives of which one would be a backup?

How much larger are the canopus hq avi files compared to the native avchd files?

Re: AE50 - Do you convert to intermediate?
 

yes, i already did last night more than one card reader function. just need to implement in the whole thing.

backup function i was thinking also. maybe i put a function to ask if backup is wanted, so it copy the video only, merging the spanned videos to one place, and the original files from card + playlist + CPI files to another place (keeping original files as backup instead the processed ones, a.k.a. "full card backup")

canpus hq have normally 4~5x the filesize at 24mbps (about 50~60gb hour)

the convert functions will be optional also. i started this just to be a automated copy script, but the spanned files where a problem (files past 2gb). so i found a way to "analyze" the playlists in the AVCHD folder, detect which files are spanned, and join them directly on copy to disk (and after copy, rename to original recording date/hour for better organization and management)

Re: AE50 - Do you convert to intermediate?
 

I work with the good old Final Cut Pro 7.
I convert all the material to ProRes LT and work on it fluidly with my 6 1/21/2 years old MacBook Pro.

I consider changing to Premiere Pro or Avid MC as I don't like FinalCut Pro X.

With Premiere Pro CS6 I can just import the material without transcoding. But it's not really fluidly running if you start to add lots of footage or some FX. I know I'll need a new computer soon, but the camera put a hole in my pockets. No money left...

Re: AE50 - Do you convert to intermediate?
 

great stuff Marlon, will be very interested to see the end version.

Re: AE50 - Do you convert to intermediate?
 

thanks for the input guys. will release when ready ;)

Re: AE50 - Do you convert to intermediate?
 

Weird no-one else responded to this, seems there is not much interest in such a piece of software? I think it would be awesome to plug in all my cards in several readers, start up a batchscript and go to sleep to find all my data on my harddrive with a backup when I wake up.

Re: AE50 - Do you convert to intermediate?
 

maybe not everyone have a large amount of events/number of cameras to make a big deal out of this.

i also don't have that much need for automation, but the fact is, i automate, do shortcuts in everything i can to speed up workflow without compromise quality, that i can now see the result: "more free time" ;)

i do shortcuts for multiple vegas filters (plugin manager) i do a bunch of scripts to speed work, i did a automated encoder for dvd (convert exported file to video + audio in few clicks), scripts to scan video for black frames, batch to record dvd-r with single click, etc...

lot of time wasted to make them, but results are amazing ;)

Re: AE50 - Do you convert to intermediate?
 

With FinalCut Pro things are so easy - no need for further scripts.

I copy the card to my backup drive.

Then import into FinalCut, where all clips are transcoded to the codec I choose. With my old labtop, that needs some time, but I can go sleep...

Re: AE50 - Do you convert to intermediate?
 

Edius can do this too but that's not what Marlon script does, it appears to go futher then that, if I understand it right it should be possible to connect several cardreaders to your pc, put all your cards in it that you used at a wedding and start up the script that could take the data from each card, join spanned clips, and move all footage from all cards to a folder, or convert to a intermediate codec and then move the entire folder structure from each card to a sepparate folder/harddrive as back up, all while you are sleeping. :)

Well, that is what I would consider a major timesaver if Marlon could pull something like this off, I wouldn't even mind paying for such a script.

Re: AE50 - Do you convert to intermediate?
 

Sounds great - but only in the windows world which I left happily years ago...

Re: AE50 - Do you convert to intermediate?
 

"only in the windows world which I left happily years ago"
Watch out, this is how windows vs mac bashing usually starts, better to concentrate on the good stuff, no matter what OS you are using. :)

Re: AE50 - Do you convert to intermediate?
 

i think i will release simpler versions first to test it, then later implement encoding functions.

will be easy to debug if i start simple.

will start with multi-card/copy/merge/rename/backup functions, then later add the extras ;)

OBS: the playlist format (required for parsing) looks like the same for all NEX cameras, but i can only test with nex-f3 and nex-ae50 (when arrive, still waiting ;P ). probably will not work with "non-nex" files.

Re: AE50 - Do you convert to intermediate?
 

Marlon ~

Regardless of what anyeones says keep going. This sounds like a great time saver. Like someone else said there are people out there who need this sort of functionality and would be willing to pay for it.

Chris Young
CYV Productions
Sydney.

Re: AE50 - Do you convert to intermediate?
 

first functional version is ready. still didn't implemented backup and multicard-reader functions yet, but the conversion process is working fine.

for now, you can run it and select if want to copy files to hard drive only (merging spanned files and renaming to the video date), or copy and convert to Canopus HQ in HD or SD or both at the same time.

also added right-click option in folders so you can just convert already copied files in the hard drive to intermediate/sd

i got a few ideas, but need more info. how you guys do backups of the video? cause i use one external HD and internals ones. (copy all videos to external and internal to have it in 2 places).

who use external backup, do you have more than one or a single one? how is your folder structure? just a folder on the root of the drive?

im recording in Full-HD 60p, to deliver in DVD (30i) and Blu-ray (30i also). how you guys deliver the video? that is the final format/video specs?

Re: AE50 - Do you convert to intermediate?
 

I create a project directory and sub directories for each camera, use Sony software to transfer from cameras to appropriate directory then backup the whole to LTO3 data tape . When I finish my editing I delete from the hard drive. Do the same for the other edited files including DVD files and Bluray. Like Noa I use Edius for my multicam editing native AVCHD and Vegas and Sound Forge for audio.

Ron Evans

Re: AE50 - Do you convert to intermediate?
 

Hi Marlon!

Is it possible to copy files to hard drive only (merging spanned files and renaming to the video date) and right after that convert all to HQ HD and/or SD? (without the extra rightclick to select the option when it's on the drive.
I think for rough edits on multicams (I use up to 4 camera's now) it might be good to have it batch copy and after that convert to a intermediate of choice and let that run over night. Then edit in a hq sd file for the first rough cut and connect native files again once you start colorcorrecting. Editing 4 HQ SD streams should be space saving and edit like a knive through butter.

Currenty I back up the contents of the sd card with sony's software to a "import" drive and then back that up to a internal "edit" drive, I make another backup from that edit drive including all projectfolders to a external drive at the end of the day as well.

Re: AE50 - Do you convert to intermediate?
 

Ron Evans, LTO3 data tape works as a "hard drive-like" or its something that need proprietary software or something to transfer?

yes, you can copy only or copy and convert, no interation needed

SD files are a bit smaller than original files (original 25-28mbps, converted about 22mbps) and yes, editing is very good ;P

i previously did a few proxy scripts for vegas, and they work perfectly. a few friends i have in the Brazilian forum edit h.264 videos from DSLR, and the video have a very high bitrate (about 44mps) making decoding even more slow than AVCHD at 25~28mbps.

the great thing is i did a script (for sony vegas) to change between files in the timeline: one click you can change the smaller ones for the original, one click again can change to proxy/sd/whatever wanted, in the middle of a project, without screwing up anything. maybe i can make it more friendly as a proxy creator also.

i was thinking about something like this: a marker file to make the program identify the backup drive and copy everything there for backup. it can be internal or external drive, so, even changing the drive letter/usb port/multiple external backup drives it can identify the connected one and backup there, so no need to specify it each time.

Re: AE50 - Do you convert to intermediate?
 

Backup tape drives need special software as they do not appear in Windows. I use Retrospect 8.1 as the backup software. Unlike Noa I am happy to edit native files so really have no need for an intermediate at all. I am interested in saving time transferring from 4 cameras I normally use for the events I shoot, even two at a time would be great. It really doesn't take me that long now though. I shoot theatre so projects are about 2 hours in length. It now takes about 12 mins a camera using the Sony software so less than an hour to transfer from the 4 cameras.

Ron Evans

Re: AE50 - Do you convert to intermediate?
 

nice, i will implement multi-card readers and backup option as soon as i can ;)

Re: AE50 - Do you convert to intermediate?
 

Ron, it's not that I don't like editing native files but if I"m adding 4 streams of native footage in a multicam sequence editing in a intermediate format makes it a much smoother experience, no matter how powerfull your system is. The biggest gain you will see is when fast scrolling your timeline. Once I have done a rough cut I delete the intermediate files and relink to the native ones for color correction and exporting. Also exporting hq files to a hq master is much faster then native files, on my older system I can export a one hour timeline under 10 minutes. Only if I would edit with hq sd files I would need to relink to the native files to be able to get HD files if needed or to go to blu-ray. I would only be interested in editing this hq sd because of the space you save and the visual quality is more then enough to do rough cutting and multicam editing is butter smooth in that way.

Re: AE50 - Do you convert to intermediate?
 

I don't scroll much as I effectively play the theatre show and switch cameras. Filters off while in multicam runs smooth enough on my i7 2600K system for 4 tracks of AVCHD and 1 track of EX3 all native. At most I run a slightly fast pace scrolling with my Shuttle Pro V2. I do all the colour balancing etc before I start and rarely need to do much afterwards to match some edits that do not quite work. I too export a HQ file to go to TMPGenc for DVD encoding. I tend to work in 5 min segments, switch out of multicam and watch what I have edited and fine tune the filter changes then if needed. Encode for Bluray with Quicksync straight from timeline. Do most all my authoring in DVD Architect 5.2

Ron Evans

Re: AE50 - Do you convert to intermediate?
 

I fast scroll all the time which saves me quite some time going quickly through the edit and choosing camera angles as I move on, once the rough edit is done I let it play from the start and finetune.

Looking forward to try the conversion tool Marlon. :)

Re: AE50 - Do you convert to intermediate?
 

I need to listen to the dialogue in a play or the beat for a dance show so really do not scroll fast unless it is a monologue or solo dance. I have settled on this short 5 mins or so segmented way of doing things that is effective for me. It is a serious retirement hobby for me so that does make a difference !!! I can finish a 2 hour show edit in a day. I may make a few changes after I watch the DVD so usually cycle the finished DVD twice before I am happy. Then just make the Bluray as the authoring is the same in DVDArchitect just substitute the Bluray file and change properties.

Ron

Re: AE50 - Do you convert to intermediate?
 

backup its now working.

just need to create a text file named "BACKUPHD.TXT" in the root of the drive, so the program will identify it as backup drive. if wanted, just put the folder name inside the text file (like "NEX BACKUP"), so the backup will be created in that drive, in the NEX BACKUP directory instead of the root folder (like if you use that drive for something more than video backup, so its more organized).

now the program ask for the filename of the destination directory in the selected folder. this remove the needed to manually create the folder on both drives before copying the files, making handy and simpler

if the program detect a drive with the BACKUPHD.TXT file, it present the backup options (yes/no) for those who don't do backups.

now for the multicard functions... (this will be "tricky" ;P )

Re: AE50 - Do you convert to intermediate?
 

Everything done. file copy function, encoding, backup, multi-card readers (up to 6)

just need to add option to install codec in the installer (if encoding function is wanted) and more testing.

Re: AE50 - Do you convert to intermediate?
 

Very nice Marlon! Can't wait to try. :)

Re: AE50 - Do you convert to intermediate?
 

well, its done.

be warned to test only in non-critical videos. im not responsible for any damage it may cause. use it "on your own"

ive made 2 separated installers, the codec and the program itself. if you do not want to do conversion (to intermediate, HD and SD) do not need to install the codec (Canopus HQ codec, x32 and x64)

first, if you want the option to backup all the card to a drive/external drive, create in the root of the backup drive a file named "BACKUPHD.txt" (if windows is set to hide extensions, will be BACKUPHD only). if you want the backup be made inside a directory in the backup drive, just type the folder name inside the text file (like "AE50 backup folder", with no quotes)

install the program

put the cards on the card readers (up to six) all together.

start the program on the start menu shortcut (AutoAVCHD v0.4)

it will show found drives, and it will ask for the output folder. this is your "work folder" or hard drive (it will create later a folder inside)

then choose if want only to copy files, with no conversion, or copying plus conversion to intermediate

if you created a backup file in the backup drive, it will ask if you want backup

then, type the folder name for the event (example: 2013-01-01 new year video), press enter

now sit and relax. i mean, see if it is working properly ;P

if file conversion was choose, it will encode after everything was copied.

verify output folders to see if everything is OK.

OBS:
to choose the options on the program just type the number of it and press enter.

do not use uncommon characters, preferably alphabet + dot, dash, space, underline.

tested only in Windows 7 x64, but should work on Windows 7 x32 too.

it only work with SD cards, not memory stick cards. the folder structure is different, later it will work on both cards type.

if you selected only to copy the files, but later you want to convert to intermediate, just right click on the desired folder, and go to "AutoAVCHD". it will give options to convert already copied files on the HD. (also is recursive, so it can convert all cards folders inside the main folder)

if any problem occur, please be descriptive about it. "didn't work" does not help.

AutoAVCHD:
AutoAVCHD_v4b2.rar

Codec:
AutoAVCHD_Codec_v0.1.rar

Re: AE50 - Do you convert to intermediate?
 

Great! :) could it be that cards from a canon 550d won't work? It didn't recognize the card, it did work with a card from my ea50 but I got a pop up error message first which said it couldn't find the drive and it mentioned "device\harddisc\dr6", I had to press close several times to get rid of this notice, after that I saw another box appear asking for the location of the files and from there on it worked.

What I wanted to ask, after the file transfer, can it do auto back up to another drive?

Thx for all the hard work :)

Re: AE50 - Do you convert to intermediate?
 

it only work with sony nex cameras. DSLR i believe you don't need something special, cause the file splitting is independent, so just copying is enough.

about the popup notice, i believe is due to empty card readers. i will try to find another method to detect drives that don't cause that ;)

if you created the BACKUPHD.TXT file i mentioned in the backup drive, it will do backup before file copy to the "main" drive. did it say it found the backup drive before asking the output name?

Re: AE50 - Do you convert to intermediate?
 

new version!

AutoAVCHD: (b3)
AutoAVCHD_v4b3.exe

Codec (v2)
AutoAVCHD_Codec_v0.2.exe

uninstall the codec and the application if you had installed previous versions before installing newer ones. (uninstall via "programs and features" applet in control panel, the start menu shortcut had a bug)

Changelog (v2 and b3):
# Fixed file encode mode (path problem)
# Fixed Uninstall shortcut (both programs had the same one, now separated)
# Fixed errors when scanning drives (new method used, no more errors on empty card readers)
# Fixed codec in Windows 7 x32
# Now tested in Win7 x32 and Win7 x64

Re: AE50 - Do you convert to intermediate?
 

Hi Erik, I too am using FCP7. Although I have recently been using clipwrap to transcode to Pros Res LT due to FCP7's inability to transcode 1080 50p. What do you archive? Do you keep the pro res quicktime files and the AVCHD PRIVATE folder?

Re: AE50 - Do you convert to intermediate?
 

I still have to test it but will let you know my findings as soon as I can.

Re: AE50 - Do you convert to intermediate?
 

I am in a kind of testing phase with my EA-50 - as I always was with DSLR videography.

I am definitely not that happy with the codec on the EA50. Thats why I convert to ProRes. I like to have a good codec for editing, but converting means a lot of time for me. I archive the ProRes files but hold the AVCHD folder structure too.

At this time I think I'll switch to another workflow this year. I have buy a replacement for my old MacBook Pro but am still not sure about what to come.

Sorry, but I am still not to confident about my workflow to recommend for anybody else. And it seems I am the only one not filming weddings at all... ;)

Re: AE50 - Do you convert to intermediate?
 

Erik, I suggest you look at Lightworks v11.1 as it is the fastest yet editing native AVCHD. However, like you I prefer to transcode AVCHD to ProRes or better still, Avid's DNxHD 185 10 bit as it works faster on the PC.

Lightworks is in 'beta' on Unix and on Mac very soon too, then all three will update together.

Re: AE50 - Do you convert to intermediate?
 

Is there a benchmark out there with all existing NLE"s to back up that claim? :)

Re: AE50 - Do you convert to intermediate?
 

Well Noa, Lightworks is one of the few NLEs apart from Avid used by professional editors to edit Hollywood movies, not weddings. But if you read what I actually wrote, you will see that I referred to the new version v11.1 being the 'fastest yet to edit Native AVCHD' That is: the fastest version of Lightworks to date.