Timecode preservation in HDV production
 

I would like to confirm which NLE tools will preserve the HDV source tape timecode on transfer from tape to harddrive, and whether the timecode that is sent out by the Z1 on HDV downsampled to DV output is the same as the timecode that is sent out as HDV.

Specifically, I would like to confirm that source tape timecode is always preserved on transfer to a drive, and that the same source tape timecode is used whether the file is transferred as HDV or DV.

The message that got me started down this path is this one:
"The Z1 looks much like the PD 170 in DV mode, it's a very good good DV camera, not amazingly remarkable. You can shoot in HDV, downconvert in the camera to SD, and save your EDL for a later re-edit in HDV if you'd like. That's the most cost-effective use of HDV cams at the moment."

Thanks,
GB

Virtually all NLE systems will retain the timecode from the DV or HDV tape on capture. If you use Cineform's tools in Vegas, however, you'll have to use a .txt EDL to recapture the media at a later date. Vegas can't capture HDV natively, it uses an intermediary. On the other hand, if you shoot HDV, capture as DV while downconverting in the camera, the T/C is retained, and the media may be recaptured later. The problem is, a lot of NLE's can't do a recapture of the HDV as native HDV and you'll be locked into using a 3rd party tool at this particular moment. That will likely change in a very dramatic way at NAB next month.
As I mentioned in the other thread, the camera embeds T/C. It's always there.
For DV, this is no issue at all for any application I'm aware of. For HDV using Cineform, you'll have to manually locate T/C, and their tool allows for that. I don't trust their T/C locator, however, and they don't have presetable I/O points. Avid isn't ready to go with their HDV solution, neither is FCP, although both have announced new versions for HDV in the near future. Premiere is now capable of batching HDV, and Ulead Media Studio Pro is not. I don't know where Pinnacle stands at the moment.

Thank you for your response. This seems at odds with this thread:
http://www.dvinfo.net/conf/showthread.php?s=&threadid=39647
which indicates timecode preservation is _not_ the norm?

But if timecode preservation is supported, and the same timecode values are output whether in HDV or HDV-downsampled-to-DV then I have no worries ...

GB

Timecode is currently only supported by Ulead Media Studio Pro.

BTW, I'm still looking for raw footage (*.m2t file) from a Z1. Preferably with timecode set to free run, drop frame and in 1080i60 mode. I need a short piece with several start/stops.

OK, now I'm worried again! Without timecode preservation I can't commit to HDV as a professional tool ...

Maybe I'm missing something here, but if the T/C is NOT on the tape then all but the most basic capture would not be possible. It is inconceivable that Sony would deliver a digital camera that does NOT store the T/C on the tape, regardless of whether it is DV or HDV.

I can't see how the process of downsampling HDV to DV by the camera during capture would have any effect on T/C. The T/C would be provided for every frame.

DISCLAIMER
I don't own, nor have I used, the Sony F1/Z1. So I can't state with absolute certainty that it always provides T/C during capture.

There is no question that TC _is_ on tape -- this is part of the spec, and displays on the device itself. The question is does the transfer software pay attention to it. Similarly on HDV>DV downsampling I'd expect it would be up to the hardware device -- if the device doesn't send the source tape timecode out on the Firewire link, clearly the transfer software isn't going to get it.

Despite an assurance early in this thread that most do, I'm having a hard time reconciling that with other posters who indicate timecode preservation on transfer is a pipedream at this point with most solutions. If I'm reading this post from Cineform's David Newman properly, this is a 'coming soon' feature:
http://www.dvinfo.net/conf/showthread.php?s=&threadid=38395

I'd have thought my initial question straight forward, but I'm confused by the apparently contradictory responses I've had.

GB

Again, Geoff, it's entirely dependent on the software. Check with your software manufacturer and they'll be able to answer the question. The T/C is part of the Z1, and the T/C is always there.

Versatile Timecode/Userbit Operation
Timecode/Userbit Preset function allows any time to be selected as a starting timecode. Also, "DF" or "NDF," "Rec Run" or "Free Run" and "Regenerate" or "Preset" can be selected for the timecode operation.

Look at timecode for HDV like EXIF data for still digital pictures. If you modify your orginal picture ina cheap program, all your EXIF data is lost.

Sidenote: I hate calling HDV, HDV. From now on imma call it HD. I got in a technical batle the other night with some Direcotr and his silly DP who tried to call my camera "Fake HD". Boy you shoulda seen me. I ripped them both a new a$$ and left them speechless and dumbfounded. Anyhow....

Timecode "IS" recorded to the "MiniDV Cassette" when your camera is in HD mode. That timecode is THE SAME timecode that camera uses when Downconverted to DV. Remember guys, the minidv tape was recorded in HiDef, so where do you think the Downconverted DV footage is getting it's timecode from??? it surely isn't 'making it up' all on its own. its getting it from the HD source on the tape.

So....when capturing downconverted DV, we get good timecode, no problem, not an issue. It doesn't matter what point you start playing the cassette, the correct time at that point will show up. (remember, its getting that info from the HD sources timecode)

BUT....the problem is, the current 'HD capturing' software we have available to us, is not yet HD TIMECODE READY as all the DV capturing software programs are. Ofcourse the camera itself can read the timecode and convert it for DV, heck, sony built the thing. But none of the software programs can yet.

however, do not fret....they will. and when they do....HD workflow is going to be the sweetest thing on earth.

I remember back when I cut a project on FCP 3, I learned about this thing called OfflineRT or something liek that. I had a P3 Pismo Powerbook and no external drive. So I used the Offline RT mode to capture the footage at this very small & easlily edittible mpg 320x180 size files that fit like 1 hour of footage in 1gigbayte or someting like that. It's timecode was identical as the large DV. SO all I did was cut the OfflineRT footage. The computer moved fast, transitions were realtime, things were just 'easy'. SO when I was done, I simply told FCP3 that I lost all my footage and I need to re-capture it again, but this time, I pointed the recapture to the real DV footage. As a result, FCP3 re-captured only the footage IT NEEDED (not the whole tape) based on timecode, and printed me a final version of the project with collor correction, transitions and everything in tact. it was sweet. and FREAKIN FAST!!

I plan to do the same thing once Vegas starts doing it's HD(v) thing. SInce my computers are twice as fast and hard drive spaqce is 10x's as large as back in the pismo/fcp3 days, I will simply Capture as DV, cut the project QUICK AND FAST, then tell Vegas that I 'Lost' the footage, and re-capture it again in HD. Then print a final version back to tape, and VOILA.....HD MASTER baby.

With a workflow like that, who needs to cut native M2T files or intermediate files or Mpeg files? For what??

- ShannonRawls.com

I hate this rumor mongering about the FX1's capabilities regarding to timecode. I'll try to give you the facts as far as I know them. I gathered these facts from various pdf files and my own investigations.

HDV uses the same tape structure as DV. See page 3 of this PDF: http://www.advance-pro.com/downloads/whatsnew/HVR-M10U.pdf There are three different sectors: a subcode sector, the main MPEG 2 sector and the ITI sector. The subcode sector contains timecode. The ITI sector contains additional track info (indexes etc.). The MPEG 2 sector contains the video and audio multiplexed in the MPEG 2 PES format.

This means that the FX1 keeps timecode on tape and does it exactly as a DV cam does (same subcode sector). This was a wise decision: the camera doesn't have to decode the mpeg 2 stream to gather timecode. It can quickly locate the timecode in the subcode sector and display it, for instance while fast forwarding and rewinding without actually playing.

The problem start when transfering footage to the computer via firewire. Current Apple software only seems to recieve the mpeg 2 sector and not the subcode sector (which we need for the timecode). This is because the camera adhers to FCC DTV standard for transmission of Digital Video over firewire (or any other medium). This standard calls for a stream of maximum 25 Mbit/s in the MPEG 2 transport stream format. The reason they did this is to have instant compability with existing DVHS and DTV devices. You can plug an FX1 into a DTV compatible device (like a DVHS or DTV capable TV) via firewire and the footage will play back. These devices will not work when there's a strange kind of subcode sector intermingled with the main MPEG 2 transport stream.

This leaves us without the subcode sector. It might be possible there is some kind of trick to get the camera to send this sector in addition to the MPEG 2 transport stream, but it's not necessary. Last week I found out that the timecode stamps can be found in the MPEG 2 transport stream too (see the previous thread called "FX1 timecode"). I just don't have enough data to predict how embedded timestamps work with free run time code, but the numbers are stable on my FX1.

This means that any program can support the FX1 with timecode, and batch capture is theoretically possible too. I suspect that the different vendors have a few problems, one of which is that they expect the timecode to be out-of-band in a special packets containing the subcode sector, and they have to look for the timecode in-band now. Another problem is that you have to capture not from the stated frame time (in point) but at the start of the group of pictures containing this frame. This is not so difficult: you are bounded to hit the start of a group of frames if you start capturing 12 (1080i50) or 15 frames (1080i60) frames earlier than the frame you need. Another problem is that the compressed audio blocks don't seem to be starting on frame boundaries.

So in short, what you want is possible, you only have to wait till the software catches up.

[I took a long time to write this reply. Various other comments appeared in the meantime.

Re: Snannon Rawls: "With a workflow like that, who needs to cut native M2T files or intermediate files or Mpeg files? For what??"

Answer: M2T files is what the camera gives you. The software will have to deal with them.]

OK -- it's getting clearer. So far, the timecode _isn't_ preserved on transfer, except perhaps with MSP, so at this point you can't generate an EDL that relates to the source tapes. This is a 'coming soon' feature ... though it is a mystery to me how the software writers can think this is an extra we can wait for.

Shannon says "That timecode is THE SAME timecode that camera uses when Downconverted to DV" but that too might be just wishful thinking -- at this point the software doesn't bother to look at the timecode it just makes it up (starts at zero) as it downloads HDV. _If_ the Sony hardware is clever enough to use the same timecode on DV conversion, we should be able to confirm that right now -- timecode preservation for DV is well established ... what happens right now with current DV capture software if you capture HDV downsampled in the Z1 to DV? Do you get the source tape timecode ..?

I too want to follow Shannon's workpath proposal: Shoot HDV, download as DV with timecode, cut DV using current RT solutions, save completed DV project file, redownload HDV material in place of DV 'proxy' and output an HDV version. This is the process I though DSE described many posts ago, and the one I thought he indicated was possible today ... it is now looking like 'someday' -- but the critical confirmation can come right now if someone with a Z1 will confirm that the timecode on tape is preserved when the HDV is downsampled and transferred as DV. Not shot as DV, but downsampled and transferred.

I appreciate your suggestion DSE that I check with the software manufacturers but I want the confirmation of my fellow working professionals -- I don't want cheerleading, or vapour ware promises ... I want to know who has working solutions right now, and what they are.

Other than that, I'm easy to please!

GB

Shannon is right. It is the same timecode in downconverted DV and HDV mode. The camera works like any other DV cam when HDV-to-DV downconversion is on. Source timecode is preserved.

One workable solution right now is: turn DV downconversion on, capture timecode and audio via firewire. Connect the analogue component output to a component to HD-SDI converter, and capture this HD-SDI stream with a HD card. It has been reported to be frame accurate. Batch capture will work with this workflow.

If it is the same timecode, then as soon as HDV timecode preservation is a reality (how hard can it be?) users of working RT DV solutions will be able to use HDV>DV downsampled files as proxies for HDV, perform the edit using the DV files, redownload the HDV files as HDV files and plug those files into the project in place of the DV proxy files ...

Is this what you see as possible?

GB

<<<-- Originally posted by R Geoff Baker : If it is the same timecode, then as soon as HDV timecode preservation is a reality (how hard can it be?) users of working RT DV solutions will be able to use HDV>DV downsampled files as proxies for HDV, perform the edit using the DV files, redownload the HDV files as HDV files and plug those files into the project in place of the DV proxy files ...

Is this what you see as possible?

GB -->>>

Yes.

The difficulties with regard to HDV timecode preservation are outlined in my first long post in this thread. The biggest problem for any program will be that you can't start capturing and decoding at the exact frame you want. You might have to go back 11 to 14 frames.

I can see the problem batch capture might have with finding the in point of a non-I frame, though forcing handles should fix this, as long as the assigned timecode value is absolute it should work OK. The first DV solutions that preserved timecode worked by always starting a capture a few frames before any in-point -- the software just clipped off unwanted frames post-transfer.

So now all I'm waiting for is an HDV solution that preserves timecode on capture. Batch capture isn't necessary, though it might be handy -- as long as the captured material includes the referenced timecode it doesn't matter if the 'container' is one big clip or each referenced clip on its own.

Thanks for your confirmations.

GB

If i recall correctly, OfflineRT in FCP3 did not start its batch capture ON the frame, it started like 2-3 seconds prior and ended 2-3 seconds after. Just to be safe I beleive.

OK...so I decided to do a test. I captured some footage in HD mode and took a photo of it with a camera flash.

Let's compare the frame timecode of the Downconverted DV file, and also take a picture of the HD footage on the cameras LCD screen to verify that the flash's BIG BRIGHT frame has the same timecode in HD on the MiniDV cassette as it does in Downconverted DV modes....

http://www.beout.com/hidef/hdvtimecode&dvtimecode.jpg

I hope those results makes everyone happy.

P.S.
#1 The flash you see in frame #18 is from my little camera taking a photo of the lcd screen.
#2 Sorry about the missing frames (#19 & #21) it was a bitch trying to stop the camera dead-on the frames so I can take a snapshot. *smile*



- ShannonRawls.com
legend: HD=HDV

Excellent work. Thank you for the confirmation -- nothing like seeing it for yourself!

As you say ... let's get HDV transfer with timecode on the table. Maybe the author of the very good Scenalyzer Live can be encouraged to add HDV to ScLive's capabilities ...

GB

Shannon's suggestion of "edit in DV, recapture later in HD" sounds very promising to me. Two questions:

1. Assuming Premiere sorts out timecode based capture is there any reason why this will not work in Premiere Pro?

2. How much can you easily automate the second HD stage - I can see you should be able to get your clip sequences to be picked up but will any other stuff pass through also eg transitions, simple color filters, deinterlace, slowing bits of clips down etc? Any thoughts on this?

Andrew

1. I can't see any .... in fact you wouldn't need Premiere to sort out the timecode capture -- as long as you could get the clips onto your HD with timecode in a format that Premiere could read (I hope there is a standard) the capture could be done with other tools.

2. In my experience, everything is carried forward. A Premiere project doesn't offer special transitions or filters based on the source clip format -- if you can apply the effect to the timeline, it can be applied to whatever source occupies the timeline. In the normal course of events any preview files that were generated would be used in the final playback ... but in this case the preview files won't match the (new) project settings so will be ignored. It has been some years since this 'proxy edit' system was the norm, but there was a time when it was -- you captured low-rez proxies of your source (usually mjpeg, back in the day), performed the edit, then saved the project file and told Premiere that you wanted to replace all the media or go on-line or some such -- the full rez clips were loaded, and everything you'd instructed was now applied to the real deal.

In another thread a poster suggested title design should be done with an eye to the 'better' results you'd get from an eventual HDV version -- quite right, but the good news is the HDV version wouldn't be worse than the DV version ... so with a little experimenting and some experience, you could build title templates that might stress the DV output but would look good in the HDV version.

Maybe I'm being optimistic, but I see little room for trouble in this workflow, as long as you have the time to reload and rerender after 'completing' the DV version.

GB

You guys are talking about what Avid guys have done for eons. It's called Offline/Online editing.

People who work in DV have never really had to do this, but in higher end formats it's the norm.

As recently as 18 months ago, many people were having to do this with SD material they wanted to finish as uncompressed. Many still do.

I have two HDV projects going right now, one with 27 reels and one with 34. One of them I'm capturing as HDV with FCE so I have something to demo to people right away, the other I'm capturing as DV using the camera to downconvert. When FCP gains the ability to capture HDV w/code, swapping in the AIC material (in a media managed timeline) will be a breeze.

What I really like about this approach is that it seems to realistically reflect the available hardware / sofware and provides a route for moving forward (especially for small players) without committing to equipment and software that is likely to evolve very rapidly (and perhaps get cheaper) in the next 18 months.

Couple more questions about this approach.

1. Since the main editing is being done at SD, with only perhaps a final rerendering for a 1080 version at the end does this mean that Premiere Pro 1.5 has all that is required and for the time being Aspect HD is not really needed with this workflow (assuming the final rendering can be done while you are away from the PC so it does not need to maximise speed).

2. Say you do have a final HD rendered version to demonstrate, is there a low / medium price route for demo-ing it to customers eg hdtv off video card or high resolution PC monitor.

3. This is more OT but perhaps someone can help me - if you have Premiere Pro to create the output and the top range Nero product to write to DVD do you still need Encore DVD as well to create DVD's (I guess I am talking SD here - or can you also do HD DVD's with Nero).

Andrew

Just a note to the Mac readers who might be a bit confused on this issue. HDVxDV will retain the TC off the FX1 or Z1.

http://www.dvdxdv.com/HDVxDVsite/Tutorials/converting.to.quicktime.htm

Thanks for the pointer to this Apple utility. Curiously, I can find no mention of timecode preservation at all in the page you referenced, or in any other page at this site. Are you confirming that in your direct experience the timecode from tape is preserved on both capture & subsequent conversion to a QT file? Or did I miss the detail in the page you referenced ..?

GB

go here
HDVxDV

the TC info is on the Download/Purchase page

As Sean says look on the Download page and yes I've seen it. It was demoed at our (lafcpug) meeting the other night. What I like more about HDVxDV is its simplicity. Download the trial. You'll like it and more very cool refinements to come.

TC and batch capturing?
 

does this

means that i can freely edit captured material on my computer, and then RE-digitize/capture (batch) it in proper resolution when edit is finished? or this is just a capturing tool?

(as i can see this - on their site - it IS just a capturing tool)

to make it more clear (or complicated) my question is following:
is it possible to do the opposite to what is explained on hdvxdv's site - to convert the video from hdv to (i.e. low resolution) quicktime and then to edit in, say FCP and after finishing - to re-capture video in highest resolution needed. but not entire material, just edited of course.

as i understand the time code is keeped just for that? or for something else?


(i'm not in front of FCP now, didn't tryed demo software yet).


thanks,


filip

No batch capturing with HDVxDV...yet. Mpegstreamclip handles that but then it doesn't handle TC. All HDVxDV allows you to do is capture HDV and then convert it to any editable quicktime codec you choose.

If you want highest resolution then get a Kona Card, a Sata Raid, a AJA HD converter and go to town.

maybe i don't understand this, but -
what for is TC - for now? if it is mentioned on HDVxDV's site?

is there ANY use of TC if you cannot use it somehow?


just curious

filip

<<<-- Originally posted by Michael Horton : No batch capturing with HDVxDV...yet. -->>>

I may be wrong here. A glance at the HDVxDV web site says it allows you to choose "Batch encode Movies" as an option. Click on "Converting to Quicktime... and scroll towards the bottom.

HDVxDV preserves the TC coming out of the Z1. If you don't need TC than ignore it.

<<<-- Originally posted by Michael Horton : I may be wrong here. A glance at the HDVxDV web site says it allows you to choose "Batch encode Movies" as an option. Click on "Converting to Quicktime... and scroll towards the bottom. -->>>

Michael,

thank you for suggestion.

i just checked the site, and as I understood - this "batch encoded movies" option is NOT "normal" batch option, like in NLE systems - it's just for batching (converting)different captured HDV files to quicktime files.
if someone has i.e. 10 captured files - there is possibility to batch it all in one pass.
but NOT the oposite - to use those edited QT files and re-capture it again.

filip

and another one :)
 

<<<-- Originally posted by Michael Horton : HDVxDV preserves the TC coming out of the Z1. If you don't need TC than ignore it. -->>>

i understand this.

but i'm just curios - what for is this option needed if nobody can't use it?

at least in this very moment.

in the future - sure, as i understand - it will be VERY useful. but now? what for?

filip

p.s.

maybe i'm too curious?

Re: and another one :)
 

-->>>
but i'm just curios - what for is this option needed if nobody can't use it?

at least in this very moment. -->>>

No idea what you are saying. Why wouldnt you need the option for Time Code?

maybe it's my english...

what i'm saying is just that:

what for is TIMECODE preserved also on QT if you cannot use it?

if you can use it to make batch digitize (capturing) - then ok.

but you can't.

i CAN ignore it, but then - i'm asking why the heck is timecode here?

filip

If you are saying that because you can't re capture the HDV, then yes the TC is irrelevant here.

yes, that's right.

i do not see OTHER useful purpose for TC. but, maybe i'm wrong,


filip

I'm lost here ...

If you can capture and preserve TC to HDV, and you can capture and preserve to DV ... then you can use the same EDL for both the HDV and DV version of the same project. 'Batch capture' is convenient, but you don' t need batch capture to work an EDL -- you just need to have the clips with timecode on your HD.

Maybe I've missed the point of these last few posts, but timecode preservation is not just for batch capture -- in fact you don't need to have batch capture at all to make good use of timecode ..?

GB

<<<-- Originally posted by R Geoff Baker : I'm lost here ...

Maybe I've missed the point of these last few posts, but timecode preservation is not just for batch capture -- in fact you don't need to have batch capture at all to make good use of timecode ..?

GB -->>>

Of course not but I think there is a language problem here and perhaps my answers are a bit confusing as I struggle with the questions.

mike

Geoff, you NEED T/C for Batch Capture, but you don't need Batch Capture to need T/C. Various people have various needs, and Batch Capture or Batch Recapture is an issue everyone has a need for from time to time.
Either way, are you satisfied with the answers to your question about original T/C being delivered on the tape with the video file whether it's in HD or SD? Or did you have a specific question still?

Yes, I am generally satisfied with the answer that developed here -- to wit; timecode preservation is not generally available (yet), but some software does offer it -- one program on the Windows platform and one utility on the Apple platform.

But I monitor this thread to be sure of developments, and to be sure I know what is available. When a poster says in this thread he 'sees no other useful purpose for TC [ but for batch]' I just want to be sure that I haven't missed the point -- especially as another poster says 'because you can't re capture the HDV, then yes the TC is irrelevant here' ... I take this not to mean 'you can't re capture' but rather to mean 'you can't batch capture' ... but if I'm parsing that statement incorrectly, I'm missing something.

Cheers,
GB