Clarification Needed for a Rather Confusing Process
 

Hi Dan, Tommy, and especially anyone else out there who's really technically inclined:

I need a clarification on several points on how the Flash XDR (This may also apply to the Nano Flash) removes the redundant frames to detect the Canon 24F format. In the Flash XDR you must have the 3:2 pull down activated in order to remove the *redundant frames* from what is always sent out of the camera circuitry to tape or HD-SDI uncompressed out as an interlaced 59.94i Hz signal. OK. I get that. I also understand why you must enable 3:2 pull down so that the removal of the redundant frames can be performed on the fly in real time during the video capture process.

*What I really don't get is how come the Flash XDR indicates a speed of 23.98 fps on its screen as it's doing this ? So I need a clarification on two points. The mathematical calculation for this process is clear enough........

59.94 minus 3 divided by 2 = 23.976 ! Not 23.98 or 24 ! Now hang on-this rabbit hole goes much deeper than even this.

When the recording is finished and my CF cards are full, I then import them into Avid Media Composer to edit via my usual clip based workflow, but I run into a rather confusing situation at this point. If I setup a media composer project for 23.976 fps with my 1080 full raster HD footage from my XDR, MC will not import the XDCAM clips ! Now, if I setup another project for 24p, then MC imports the XDCAM clips from my XDR and all is well.

But this doesn't make any sense ?!?!?!?

1. After recording with 3:2 pull down removal from a Canon camcorder with the speed switch set at 24F, what is *The Final Resulting Speed of the Recorded Clips ?*

i.e. 23.976 fps, 23.98 fps, or 24p ?

2. According to the import specifications of my Avid Media Composer editing app, the Sony XDCAM HD 4:2:2 1920 x 1080 clips which are coming from my XDR are in 24p ! This is impossible ! How can the resulting clip actually be in true 24p ?

** What is the XDR doing to change this clip to that speed, because math is math and there's no way you can get 24p from the equation I listed above. No way ! It's impossible !

A) Is the XDR speeding up the clip speed after it comes in from the camera ?
B) How is the audio treated ?

i.e. Is the audio being stretched (Sped up slightly to match the higher speed of 24p) ? Please clarify if you can. I would like to understand this in my own brain.

23.98 and 23.976 are the same, except one is rounded up to two decimal points and the other to three. The actual exact rate is 24000/1001, i.e. 24000 frames per 1001 seconds.

Mark, what error message are you getting when you try to import the file into a 23.976 project?

If you have a small nano file you want me to try to import in my MC install, I'll give it a shot and see if I see the same 24P issue.


-Peter

Doesn't Make Sense (???)
 

...Hi Adam: How can 23.976 fps be the same thing as 23.98 fps when it must be "rounded *up two decimal points* and the other three," as you expressed it ?

Avid Import Error Message
 

....Hi Peter: I'll shoot something small and email it over to you. I'm really busy today, so it may not be until this weekend that I get one ready for you to try. Thanks for your offer to test this in your Avid Media Composer. The thing is this could also be an MC bug, because MC in 4.0.5.x release is supposed to allow multiple frame rate and resolutions within the same project *anyway,* but this feature doesn't seem to be working right.

The rounding happens when some software (or hardware) is telling you what rate the clip is using. The actual rate is 24000 / 1001, so it is neither 23.98 nor 23.976. 24000 / 1001 is the only way to say the precise rate because if you divide 24000 by 1001 you will get a periodic number which has an infinite number of decimal points (23.9760976097609760... with the "9760" being repeated forever).

Hence the user interface of software and hardware has to round the number (or show you 2400 / 1001, but most do not as it would confuse many people). And some round it to 2 decimal places, some to 3 decimal places. But only in what it shows you. Internally they use the precise 2400 / 1001 in either case.

Mark, it's basic rounding of numbers. 23.976 rounded to two decimal places is 23.98. And both would be rounded to 24.0 if carrying to 1 decimal place. It's just a way of representing the number with less precision.

Garret

test with FCP
 

Agreed... and save screen space on NF/XDR.
Mark, why don't you use CD converting tool and try one of this files on your FCP and see if it shows 24fps or 23.976fps. or send me a small file via FTP and I'll try it if you don't have a FCP. FCP always shows the exact frame rate.
Also, please check what pulldown your MC is set to show in your settings. Not that it matters but see if it is 2:3:2:3 or it is 2:3:3:2. You could capture the tape with MC set for one of the above pulldowns and audio rate at 30fps and see after that what your MC would show for that file - 23.976fps or 24fps.
Let me know please if I could help you with FCP
Luben

Something is Missing Here (??????????)
 

Hi Adam, Garrett, and Luben: OK. I get the numbers rounding. I follow your logic wit this, but the rounding issue is not what's got me seriously stumped here. To me, and please do correct me if I'm wrong about this, but how can there be anything other but one mathematical certainty to the following math equation ?

Ergo: The process by which we are extracting the Canon 24 F (Not P) from my XL H1 HD-SDi output OR HDV recorded cassette, would be 59.94i minus 3 divided by 2 (3:2 pull down removal) = 23.976. (Not 23.98, nor 24.1001) It is absolutely mathematically impossible to arrive at any other frame rate other than 23.976 fps which is exactly 50 % of 59.94 Hz..

So one of my questions was how does the Flash XDR or Nano Flash for that matter, come up with a true 24p, because this is what I understood from Dan of what you get at the end of the day on your recorded CF cards.

The secondary question was about importing my XDR clips into AMC and AMC has a project setting for clips recorded at 23.976 (But not for clips recorded at 23.98 (Rounded) Nor is their a project for 24.1001), but there is also a project for 24p - which works perfectly.

Mark, it is not 24.1001, it is 24000 / 1001, or twenty-four thousand frames per one thousand and one second.

The same with 59.94. It is actually 60000 / 1001 (sixty thousand fields, i.e. thirty thousand frames, per one thousand and one second).

So, yes, the conversion is mathematically precise.

As for the pulldown, the 3:2 name is kind of mathematically incorrect (but commonly used). You actually get 4:5 of the original rate. You are converting from 60i to 24p. 60i has 60 fields, that is 30 (interlaced) frames. 30 * 4 / 5 = 24. And 30000 / 1001 * 4 / 5 = 24000 / 1001 * 4 / 5 = 23.98 (when rounded to two decimal places).

I'm Totally Confused :-(
 

Hey Adam: OK. Now I'm totally confused ! My camera does not record a *True 60i. My camera records everything @ 59.94i Now you're telling me 3:2 is not really 3:2 - It's 4:5 ?!?! Logic is little bird in tree !!!

Is My XDR Stretching the Video Speed ?
 

Hi Adam, Luben, and Garett: OK. So then this explanation would seem to imply some sort of speed stretching. (??) Is this correct ? The 23.976 is being S T R E T C H E D over to 24.00000 ?

Still confused? I guess I am not as good a teacher as I thought. :)

OK, here is a more detailed explanation.

When TV was invented, it was 60i in the US, 50i in Europe. When color was added, they slowed it down to 60000 / 1001 and 50000 / 1001 respectively.

Now, when film was being converted to TV video by telecine, they decided not to resample it but came up with the 2:3 pulldown. It works like this (and has for decades):

You start with four 24fps frames and you scan them. These are progressive frames, let's call them Ap, Bp, Cp, and Bp. You convert each progressive frame into two "interlaced" fields (I put it into quotation marks because they are not true interlaced, since they were shot progressive). Let's call them Ao, Ae, Bo, Be, Co, Ce, Do, De (o for odd, e for even).

And now to make the 48 fields per second into 60 fields per second, you send them to the video stream like this (or a variation of this):

Ao, Ae, Bo, Be, Bo, Ce, Co, De, Do, De.

It is called 2:3 pulldown because you convert one frame to 2 fields, the next one to 3 fields, then again 2 and 3 and 2 and 3, etc.

Many video cameras shoot at 24p (or 24000 / 10001) and apply the 2:3 pulldown to the stream before saving it or sending it out over HD-SDI and such.

The nanoFlash can determine whether the signal it receives was processed that way (by comparing the different fields and seeing the pattern). It can convert the above mentioned 60"i" stream into:

Ao, Ae, Bo, Be, Co, Ce, Do, De.

In other words, it throws out the redundant fields. And then it can save it as a 24p stream:

Ap, Bp, Cp, Dp.

That is all there is to it.

No time-stretching happens. If your camera shoots at 23.98p and applies the 2:3 pulldown, the nF will restore the original 23.98p and your player will play it at 23.98p. If your camera shoots at true 24p and applies the 2:3 pulldown, the nF will restore the original 24p and your player will play it at 24p if it can (only the more recent ones can, and DVD players generally cannot) or 23.98 otherwise.

The 60000 / 1001, 50000 / 1001, and 24000 / 1001 are the video standards. The exact 24p is a film standard. Video has evolved from television, hence the weird looking rates.

And yes, I wish in the days of HDTV the cameras would shoot true 24p, but most do not.

Great job Adam
 

Well done!
Mark, please look at Telecine - Wikipedia, the free encyclopedia for other way presented, basically the same info.
Sorry for not be able to reply earlier..... editing with Director....

I Now Understand
 

Hey Adam & Luben: Thanks Adam for taking the time to run through the more detailed explanation of 2:3 cadence :-) I hope you and Luben & Garrett won't be annoyed to learn I already well understood this longer explanation. What was tripping me up was thinking Canon's 24 *F* was not really a 24 *P*, when in fact **IT TRULY IS** only it is burried in a 60 Hz interlaced signal. It is not that the XDR is stretching anything, rather, it's simply ***DISCARDING*** the unnecessary cadence (Frames) to rebuild the 24 Fps which my Canon camera was shooting in the first place.

** I guess Canon would have to come up with their Funky 24 F format because...

A) The built in Helical Scanning HDV VTR cannot record in a 24 fps progressive manner.
B) Canon was too cheap to pay the guy who owns the patent on 24p the royalties ;-)

No, I am not annoyed. I am glad you got it all straightened out. :)

I think the main reason Canon calls it 24f is for marketing purposes. It makes it sound like film, even if it is video.

Hey Mark,

I'm having to jog my memory to when I had my Canons but Canon's 24F is actually laying down true 23.976fps. They labeled it 24F because the sensor in is actually not a progressive sensor and the way they build their progressive image is to take two images with a slight shift (this is where my memory is fuzzy, I don't believe there is a temporal shift in Canons flavor of progressive HDV) and creates the progressive image. So on the tape or on a solid state unit via the firewire, there is no pull down.

The issue is taking the feed from the SDI or HD-SDI out. In order to make the signal coming out of there compatible with all monitors, most SDI signals are 60i. That means in order to feed a 24p signal you of the SDI you need to introduce pull down and the frame is acutally broken into two identical half frames (each frame has every other line). So the Flash XDR or NF will actually be reading a 60i signal unless you can have the camera output 24PsF which the XDR and NF will then understand and create a 24fps file with no pull down.

As far as why Canon called their progressive form F instead of p isn't because they didn't want to pay any royalties. And, in fact as I said, the tape actually is recording a progressive image. It was to differentiate them from the other makers way of creating a progressive image from a non progressive imaging chip. The big issue when the XL H1 and XH series cameras came out was that if you recorded in Canon's 24F, only Canon cameras could play them back. And since Canon did not make a deck it became an issue for post production as people were worried that they would were their cameras down if they used them as decks in post. Then they created the HV series which for a lot of people, myself included, became their deck.

Hope that's not too long winded but I thought a little clarification was in order.

Garrett

You guys have done an AMAZING JOB explaining all this.

... But there is one issue that Mark mentioned that still sounds odd. The nano files w/ the pulldown removed should be 23.976 not true 24P. But he is saying that the files will not import into an Avid 23.976 project, but will into a 24P project.

This sounds like something is wonky.

Like I said above, I'd gladly try importing a small portion of the file to MC and see what I can come up w/.

Canon's Deep & Nebulous 24 F HDV Format
 

...Hey Garrett: No, thank you so much for that extra info. I really appreciated knowing this. I didn't realize Canon's way of doing this was way more involved than what I earlier understood. So you're saying the Canon XL H series cameras are actually recording a 23.976 fps progressive image onto the HDV tape. I bet the processing circuitry and internal processing CPU was something of an electronic work of art to make a helical scanning internal miniature VTR record a progressive image. This is complex stuff ! Frankly, I'm amazed any of this stuff works at all !

This Issue is Cooking My Noodle !
 

...Hi Peter: Yeah ! What the xxxx ! I just can't wrap my mind around this fact ! I had a face to face conversation with the Canadian National Broadcast Manager for Avid Media Design's this past Thursday Morning in Montreal, and we discussed Convergent Designs's Flash XDR and Nano Flash device at great length. I'm pushing to get some of the bugs out of the function of importing mixed frame rates and mixed resolutions (Including SD) into whatever speed or format Media Composer editing project you please. This is the way the new Media Composer is supposed to work, but it doesn't work properly.

Also, Avid's AMA feature *does not work* with the flavour of Sony XDCAM HD 4:2:2 video files that CD records in their XDR & Nano. This is no fault of Convergent Design per se, it is because the programmers have written the current Media Composer to look for Sony XDCAM HD 4:2:2 files in the certain structure extant on an XDCAM disk. If the XDCAM HD files are not in that exact layout structure, then MC returns an AMA error ! I pointed out to the Avid editors and technicians present in the demo meeting that I thought Convergent Design had to write them in the way they do, or else their recorders simply would not function correctly, so maybe it's time to broaden the file structure search parameters to include Convergent Design's current file structure layout of XDCAM HD as they exist on recorded CF card media coming out of the Nano & Flash XDR.

**Dan & Mike, are you reading this ? I'm pushing as carefully from my end as I can regarding this subject, and I do believe I was heard. Let's hope we can get both higher data rate Long GOP, as well as correct Media Composer AMA working straight from Nanos, Nexto's, and Flash XDR's without the need to import the material.

**Just think about it. What if you could simply plug in your Nano to a laptop via the FireWire and have Avid simply make *Pointers (AMA) to the card sitting in a Nano recorder itself, or sitting in a Nexto and read that in bins and drag that onto the editing time line and play and edit it in realtime **without* having to clip import all this CF card material ? Wow ! But I digress.

Adam, your explanation of the video cadence intricacies has been one of the best I heard/saw.

However, one thing comes as a surprise to me: I always thought that we Europeans can afford to know nothing about it, as the Area 50 (PAL) always uses (and actually IS) round integer numbers (25 fps, 25p, 50i). But you made me realize that it's not so for the very same reasons as with NTSC (accommodating the color info) - hence 50i is really 50000/1001...

If so, why all NLE uses integers (for both 50 and 25 fps)?

Mark, send me a small nano file and I'll see what I can make out of it. There are a lot of moving parts to all this and w/o a file, I can't shed much light on it.

They are probably just rounding it up.