Confused about 24p
 

Note from Admin: I've changed the title of this thread from "HV20 does not record 24p" to the much more appropriate "Confused about 24p." --CH

I dont understand why I seem to be the only person on the planet to notice how the alledged 24p signal recorded by the HV20 can not be properly 'pulled up' from tape, no matter what process you use. I am a pro videographer of 27 years and pro visual FX artists of 12 years. Ive read countless forums about the topic and yes, I understand exactly how the post pullup is suppose to work. But the HV20 is most certainly falsely advertised as being 24progressive. First, the HV20 camcorder does not record 24p, it records only 30 frame interlaced with 2:3 pulldown. This is not progressive by any definition. But the bigger issue is that the 24p can not be properly pulled up from the tape, whether performed during or after capture.

Example- http://www.box.net/shared/static/nfs5rly5x7.jpg

Just as everyone else, I would gladly overlook the 2:3 pulldown recorded by the HV20, except that after executing the required 2:3 pullup, my wife (also a professional videographer and visual FX artist) and I immediately noticed 'strobey' playback. Looking closely, I see that every 4th frame has residual interlacing. ... how is this 'progressive?' A signal is either progressive or interlaced. The HV20 records interlaced, and the interlacing can not be entirely removed with a pullup in any software you care to name. It is bizarre to us that no one else seems to notice the problem.

Cineform CTO, David Newman explained the problem very clearly-

"CineForm Aspect HD does support automatic extraction of 24p from HV20's 60i stream. However, Ridlen is correct that the 60i encoding of 24p is not completely reversible, whereas the in 24F in the XH-A1 and XL-H1 is it. The reason the lack of completely reversibility is the 4:2:0 60i encoding which has only chroma value for pixels over two scan lines -- these scan lines may contain data from different frames, cause chroma cross-talk. The luminance is perfectly correct. Luma is encoded at 60i, and chroma at 30p, you can extract 24p from luma but not fully from chroma. If the pulldown is extracted correctly, the chroma cross-talk will only appear on every fourth frame. As our eyes are far more sensitive to luma, it is very hard to see this in motion. The only time this can be an issue is when keying, for that I recommend using HDMI and shooting live via the Black Magic Intensity card (using the new 1.5 drivers.)"

Why is no one else seeing this? I see no mention of it anywhere on this forum.

Heres a brief (10mb MPG2) example clip recorded in 24p with the HV20-
http://www.box.net/shared/static/grc3vdk59d.mpeg

Here are examples of every fourth frame after pullup in Digital Fusion.
http://www.box.net/shared/static/qg6v6mc7re.tga
http://www.box.net/shared/static/ym1bt6d2ud.tga

Fields remain in every 4th frame. This is NOT 'P' as in 'progressive.'

The only way I could completely get rid of the 4th frame interlacing after pullup was to run a 'strip and interpolate field 1,' then a 'strip and interpolate field 2' in Fusion. But then you quite noticeably lose vertical resolution from two passes of vertical interpolation.
http://www.box.net/shared/static/rgvfqtzu9r.jpg

I only recently got the camera, but I don't doubt this issue at all ... though in the tiny amount of footage I've shot (plus that of others that I've reviewed) this anomaly doesn't appear to be glaring enough to be that noticably on typical footage.

I'm not sure that this invalidates the fact that it does capture a progressive image and encode it into a 60i stream on tape. It may not do it exactly correctly, and I wish it did, but in perspective it doesn't seem like a huge issue for a $1k camera.

I stupidly shot, before fully testing all aspects of the camera, ten hours of footage in 24p for a doc and . . . . I couldn't be more upset about the state of affairs. I've emailed canon and they've been polite but basically they just keep telling me to buy the A1 if I want true 24p. I was planning on buying cineform but, in layman's terms, are you saying that even with cineform the HV20's 24p is not really 24p and the footage will be sub par?

if you bypass the tape recording and record straight to hard drive you'd get 24p though

The HV20's 24p mode is awesome. Unfortunately, due to user errors, not everyone's footage turns out correctly from this camera.

Don't bash a camera that's already been tested and proven to work as advertised. Maybe consider yourself as the culprit and ask for assistance in how to process the footage correctly. You also may want to consider removing your "review" on CNet, as it too is completely inaccurate.

-Pasty

>if you bypass the tape recording and record straight to hard drive you'd get 24p though

Is this true? Do you need extra hardware or through plain firewire this would work? Which apps can record and in which format?

Regarding 24f, it is known that the HV20 is not 24p when it writes on tape. However, using several methods of removing pulldown, I got a very good result. Apparently Premiere LE does NOT do it the right way btw resulting in these artifacts David is showing in his pic.

I am not sure what the issues are technically, but I've been shooting HV20 in 24p, capturing with NeoHDV from Cineform. In Vegas, I have sent the footage to HD 24p and if it ain't real, it still looks pretty good....

While everything David Ridlen is saying is true, I don't see much cause for alarm. We at a CineForm recently extracted 24p from tape based captures using a HV20 and displayed to results via a Sony 4K projector at a local Landmark Theater -- looked pretty good. Yes direct capture from the HDMI (not Firewire) is better, both for pulldown to 24p, avoiding MPEG compression artifacts, and the evil 4:2:0 subsampling. You see the last two issues well before you see the 24p chroma crosstalk. Live capture from HDMI port is "perfect", for that you need a Blackmagic Intensity card and preferrably CineForm's NEO HDV or HD. Note: The Sony V1U has the same minor 24p issues if you record to tape. As for using Canon A1 24F instead of the HV20 in 24p, yes the A1 is a better camera, however I find the progressive look of the HV20 is more natural -- the Canon 24F mode lacks vertical resolution and introduces aliasing.

David, I'm sorry for asking such a basic question, but my head is spinning from trying to keep so many plates up in the air.

Does Cineform NEO HD deinterlace 60i (I'll probably be using a combination of a Sony FX-1 and a Canon HV-20) to 24 progressive frames per second? And if so, how does it handle motion and how much vertical resolution is lost?

I understand that Apple's Compressor 3 is quite good at this, but I'm in the pc world, and of course would rather give CineForm the business.

Thanks MUCH!

I shot quick tests of a black glove waving in front of white, red, green and blue backgrounds. As expected, the chroma/field bleed is most apparent in the red channel, green next, a little less in the blues, while the white background shows no interlacing problem since the luma values are not affected. And if you post process the HV20's neutral 'cinemode' footage by pushing saturation or black levels, as anyone might quite naturally do, the problem quickly looks much worse. I really dont know how anyone can think that this is okay to have playing back on every fourth frame.
red
http://www.box.net/shared/static/zmvek4rvan.jpg
green
http://www.box.net/shared/static/2vqcguo1oc.jpg
blue
http://www.box.net/shared/static/uhdlh6rtky.jpg


Yes, but again I ask, who expects a 24p camcorder to neither record nor playback 24p? How many HV20 owners fully anticipated always having to shoot while hooked to a computer in a studio situation to obtain clean 24p? I sure did not. I also anticipated being able to key elements I shot with the HV20, which is an even a bigger mess with the fourth frame interlacing.

Wikipedia defines a camcorder as "... a portable electronic device for recording video images and audio onto an internal storage device." Therefore, a '24p camcorder' should record '24p' on an 'internal storage device.' The Hv20 does not record 24p, so it is not a 24p camcorder. And the 24p can not be properly converted from the tape due to this fourth frame residual interlacing problem.

I dont see how anyone can find the 24p aspect of the HV20 anything but dissapointing. I also fail to see how Canon's '24p' claim of the HV20 camcorder is not at the very least, misleading advertising.

It's a $1k camcorder for soccer mums....
GET OVER IT

Be grateful Canon is testing their new 24p-capable CMOS chips on a consumer cam. They know full-well a lot of pros and indie people are using the HV20 and pushing it to its limits - the feedback they give to Canon will hopefully go into their next generation 3-CMOS pro camcorder without the headaches if they had released a "pro" HV20 simulatenously with the HV20.

Once again.... the HV20 is a TOY...... a very nifty, very cool toy, but a toy all the same.
GET OVER IT

Life is so much easier with PAL and 25p could you consider importing a PAL unit?

I suspect the problem being described is nothing to do with capture but is due to the way the image data is being compressed.
Image compression algorithms are typically designed to exploit pixel-to-pixel coherence in the image. Consequently smooth images compress better than un-smooth ones.

The way the (NTSC) HV20 is laying down frames means that every other frame is a "mouse toothed" interleaved frame. These frames will be a tougher to compress and compression errors will cause some crosstalk between fields.

On the other hand, the PAL version doesn't need to do this.

Why doesn't the Camcorder industry just drop this regional / historical nonsense?

The 24P mode looks pretty good to me. I have no complaints on that issue...

It sounds more like, can Mr Ridlen come up with a better solution for $1000 ??

Please let us know...

Thanks

There's a thread over on DVXUSer about this that pretty much contradicts everything in the orginal post. Barry Green, in particular says that it is true 24p.
http://www.dvxuser.com/V6/showthread.php?t=104714

The camera encodes the 24p sequence into a 60i data stream ... an incredibly common practice. Are you trying to claim that any method that uses reverse telecine is not 24p?

I undertstand your frustration that there are some undesirable artifacts, but the manner in which you are presenting your argument makes it difficult to see past your "sky-is-falling" tone.

P.S. I've seen some very clean keys from the HV20 when a good chroma smoothing filter was used first (Nattress, etc.)

Besides the flags....It's no different from the DVX in how it implements 24P.... so what's the problem. How is it not real????.....then you are saying the DVX is a fake. Also the Hollywood movies you watch on DVD must also be fake.

I'll try to make this as clear as possible. 60i is not 24p. 'i' is for interlacing, 'p' is for progressive, meaning no interlacing. 'Progressive' and 'interlacing' do not exist simultaneously within the same signal. The HV20 either records with 2:3 interlacing, which is in no way 24 progressive, or it records 24 progressive, which is in no way interlaced. It does not somehow simultaneously record both progressive and interlaced contained in one magic signal. Sort of like how a specific volume of water may exist as liquid, solid ice, or a gas and may be converted from one to the other. But the water does not exist as all three simultaneously. The same applies to how dvds are burned. But there seems to be an odd interpretation that there is a 24p signal somehow simultaneously mixed in along side the interlaced signal (?). The signal exists only as one or the other at any given moment. Canon support, the HV20 spec sheet, Cineform and elsewhere in this forum clearly explain how the HV20 records only with interlacing, not progressively. The Hv20 does not record 24 progressive, and it does not playback 24 progressive. It only records and plays back with interlacing.

Curiously, what Canon's site and pro reviewers fail to mention is that you will have to own or buy a separate, third party piece of hardware/software to interpret the 24p out of the HV20 since it does not independently record or playback 24p on it's own. I equate this with an 'HD ready' TV which is capable of dealing with HD television, but you have to purchase a separate HD tuner to interpret the HD signal. I consider the HV20 '24p ready' but not 24p on its own since it requires more parts to deliver 24p.

Most of us are prepared to overlook this situation given the relatively simple conversion process provided by Cineform, Black Magic and the like. But everything I capture, and convert to 24p is playing back strobey because of another unmentioned problem with every 4th frame hanging onto the interlacing. What frustrates me is that Canon support entirely denies the problem. They blame Cineform, or whatever app you use to convert to 24p as 'doing it wrong.' We all know that is not the case. The problem is in the HV20s recorded signal. And if every 4th frame still contains interlacing after proper 24p extraction, then it is not true progressive. The HV20 records interlaced, it plays interlaced, and it remains partially interlaced after the 24p conversion that everyone says is suppose to do the trick. So as a portable camcorder (for me, the whole reason for buying it), the HV20 is never true 24p.

You may only get 24p while capturing live, hooked to a computer with an HDMI capture card. Where is that explained by Canon?

You say, 'You get what you pay for?' Always with video gear. But that doesnt make the claims of 24p any less misleading by Canon.

Ian,

The issue 4:2:0 encoding, DVX100 using 4:1:1 encoding which doesn't mix chroma encoding between scanlines.

Peter,

>Does Cineform NEO HD deinterlace 60i (I'll probably be using a combination of a Sony FX-1 and a Canon HV-20) to 24 progressive frames per second? And if so, how does it handle motion and how much vertical resolution is lost?

Yes you can deinterlace 60i material to make it 24p using NEO HD. When you removal pulldown form 24p sources like the HV20, no vertical resolution is lost. When you deinterlace 60i source, the resolution drops about 30% vertically, not in half like some dinterlacers. So you should get a reasonable mix between the FX1 and the HV20.

David Ridlen,

You are complaining as much as you are because you are removing the pulldown incorrectly -- and that will look bad (not a camera fault.) The lumanence is completely separatable, yet you are showing images with luma crosstalk. You should not see any interlacing artifacts. The problem is only in chroma and that is much more subtle. So while you have been quoting me, you have miss-understood the issue. Yes there is a issue, but not what you are showing. Please send me a link to those M2T files and I will show you how they look when the pulldown is removed correctly.

David Ridlen,

Dude, brother, the more you try to "make it clear" the more you dance around a dead horse, and obfusicate the real issue.

The point is, the camera has a progressive sensor, it "captures" a true progressive frame, it just happens to encode those frames in a 60i stream in order to record them to tape.

Yes, yes, yes, you can encode a progressive frame inside of a sequence of interlaced fields, and then extract those frames back out.

Hell, one can encode a picture into an audio signal, or encode a recipe into a still photo ... IT'S DATA!

Just because current software makes it difficult to do, doesn't mean it can't be done. This morning, I used both Compressor and JES Deinterlacer (free) to properly deinterlace. Sure I wish Canon had included flags and I didn't have to take an extra step, sure I hope FCP and other editors get smarter about injesting unflagged footage, but to imply that we're all suckers to Canon's misleading marketing is off mark.

Canon can make the claim that its progressive because it has a progressive sensor. It only lays it down on tape (like the DVX) with pulldown added. Since you used your own analogy I'll try my own. To me what you are stating is like saying, even though Hollywood movies are known to be shot at 24fps but telecined when put on DVD, that it's not real 24p. That does not make any sense to me. Canon can also make the claim that the cam shoots at 1920x1080 even though it lays it down to tape as 1440x1080. All they are doing is meeting industry standards when they implement these steps.

Do us a favor and show us an example of your intelaced fourth frame shot. (EDIT: Ok I took a look at your pictures on the first page...seems like you must be doing something wrong) I have stepped through, frame by frame, many of my reverse telecined footage and I witness no interlaced frames. The extra frames that were there at first (which caused the ghosting effect) were all gone. So I am confused by your statements.

This statement you make here is very funny: You are contradicting your own previous statement. It might be true that it needs more parts to "deliver" the 24P footage but it's there...even in your own previous admission.

It's one thing to be upset about the 24p being wrapped in a 60i timeline but to say that it's not 24p is too much of a stretch and misleading to me.

I was really addressing the other David's original statement at the top of this thread about it not being real 24p. I think his issues are really with the HDV standard and how it's recorded to tape. I guess if we were talking about HDMI capture then the croma bleeding issue would be a different story (as you suggested earlier).

I don't plan on making this lengthy, because I have better things to do, but there is nothing that uses magnetic tape as its medium that embeds a progressive image. IT IS IMPOSSIBLE due to the helical scan head. Look THAT on Wiki. The sensor, where it matters, is progressive. It processes in progressive, but records to tape in interlaced, THE ONLY WAY IT CAN! I love the image it produces and have not seen this problem when I reverse telecine. Even your images zoomed in and paused at one particular frame is hard to see, especially in the green and blue examples. All my distribution will be on DVD and viewed interlaced anyway. See Ya.

Could the strobing David's talking about be exacerbated by not shooting in 1/48 or 1/24. Even though I just downloaded a trial version of Neo I'm getting some strobing and I'm worried it's because I did not set the camera up properly? Ie I was using a higher shutter speed. Or is there something I'm missing in regards to the pulldown with neo?

Getting the interlacing artifact on every x frame is not because of the shutter speed, but rather that the inverse telecine process is not detecting the proper "cadence" that the 24P frames have been encoded into. With HV20 clips, you never really know where the sequence starts unless you actually analyze the footage to determine the duplicates. My first attempt at de-interlacing was with a tool that expected the sequence of A-B frames a certain way, and it gave me the same nasty interlacing. Then I switched to a tool that actually analyzed the footage and than I got the expected clean results.

If you're capturing a bunch of takes as one big stream, then you need to be double sure that your tool looks for all of the breaks and cadence changes, not just the first. I got bit by this, too. My first clip looked great, but all of the rest had interlacing. I had to instruct the tool to look at the entire stream.

Now, as far as capturing a "look" to the motion that people will subliminally associate with film, yeah, then using a 1/48 shutter speed really does make a nice difference.

Shoot 60i, Using motion bob convert to 60p



Why is everyone obsessed with that "film" look?

You have a video camera... you are shooting videos...

If you wanted to shoot a film... you would buy a film camera...

I never understood that... meh... Film isn't 1920 x 1080... ? You're trying to copy the "film" look... I guess I'll never understand... haha...

1) You'll get more spacial resolution than by deinterlacing a 60i capture. (Of course, you lose a lot of temporal resolution.)

2) Editing in 24P uses less resources.

3) But most importantly, the average consumer associates the 24p "feel" with "quality" narrative work. HD can't undo 100 years of ingrained perception overnight. 24p/48th sec. motion rendering just feels "cinematic."

I only have a "video" camera because I can't afford to buy a Panaflex rig and pay for the processing of miles of 35mm film. ;-)

I doubt the average consumer could tell the difference between 24p and 30p...

60i to 30p is simple...

Ha..I think most of us would shoot film if it was affordable. I know you can't see the obsession....neither did I until I got more and more into these forums. But Joe....you shoot some amazing videos with your HV10 (and FX7 now???). If I were a major film studio interested in your stuff to put on the big screen for some sort of say.....paintball documentary....then.....I bet you would become "obsessed" with what's being talked about here also. Maybe most of us have delusions of grandeur hoping one day our stuff will end up on the big screen....(ok maybe its just me) but it's great to know there are tools out there that can help us along the way.

Plus there are entities in Hollywood doing the same thing (with more expensive cams maybe but....think Michael Mann) and I think think it's only going to keep growing and going in that direction for a long time to come. Digital cams are getting better and better these days.

A 60i capture, de-interlaced to 30p has the exact same issue: The image changes between fields, you can't reconstruct one solid frame without sacrificing some spacial resolution. (Inconsequential if the subject hasn't moved much, as much as half the resolution if it is fast moving.)

Now, if you captured 30p, yes, it has most of the same feel as 24p. The key is a progressive sensor. Interlacing is an NTSC hack whose time is up.

David, I imagine shooting the HV20 at 24p and removing the pulldown will yield a better picture than shooting the HV20 at 60i and deinterlcaing to 24p.

Do you feel that is correct?

Thanks!

Yes, many times better to shoot 24p than 60i.

The author of this thread clearly does not understand the industry-standard practise of embedding progressive material in an interlaced stream by using Progressive Segmented Frames (PsF).

The HV20 records true 24p material from the camera head into a 60i stream by implementing the 60 year old technique of 2:3 pulldown and embedding PsF in the 60i stream. This is the EXACT same recording process as used in the HIGHEST-end HDCAMSR cameras and no-one but the author of this thread would try to make the futile argument that those camera are not 24p either.
The author of this thread loses all credibility by the aggressive and ignorant nature of his approach, and his inability to comprehend basic concepts of decades-old telecine methods is simply mind boggling.

The HV20 contains an ingenious and economical true 24p camera head coupled with a robust and industry standard telecine engine to deliver the 24p material in a 60i HDV stream. Users who want true 24p performance can extract full HD 24p via HDMI from the HV20, or they can fork out 3 times the price for the A1.

Not trying to go off topic (much) but this statement made me think. I don't own an LCD TV yet but with the airwaves going totally digital in a couple of years here it makes me wonder if the industry standards over this neck of the woods will change. There really does not seem to be a need for interlaced footage even now. I don't kow if it has anything to do with going fully digital but i ask because that will certainly affect the way manufacturers make their cams.

Joseph, I was wondering if you could elaborate on this more technically. I guess what I'm asking is how is most HD content being captured these days? What kind of gear and frame rates? Is most of it it all done fully digital?

I love the look of HD compared to SD and also compared to most movies - but that's because I really like seeing the sharp details you see in the footage. And not all HD shows being broadcast have the sharp image details, some widescreen HD broadcasts appear as though they are better than SD but far less than the HD you see in other shows. Even within the same shows you can see when they make use of SD footage (most often a clip from the recent past).

To me, HD initially seems like a medium for documentaries - but as you've rightly said, the public has a "specific image perception" ingrained and that's why HD captures people visual attention, because IT LOOKS NEW. But many years from now it will all look normal and typical.

Nathan,
Not only does HD offer a big jump in spacial resolution from standard definition, but it offers a big jump in *temporal* resolution from film. 60 FPS is a lot more visual information per second than 24, so it's much smoother and more lifelike. For content like nature documentaries, that temporal resolution really helps sell the image as "real." But for some uses, it's too real.

Huh? Explain?

Well, film, which has traditionally been a more narrative medium ... storytelling ... has always been at 24 FPS. That stacato motion is one of several cues (motion blur, controlled depth of field, etc.) from film that tell our brains "sit back, enjoy, this is fine storytelling, this isn't real, this is something different."

That's why you'll find HD dramas and comedies, etc. on network TV shooting at 24p, but HD sports and a specials about sharks shooting at 60i/p.

And how about with the Sony HDR-FX1? Would recommend shooting in CineFrame (Sony's verson of 24p simulation) and running the footage through Cinefom? Thanks A LOT!

I that is less obvious, we do remove pulldown for Sony CineFrame 24, but you get more resolution using 60i and converting that to 24p.

Try it out and see which mode you prefer -- all our software has a 15-day trial.

Mr. Newman, using neo, is my HV20 footage going to have more resolution if I shoot in 60i and then convert to 24p? than if I shot in 24p and then did the pulldown and kept it at 24P?

The reverse is true. Shoot 24p.