TMPGEnc 24p pulldown removal for AVCHD at DVinfo.net

Artem Lipatov · February 4th, 2008, 11:50 AM

Hi there!

I saw a thread somewhere else on how to properly remove pulldown and get a true 24p stream for Canon HV20 HDV clips.

I am wondering, before I spend my money on TMPGEnc (which is a really nice tool anyway), will the process work on AVCHD files or I am missing something very important and this is not possible?

Here is my setup:

- CoreAVC Professional + Haali Media Splitter:

CoreAVC settings: Deinterlace - None, Agressive Deinterlacing - Off, Preferred Decoder - On

- TMPGEnc

Import the clip
Filters: Deinterlace - "24p Special Animation", "Inverse Pulldown"

Format: MPEG2 HDV set to 23.976 Progressive

I tried it on my sample clips and I don't really see anything that will tell me exactly whether this is working properly or not...

Thanks!
Art

Eugenia Loli-Queru · February 4th, 2008, 03:23 PM

HDV is lossy. By exporting in HDV you lose quality. It might be an easy solution for you, but for me that would be unacceptable. Especially because AVCHD is still not as good as pure HDV in quality. Quality would be like a 3x compressed HDV stream.

I would just pay an extra $100 and get the NeoHDV utility. Works with AVCHD, and saves as lossless.

Artem Lipatov · February 4th, 2008, 03:37 PM

Well, Exporting settings into HDV was just to say it was set to 23.976, I could export lossless, this is not the point.

Eugenia, was that your blog I got the information from - http://eugenia.gnomefiles.org/2007/0...-24p-pulldown/ ??

I'm just curious that you (if that is you) list TMPGEnc as a solution for HV20 and not for HG10.

Theoretically speaking, I do not see the difference, as long as the software can correctly extract 24p video from 29.97 stream, but I am really just a newbie here... What if AVCHD compression impacts ones ability to even extract it?

I tried the "freeware" approach - it is severely cumbersome and almost always failed somewhere - one of the lossless codecs would generate an error on encoding and kill my Explorer process... MEncoder is not a stable piece of software nor it is very transparent...

As for compression, I'm fine with AVCHD, I am not a professional and/or a purist and judge from what I see and I see very good video quality for my home videos even if re-encoded to another lossy format...

Thank you very much!
Art

Artem Lipatov · February 4th, 2008, 03:47 PM

I just though of something -

Is the 24p-in-29.97i encoded in-camera AFTER the pulldown or before?

If it is BEFORE - Capture and encode 24p, then perform pulldown to fit the progressive *encoded* frames into 29.97i stream - then it should really just allow re-assembling the stream (but I'm guessing this is not the case)

AFTER - Capture 24p -> Perform pulldown -> Encode - regardless of which initial format you used - whether it is HDV or AVCHD - you are loosing quality on pulldown removal because the removal will work like this:

Decode -> Get Frame Data -> Deinterlace/Remove Pulldown -> Encode

So, the tutorial for TMPGEnc here http://hv20.com/showthread.php?t=1417 still will produce lower quality than the source?

Sorry if I am completely off-course and lost in this whole topic...

I guess 24p is not for me right now....

Thanks!
Art

Aaron Courtney · February 4th, 2008, 04:16 PM

Quote:

Originally Posted by Artem Lipatov

...regardless of which initial format you used - whether it is HDV or AVCHD - you are loosing quality on pulldown removal because the removal will work like this:

Decode -> Get Frame Data -> Deinterlace/Remove Pulldown -> Encode

Thus, the whole reason to use a visually lossless intermediate like Cineform. And AVCHD does not prevent one from correctly extracting the 24p frames. My personal experience and that of several others on this site have proven that point several times over. That's why you use your AVC decoder, which you already have.

So I'd say you're all set as soon as you decide on a pulldown removal/intermediate utility. FYI, I'm using Cineform's NEO HDV and it works as advertised.

Artem Lipatov · February 4th, 2008, 04:21 PM

Quote:

Originally Posted by Aaron Courtney

Thus, the whole reason to use a visually lossless intermediate like Cineform. And AVCHD does not prevent one from correctly extracting the 24p frames. My personal experience and that of several others on this site have proven that point several times over. That's why you use your AVC decoder, which you already have.

So I'd say you're all set as soon as you decide on a pulldown removal/intermediate utility. FYI, I'm using Cineform's NEO HDV and it works as advertised.

Exactly what I needed to hear! :) thank you

Eugenia Loli-Queru · February 4th, 2008, 04:33 PM

>I'm just curious that you (if that is you) list TMPGEnc as a solution for HV20 and not for HG10.

Because I have never tried TMPGEnc with AVCHD footage. I don't even know if it supports AVCHD source footage.

I only include TMPGEnc on my HV20 tutorial because people use it. But personally, I don't like the solution, because these people usually export back to mpeg2, which is lossy. Cineform is a much superior solution in my opinion.

Artem Lipatov · February 4th, 2008, 04:47 PM

Quote:

Originally Posted by Eugenia Loli-Queru

>I'm just curious that you (if that is you) list TMPGEnc as a solution for HV20 and not for HG10.

Because I have never tried TMPGEnc with AVCHD footage. I don't even know if it supports AVCHD source footage.

I only include TMPGEnc on my HV20 tutorial because people use it. But personally, I don't like the solution, because these people usually export back to mpeg2, which is lossy. Cineform is a much superior solution in my opinion.

It does support AVCHD footage as long as your system is capable of decoding it - hence the CoreAVC with a splitter

And, you can export to anything you want including any lossless codec as long as you have it installed.

Basically, I guess I just did not know what to look for in my footage - I performed the pulldown removal and it still looked kind of choppy to me, but this might have been my camera movement or my computer slowness.

When I tried looking at individual frames to see if I can find ghosting - I could not tell the difference between 2 encoded files - one with pulldown and one without... Maybe I just don't know what to look for...

Peter Moretti · February 13th, 2008, 12:11 AM

Quote:

Originally Posted by Eugenia Loli-Queru

>I'm just curious that you (if that is you) list TMPGEnc as a solution for HV20 and not for HG10.

Because I have never tried TMPGEnc with AVCHD footage. I don't even know if it supports AVCHD source footage.

I only include TMPGEnc on my HV20 tutorial because people use it. But personally, I don't like the solution, because these people usually export back to mpeg2, which is lossy. Cineform is a much superior solution in my opinion.

That people choose m2t is not really the program's fault. They are free to choose lossless formats (like Lagarith).

Artem Lipatov · February 13th, 2008, 08:45 AM

Concerning Lagarith and such -

Are people so concerned with preserving the quality up to a point when it does not matter anymore?

If you're shooting professional footage at, say 50Mbps then yes, transcoding into HDV is not wise.

But frankly, I inspected a clip frame by frame in both the original AVCHD 15Mbps and transcoded into HDV Mpeg2 25Mbps and I see very little to no difference whatsoever.

I see a lot of people working with software that cost over a $1000 and producing intermediate files that are several gigabytes a minute... I don't have that kind of money to spend on software and storage only to be able to hypothetically preserve whatever quality in my home video.

I like to think about "good enough" solutions and quite honestly more often than not the bundled software does the trick producing DVDs and HD DivX files to send my relatives and friends

I also bought Sony Vegas to be able to do more, but so far only a $14.99 CoreAVC purchase actually being used all the time only because it yields visibly better results

(NOT A RANT, just an observation)

Aaron Courtney · February 13th, 2008, 08:57 PM

Cineform's NEO HDV is only ~$200 and (1) it removes pulldown for proper 24P workflow and (2) it totally eliminates the whole AVCHD resource hog editing problem experienced even with modern hardware.

The fact that it is a lossless intermediate was just icing on the cake for me. I didn't buy it for the "lossless" part but for the two reasons listed above. Not sure if this applies to you, but look, 24P is not going to go away even though there are those who want it to die. That frame rate is much too embedded in the film community from what little I have gathered. So what is happening is that these consumer electronic progressive display manufacturers are beginning to natively handle 24P material, fed via the high def players, by incorporating 3:3, 4:4, and 5:5 pulldown intelligence in the display to completely eliminate judder caused by 3:2 pulldown and to provide smoother 24fps motion (although many of the 120Hz displays seem to be doubling 3:2 pulldown to hit that 120Hz refresh rate - look at the Pioneer Kuro's for correct 3:3/72Hz when presented 24P material).

So I'm banking on this progression becoming mainstream, and I wanted to get into the 24P workflow right now with all of my footage. Also storage is stupid cheap right now and will continue fall in terms of $/GB. I remember when 1GB SCSI drives were >$1K, LOL.

Amit Mantri · February 27th, 2008, 06:04 AM

imho Aaron is right in the sense that consumers displays are heading in that direction.

One of the new features in panasonic's new 2008 line of plamsa displays is the ability to handle 24p material natively (mostly from 1080/24p outputting Blu-ray players).

The lower end models 80u/85u will accept 24p but do 3:2 pulldown to display at 60Hz. the higher end models 800u/850u will do 2:2 frame repeat to display it correctly without judder at 48Hz. They are not out yet so no one knows if they do that as well as the 3:3 72Hz pioneer kuro plasmas.

I have a HG10 and shoot (casual home movies) in 24p only. I don't do any editing (no interest so far). I have used WME to encode wmv files from the m2ts files and it does the IVTC correctly to extract the 24p material. clips look great.

Artem Lipatov · February 27th, 2008, 09:34 AM

Quote:

Originally Posted by Amit Mantri

I have a HG10 and shoot (casual home movies) in 24p only. I don't do any editing (no interest so far). I have used WME to encode wmv files from the m2ts files and it does the IVTC correctly to extract the 24p material. clips look great.

I tried loading .m2ts file into WME (CoreAVC is the codec) and WME crashes on encoding every time.

Can you describe your process, if it is not too much trouble? and how do you force WME to perform IVTC? As far as I know it is not automatically detectable from Canon's files

Thanks!
Art

Amit Mantri · February 27th, 2008, 02:32 PM

Sure, I have the process written down in a doc at home, i'll try to post it tonight.

Dan Lewandowski · February 27th, 2008, 04:02 PM

The complexity of options around AVCHD processing is fascinating, and Aaaron has already provided some great info for me.
But, as a beginner, could I run this plan past you all?

Ordered an SR11 for mid March delivery (based on rave reviews by Canadian users, where the camera has been available for about a week or so).
Even tho Vegas Pro is on a super upgrade deal now, I am planning to stick with the included Sony software for a bit to see if it meets my needs (maybe you already know the answer).
If AVCHD procesing with the simple software is too demanding for my computer (dual core 2.2GH, 2.2Gb), I would then get NEO HDV to convert files.
Here is where I still am not clear -
Now I would have a "lossless intermediate" file that can be used "with any NLE".
What simple, low cost NLE could I start with for fast, basic editing (simple color correction, brightness/contrast, editing clips together perhaps with simple transitions/fades, slo mo, and captioning and sound editing)? Maybe the above is beyond basic? Is Windows Movie Maker evn an option?
Eventually, for a large screen (60"x60") presentation, would I need higher powered software to prep the vids for that magnification? Again, no fancy editing , just basic stuff.
Net - for starters, would like fast, low cost, simple editing workflow and software. Your advice much valued.
Thanks,
Dan

February 4th, 2008, 11:50 AM	#1
Artem Lipatov New Boot Join Date: Feb 2008 Location: Framingham, MA Posts: 11	TMPGEnc 24p pulldown removal for AVCHD Hi there! I saw a thread somewhere else on how to properly remove pulldown and get a true 24p stream for Canon HV20 HDV clips. I am wondering, before I spend my money on TMPGEnc (which is a really nice tool anyway), will the process work on AVCHD files or I am missing something very important and this is not possible? Here is my setup: - CoreAVC Professional + Haali Media Splitter: CoreAVC settings: Deinterlace - None, Agressive Deinterlacing - Off, Preferred Decoder - On - TMPGEnc Import the clip Filters: Deinterlace - "24p Special Animation", "Inverse Pulldown" Format: MPEG2 HDV set to 23.976 Progressive I tried it on my sample clips and I don't really see anything that will tell me exactly whether this is working properly or not... Thanks! Art

February 13th, 2008, 08:57 PM	#11
Aaron Courtney Major Player Join Date: Feb 2006 Posts: 229	Valid points but... Cineform's NEO HDV is only ~$200 and (1) it removes pulldown for proper 24P workflow and (2) it totally eliminates the whole AVCHD resource hog editing problem experienced even with modern hardware. The fact that it is a lossless intermediate was just icing on the cake for me. I didn't buy it for the "lossless" part but for the two reasons listed above. Not sure if this applies to you, but look, 24P is not going to go away even though there are those who want it to die. That frame rate is much too embedded in the film community from what little I have gathered. So what is happening is that these consumer electronic progressive display manufacturers are beginning to natively handle 24P material, fed via the high def players, by incorporating 3:3, 4:4, and 5:5 pulldown intelligence in the display to completely eliminate judder caused by 3:2 pulldown and to provide smoother 24fps motion (although many of the 120Hz displays seem to be doubling 3:2 pulldown to hit that 120Hz refresh rate - look at the Pioneer Kuro's for correct 3:3/72Hz when presented 24P material). So I'm banking on this progression becoming mainstream, and I wanted to get into the 24P workflow right now with all of my footage. Also storage is stupid cheap right now and will continue fall in terms of $/GB. I remember when 1GB SCSI drives were >$1K, LOL.

February 27th, 2008, 04:02 PM	#15
Dan Lewandowski New Boot Join Date: Feb 2008 Location: Minneapolis MN Posts: 15	Advice to a rookie The complexity of options around AVCHD processing is fascinating, and Aaaron has already provided some great info for me. But, as a beginner, could I run this plan past you all? Ordered an SR11 for mid March delivery (based on rave reviews by Canadian users, where the camera has been available for about a week or so). Even tho Vegas Pro is on a super upgrade deal now, I am planning to stick with the included Sony software for a bit to see if it meets my needs (maybe you already know the answer). If AVCHD procesing with the simple software is too demanding for my computer (dual core 2.2GH, 2.2Gb), I would then get NEO HDV to convert files. Here is where I still am not clear - Now I would have a "lossless intermediate" file that can be used "with any NLE". What simple, low cost NLE could I start with for fast, basic editing (simple color correction, brightness/contrast, editing clips together perhaps with simple transitions/fades, slo mo, and captioning and sound editing)? Maybe the above is beyond basic? Is Windows Movie Maker evn an option? Eventually, for a large screen (60"x60") presentation, would I need higher powered software to prep the vids for that magnification? Again, no fancy editing , just basic stuff. Net - for starters, would like fast, low cost, simple editing workflow and software. Your advice much valued. Thanks, Dan

February 4th, 2008, 03:23 PM	#2
Eugenia Loli-Queru Major Player Join Date: Jun 2007 Location: Bay Area, CA, USA Posts: 348	HDV is lossy. By exporting in HDV you lose quality. It might be an easy solution for you, but for me that would be unacceptable. Especially because AVCHD is still not as good as pure HDV in quality. Quality would be like a 3x compressed HDV stream. I would just pay an extra $100 and get the NeoHDV utility. Works with AVCHD, and saves as lossless.

February 4th, 2008, 03:37 PM	#3
Artem Lipatov New Boot Join Date: Feb 2008 Location: Framingham, MA Posts: 11	Well, Exporting settings into HDV was just to say it was set to 23.976, I could export lossless, this is not the point. Eugenia, was that your blog I got the information from - http://eugenia.gnomefiles.org/2007/0...-24p-pulldown/ ?? I'm just curious that you (if that is you) list TMPGEnc as a solution for HV20 and not for HG10. Theoretically speaking, I do not see the difference, as long as the software can correctly extract 24p video from 29.97 stream, but I am really just a newbie here... What if AVCHD compression impacts ones ability to even extract it? I tried the "freeware" approach - it is severely cumbersome and almost always failed somewhere - one of the lossless codecs would generate an error on encoding and kill my Explorer process... MEncoder is not a stable piece of software nor it is very transparent... As for compression, I'm fine with AVCHD, I am not a professional and/or a purist and judge from what I see and I see very good video quality for my home videos even if re-encoded to another lossy format... Thank you very much! Art

February 4th, 2008, 03:47 PM	#4
Artem Lipatov New Boot Join Date: Feb 2008 Location: Framingham, MA Posts: 11	I just though of something - Is the 24p-in-29.97i encoded in-camera AFTER the pulldown or before? If it is BEFORE - Capture and encode 24p, then perform pulldown to fit the progressive encoded frames into 29.97i stream - then it should really just allow re-assembling the stream (but I'm guessing this is not the case) AFTER - Capture 24p -> Perform pulldown -> Encode - regardless of which initial format you used - whether it is HDV or AVCHD - you are loosing quality on pulldown removal because the removal will work like this: Decode -> Get Frame Data -> Deinterlace/Remove Pulldown -> Encode So, the tutorial for TMPGEnc here http://hv20.com/showthread.php?t=1417 still will produce lower quality than the source? Sorry if I am completely off-course and lost in this whole topic... I guess 24p is not for me right now.... Thanks! Art

February 4th, 2008, 04:33 PM	#7
Eugenia Loli-Queru Major Player Join Date: Jun 2007 Location: Bay Area, CA, USA Posts: 348	>I'm just curious that you (if that is you) list TMPGEnc as a solution for HV20 and not for HG10. Because I have never tried TMPGEnc with AVCHD footage. I don't even know if it supports AVCHD source footage. I only include TMPGEnc on my HV20 tutorial because people use it. But personally, I don't like the solution, because these people usually export back to mpeg2, which is lossy. Cineform is a much superior solution in my opinion.

February 13th, 2008, 08:45 AM	#10
Artem Lipatov New Boot Join Date: Feb 2008 Location: Framingham, MA Posts: 11	Concerning Lagarith and such - Are people so concerned with preserving the quality up to a point when it does not matter anymore? If you're shooting professional footage at, say 50Mbps then yes, transcoding into HDV is not wise. But frankly, I inspected a clip frame by frame in both the original AVCHD 15Mbps and transcoded into HDV Mpeg2 25Mbps and I see very little to no difference whatsoever. I see a lot of people working with software that cost over a $1000 and producing intermediate files that are several gigabytes a minute... I don't have that kind of money to spend on software and storage only to be able to hypothetically preserve whatever quality in my home video. I like to think about "good enough" solutions and quite honestly more often than not the bundled software does the trick producing DVDs and HD DivX files to send my relatives and friends I also bought Sony Vegas to be able to do more, but so far only a $14.99 CoreAVC purchase actually being used all the time only because it yields visibly better results (NOT A RANT, just an observation)

February 27th, 2008, 06:04 AM	#12
Amit Mantri New Boot Join Date: Nov 2007 Location: NYC, New York Posts: 18	imho Aaron is right in the sense that consumers displays are heading in that direction. One of the new features in panasonic's new 2008 line of plamsa displays is the ability to handle 24p material natively (mostly from 1080/24p outputting Blu-ray players). The lower end models 80u/85u will accept 24p but do 3:2 pulldown to display at 60Hz. the higher end models 800u/850u will do 2:2 frame repeat to display it correctly without judder at 48Hz. They are not out yet so no one knows if they do that as well as the 3:3 72Hz pioneer kuro plasmas. I have a HG10 and shoot (casual home movies) in 24p only. I don't do any editing (no interest so far). I have used WME to encode wmv files from the m2ts files and it does the IVTC correctly to extract the 24p material. clips look great.

February 27th, 2008, 02:32 PM	#14
Amit Mantri New Boot Join Date: Nov 2007 Location: NYC, New York Posts: 18	Sure, I have the process written down in a doc at home, i'll try to post it tonight.