How PsF video from the V1 is different than "p" or "F" video

[Douglas Spotted Eagle]

Originally Posted by Steve Mullen

The connection path has nothing to do with encoding and decoding nor deinterlacing so it's unclear why you have posted about component video. It "clarifies" nothing this group is talking about.

There are questions above in the thread about printing back to the V1, and how the V1 outputs. It also clarifies that the component and HDMI outputs on the V1 are always "i" unless you're viewing on a 480p display, in which case it converts 1080i to 480p.
Thanks for the catch.

[Piotr Wozniacki]

I have done some more tests today; in 25p I changed the sharpness from 4 through 7 (default), 9 up to 12! And I can tell you that unless you're fixed on those offending areas (where contrasty lines cause aliasing artifacts of dancing dots), the progressive video is just stunning, especially at 9 sharpness setting - but only when played back through DVI as a computer file, without deinterlacing.

Which doesn't change the fact that - after having established that the component playback on an 1920x1200 LCD engages deinterlacing and cuts the resolution - I really don't know which workflow I should adopt to actually deliver the progressive video at full resolution. After HD or BlueRay DVD becomes available, it will have to be connected through the same component (or HDMI) inputs, and the deinterlacer will kick in...

Basing on what Steve said about 24p being the only format that can be played back without deinterlacing (thus at full v-rez), and on my own testing - I can't help feeling frustrated as a PAL user. Have even been thinking on replacing the V1E with a V1U (no progressive issues plus 24p, for less money) - considering any possible delivery methods, what disadvantages can you see of using it in a PAL land?

[Thomas Smet]

I suggest down converting to 720p. While it may not be as good as true 1080p, although the difference you will see is very very small, you will at least get 720 vertical lines on pretty much every single HDTV on the market. With 1080p some HDTV's may give you 1080p while a lot of them will at best give you 540 lines. Using 720p gives a constant level of quality. This is my opinion of course but you do pretty much eliminate any HDTV deinterlace issues and sharpness issues by delivering as 720p.

[Tony Tremble]

Originally Posted by Piotr Wozniacki

Basing on what Steve said about 24p being the only format that can be played back without deinterlacing (thus at full v-rez), and on my own testing - I can't help feeling frustrated as a PAL user. Have even been thinking on replacing the V1E with a V1U (no progressive issues plus 24p, for less money) - considering any possible delivery methods, what disadvantages can you see of using it in a PAL land?

You would have no Prime Support.

You'd constantly have to worry about flicker with lighting.

You testing or you post production workflow is at fault. How are you testing? Include your entire workflow, make and model of TVs and monitors and methods of connection.

Don't let your lack of knowledge/experience kid you into making a decision you'll regret.

TT

[Piotr Wozniacki]

Originally Posted by Thomas Smet

I suggest down converting to 720p. While it may not be as good as true 1080p, although the difference you will see is very very small, you will at least get 720 vertical lines on pretty much every single HDTV on the market. With 1080p some HDTV's may give you 1080p while a lot of them will at best give you 540 lines. Using 720p gives a constant level of quality. This is my opinion of course but you do pretty much eliminate any HDTV deinterlace issues and sharpness issues by delivering as 720p.

In Vegas 7.0d, I encoded some hand-held video (720/25p, sharpness at 9):
http://rapidshare.com/files/18897429...9_run_gun_.m2t

All the artifacts (aliasing as dancing dots; look at the car body or the green shed roof of corrugated iron sheet) that 1080/25p showed are still there, but IMHO you must be watching very carefully for them to actually be distracting. I welcome your opinions.

PS. And here's the link to the same video, in 1080/25p format - can you see any quality differences (apart from resolution)?
http://rapidshare.com/files/18916880..._hand-held.m2v

[Bob Grant]

Originally Posted by Steve Mullen

WHAT PROOF DO WE HAVE THAT THE V1 ENCODES AS FIELDS? MUST IT USE FIELDS TO BE COMPATIBLE WITH OTHER 1080I HDV EQUIPMENT? I ASSUME SO.

-------------------------------------------------------------------
IN SHORT, A DEINTERLACER MAKES THE ASSUMPTION THAT 25P AND 30P ARE TO BE TREATED AS 50I and 60I.

IN SHORT, 25P, 30P, 50I, AND 60I NEVER CAN LOOK PERFECT.

From -- http://www.hometheaterhifi.com/volum...e-10-2000.html

I believe the V1 is using fields so that it is compatible with other 1080I HDV equipment.

How can you say 25p etc can never look perfect, you keep hammering this point and yet offer no justification. Every R50 country has been broadcasting 25p SD since the first day of broadcasting. Now they're doing the same thing in HD, sorry but it works and works very well. Many R50 shooters spend considerable time/money running their footage through de-interlacers to produce 25PsF, simply because it looks better then 50i, the de-interlacers in the display devices (if not CRTs) don't have to think very hard to do a perfect job of combining the fields to produce a frame.

The implication of what your saying to anyone with a vague concept of how this stuff works is that the V1 is producing defective 25PsF that is somehow fooling de-interlacers. Maybe your right, I can't really say if you've unconvered some new concern. From what I've seen though I doubt it.

As you said the de-interlacers should be able to detect that there's no temporal separation between the fields in the 50i and simply weave (merge) the fields. This is a no brainer. I'd have serious doubts that this wasn't reasonably well taken care of in most EDTV and HDTVs, the device designers would obviously realise that telecined film would be one of the most common things displayed on their products.

What DOES pose a challenge for de-interlacers is 50i, not 25PsF. If you want to hammer the point about de-interlacers in display devices 50i is their nemesis, not 25PsF.

How do I manage to get 25PsF back to tape?

Simple.

Vegas can capture the 25PsF from tape to a file. The file has the P flag set. I can print the same file back to tape, the flag goes along for the ride and the file is not re-encoded. Vegas's HDV PTT permits a direct m2t file to tape print.

What I cannot do as Vegas doesn't support it, is encode a file with the same flag, that's all.

Why doesn't Vegas support encoding with the P flag?

Well according to Sony it doesn't matter, if mastering to HDCAM or XDCAM the results the display device gets will be the same whether going directly to tape or mastering to HDV (compression isssues aside) and then dubbing to HDCAM or XDCAM for broadcast.

[Piotr Wozniacki]

Originally Posted by Bob Grant

Vegas can capture the 25PsF from tape to a file. The file has the P flag set. I can print the same file back to tape, the flag goes along for the ride and the file is not re-encoded. Vegas's HDV PTT permits a direct m2t file to tape print.

What I cannot do as Vegas doesn't support it, is encode a file with the same flag, that's all.

Why doesn't Vegas support encoding with the P flag?

Well according to Sony it doesn't matter, if mastering to HDCAM or XDCAM the results the display device gets will be the same whether going directly to tape or mastering to HDV (compression isssues aside) and then dubbing to HDCAM or XDCAM for broadcast.

What makes you think Vegas doesn't convey the P flag to it's 1080/25p or 720/25p encoded output from a V1E's 25PsF file? Both Vegas-encoded clips I posted above are properly recognized as progressive by Vegas, Canopus Procoder 2, Ulead Mediastudio and VideoStudio, and other NLE's. Unlike the raw .m2t files, which are only recognized as P by Vegas; other NLE's treat them as 50i (Upper Filed First).

[Piotr Wozniacki]

Originally Posted by Bob Grant

[...]the de-interlacers should be able to detect that there's no temporal separation between the fields in the 50i and simply weave (merge) the fields. This is a no brainer. I'd have serious doubts that this wasn't reasonably well taken care of in most EDTV and HDTVs, the device designers would obviously realise that telecined film would be one of the most common things displayed on their products[...]

Bob, pure logic says it should be done this way - just weaving when no temporal difference is detected between fields, but apparently it doesn't work this way, at least not with the majority of HD tvs and monitors.

Why?

It beats me. It'd be so much easier to be done than all the sophisticated and expensive methods of deinterlacing 50i...

[Steve Mullen]

Originally Posted by Piotr Wozniacki

I have done some more tests today; in 25p I changed the sharpness from 4 through 7 (default), 9 up to 12! And I can tell you that unless you're fixed on those offending areas (where contrasty lines cause aliasing artifacts of dancing dots), the progressive video is just stunning, especially at 9 sharpness setting - but only when played back through DVI as a computer file, without deinterlacing.

You just confirmed my tests -- P does not get a "busy" pix until you go above 10. And, you can go to 12+ with I. The V1 does not add much EE, it only increases bandwidth.

And, you are correct -- because the deinterlacer can find the 2-3 cadence, 24p does have an advantage IF you are willing to live with motion judder and cadence judder and editing issues and monitoring issues. It seems far simpler to only shoot 50i -- it solves all problems and IMHO it looks so much better.

"Bob, pure logic says it should be done this way - just weaving when no temporal difference is detected between fields, but apparently it doesn't work this way, at least not with the majority of HD tvs and monitors."

You found the issue. Bob always says "should" but he is clearly not reading what the experts are saying about how deinterlacing "actually" works. I'm not an expert, but I'm BEGINNING to understand what the experts say. Everyone is free to read the original, and they should.

For example, 24p is perfect ONLY if the logic that checks the cadence works. Testing revealed 80% of our HDTV'S fail to pick up the 2-3 cadence. This is why there is so much interest in 1080p HDTVs. Logic says it should make no difference if the signal is sent from your HD DVD player via 1080i because the the deinterlacer should be able to turn it back to P. Yet, tests revealed this doesn't actually happen. So although it should't matter, but it does.

(Watching the Oscars, so many of the clips they showed were I that was not --even at the pro level -- correctly deinterlaced. It's clear many video pros do not understand how to convert I to P.)

And, the idea of converting to 720p is an interesting one for production. Some folks do convert to DVCPRO HD which is very EZ to edit and is 4:2:2. By the way, this might filter out the aliasing.

[Bob Grant]

Originally Posted by Piotr Wozniacki

What makes you think Vegas doesn't convey the P flag to it's 1080/25p or 720/25p encoded output from a V1E's 25PsF file? Both Vegas-encoded clips I posted above are properly recognized as progressive by Vegas, Canopus Procoder 2, Ulead Mediastudio and VideoStudio, and other NLE's.

Sorry maybe I'm not being clear. You can encode to 25p but that will NOT PTT, you can try, Vegas will try, the deck will just come up with a "Invalid Format" error.

You can bring that file into Vegas and yes sure it'll see it as "P" as you've seen.
As Steve pointed out with mpeg-2 you've got three options:

1) Encode frames - flag says P This is 25P
2) Encode fields - flag says I This is 50i
3) Encode fields - flag says P This is 25PsF

Option 1 is what the Canon cameras do but it's outside the HDV spec and no VCR will play or record it.
Option 2 is obvious.
Option 3 is what the V1E/P does. This is what I trying to get Vegas to record to tape. This is what the Vegas engineers have confirmed Vegas cannot do.

This encoding needs to be done at the encoder level, I suspect it's a limitation of the MainConcept encoder.I t might be possible using a mpeg-2 utility that gives you direct access to the flags.

[Piotr Wozniacki]

Sorry Bob, I misunderstood. I'll try printing back to tape over the weekend. So you say it's impossible from Vegas timelime, or from a separate (rendered to 1080/25p) file within Vegas?

[Bob Grant]

Originally Posted by Steve Mullen

You just confirmed my tests -- P does not get a "busy" pix until you go above 10. And, you can go to 12+ with I. The V1 does not add much EE, it only increases bandwidth.


"Bob, pure logic says it should be done this way - just weaving when no temporal difference is detected between fields, but apparently it doesn't work this way, at least not with the majority of HD tvs and monitors."

You found the issue. Bob always says "should" but he is clearly not reading what the experts are saying about how deinterlacing "actually" works. I'm not an expert, but I'm BEGINNING to understand what the experts say. Everyone is free to read the original, and they should.

For example, 24p is perfect ONLY if the logic that checks the cadence works. Testing revealed 80% of our HDTV'S fail to pick up the 2-3 cadence. This is why there is so much interest in 1080p HDTVs. Logic says it should make no difference if the signal is sent from your HD DVD player via 1080i because the the deinterlacer should be able to turn it back to P. Yet, tests revealed this doesn't actually happen. So although it should't matter, but it does.

(Watching the Oscars, so many of the clips they showed were I that was not --even at the pro level -- correctly deinterlaced. It's clear many video pros do not understand how to convert I to P.)

And, the idea of converting to 720p is an interesting one for production. Some folks do convert to DVCPRO HD which is very EZ to edit and is 4:2:2. By the way, this might filter out the aliasing.

Steve,
you're still ignoring the issue of what can happen even if it doesn't do what it should. And you're still unable to explain how us PAL folk have been coping with 25PsF all these decades without this problem coming to light.

Now here's something new. I've always though this aliasing, or line twitter or crawling ants or whatever you want to call it thing was a bit of a red herring, I really hadn't seen it, until today. Funny thing is I wasn't even looking for it, I was testing our new HDMI to HD/SD SDi converter, great little box, stunning SD from that little V1 and box.

Anyway I just wanted to compare the HDMI from the camera straight into the monitor.

The camera was locked of, the subject was totally static. In this test scenario the camera is always sending 50 fields per second to the display device and there's no temporal separation between those fields, no motion vectors or whatever can be derived because nothing moves between the two fields. Regardless of that, no matter if the camera is in 25p or 50i the display device has no way of knowing, it's always getting 50 fields. Whatever de-interlacing errors the display device might be making it'll make the same mistakes whether it's getting 50i or 25PsF, it simply has no way of knowing. Any differences in the results have to be the result of what the camera is doing.

Now, hopefully you've taken that on board.

In 25p there's serious artifacts on horizontal lines, anything within a few degrees off horizontal has serious issues.

In 50i the problem decreases dramatically but I'm far from 100% certain that it's gone either.

Now please keep in mind this is not recorded video, this is straight from the camera head, live. No mpeg issues involved, no flags, nothing.

But what are those issues? They're certainly not line twitter, they're something quite unique. If you look very carefully you see that they seem to be happening at around the pixel level.

Now when we looked at the rest of the frame where we had white paper there was the typical chroma noise, all looked 100% normal. However when we pointed the camera at our blackout cloth again we go that really major noise as the gian came up. But this stuff looks nothing like normal video noise. The noise pixels are black, there's no chroma in them and they're bigger than a pixel, now that's really odd. Yet we still had some bright white areas in the frame, they're fine, a reasonably amount of pixel sized chroma noise. Getting back to the noisy parts of the frame, the noise is worse in as the levels fall off and can suddenly change nature at an edge or as you approach a high contrast edge.

So, going back to the horizontal lines and the whatever it is. I really can't say just what's happening there. I've spent enough time staring at problems with interlace video etc to know this is something new. I've eliminated display device issues what else could it be?

Well, elsewhere in the frame, I looked at the black text on the white paper. That text was riddled with that same blocks of noise, the stuff bigger than a pixel with no color in it. So what happens I wonder when that hits the EE circuits in the camera. Maybe this is something. Those lines were very thin, maybe only a pixel or two high. so the EE in the camera is trying to decide if there's an edge to enhance or not but the noise is fooling it. So the edge is being enhance sometimes and not others due to the random noise. Bingo, this fits the observed effect.
We know from other observations that the noise level is dramatically worse in P than I, we know the display can't tell the difference between I or P from the camera as it's always getting fields.
And it might explain why Sony suggest turning Sharpness down to 3. With too much EE the circuit can push the edge into an adjacent pixel but the noise confuses the logic, sometimes it does, sometimes it doesn't. The effect of that on a horizontal edge would look like line twitter or more precisely as some have described it "marching ants"

[Tony Tremble]

Originally Posted by Bob Grant

Steve,
you're still ignoring the issue of what can happen even if it doesn't do what it should. And your still undable to explain how us PAL folk have been coping with 25PsF all these decades without this problem coming to light.

Now here's something new. I've always though this aliasing, or line twitter or crawling ants or whatever you want to call it thing was a bit of a red herring, I really hadn't seen it, until today. Funny thing is I wasn't even looking for it, I was testing our new HDMI to HD/SD SDi converter, great little box, stunning SD from that little V1 and box.

Anyway I just wanted to compare the HDMI from the camera straight into the monitor.

The camera was locked of, the subject was totally static. In this test scenario the camera is always sending 50 fields per second to the display device and there's no temporal separation between those fields, no motion vectors or whatever can be derived because nothing moves between the two fields. Regardless of that, no matter if the camera is in 25p or 50i the display device has no way of knowing, it's always getting 50 fields. Whatever de-interlacing errors the display device might be making it'll make the same mistakes whether it's getting 50i or 25PsF, it simply has no way of knowing. Any differences in the results have to be the result of what the camera is doing.

Now, hopefully you've taken that on board.

In 25p there's serious artifacts on horizontal lines, anything within a few degrees off horizontal has serious issues.

In 50i the problem decreases dramatically but I'm far from 100% certain that it's gone either.

Now please keep in mind this is not recorded video, this is straight from the camera head, live. No mpeg issues involved, no flags, nothing.

But what are those issues? They're certainly not line twitter, they're something quite unique. If you look very carefully you see that they seem to be happening at around the pixel level.

Now when we looked at the rest of the frame where we had white paper there was the typical chroma noise, all looked 100% normal. However when we pointed the camera at our blackout cloth again we go that really major noise as the gian came up. But this stuff looks nothing like normal video noise. The noise pixels are black, there's no chroma in them and they're bigger than a pixel, now that's really odd. Yet we still had some bright white areas in the frame, they're fine, a reasonably amount of pixel sized chroma noise. Getting back to the noisy parts of the frame, the noise is worse in as the levels fall off and can suddenly change nature at an edge or as you approach a high contrast edge.

So, going back to the horizontal lines and the whatever it is. I really can't say just what's happening there. I've spent enough time staring at problems with interlace video etc to know this is something new. I've eliminated display device issues what else could it be?

Well, elsewhere in the frame, I looked at the black text on the white paper. That text was riddled with that same blocks of noise, the stuff bigger than a pixel with no color in it. So what happens I wonder when that hits the EE circuits in the camera. Maybe this is something. Those lines were very thin, maybe only a pixel or two high. so the EE in the camera is trying to decide if there's an edge to enhance or not but the noise is fooling it. So the edge is being enhance sometimes and not others due to the random noise. Bingo, this fits the observed effect.
We know from other observations that the noise level is dramatically worse in P than I, we know the display can't tell the difference between I or P from the camera as it's always getting fields.
And it might explain why Sony suggest turning Sharpness down to 3. With too much EE the circuit can push the edge into an adjacent pixel but the noise confuses the logic, sometimes it does, sometimes it doesn't. The effect of that on a horizontal edge would look like line twitter or more precisely as some have described it "marching ants"

Bob

If it is any help. I observed _exactly_ what you have detailed above. I have been calling those running horizontal lines twitter right from the start. I think it might have been confused with twitter being generated by poor displays. I know that what I have, many examples of, is generated in camera and not as a result of the display.

I posted frame grabs which prove that fact. But again things like that get missed. Turning down the sharpness can mitigate those artefacts to some extent but not before drastically affecting the resolution of the image. I describe it as turning to "mush."

Steve's explanations have never come close to explaining the problems. As I stated in discussions way back the artefacts have nothing to do with de-interlacing.

My dealer explained a what the issue is and that it inherent to the the way the camera works in progressive mode. He was given the info by Sony UK.

All the firmware fix did was to bring the V1E more into line with the V1U by reducing the oil paint effect. Brett Sherman's video of the red shed showed the V1U has exactly the same rendering problems as the V1E when in progressive.

The remaining artefacts are nothing to do with PsF. Fact.

There should not be the necessity to endure all these crazy workarounds and I can't believe it has been suggested to render to 720P. The whole selling point for the V1 was 1080P!!!

I viewed Piotr's video at sharpness 9. Well, if the noise on the green roof is seen as being acceptable then good luck!!

TT

[Piotr Wozniacki]

Originally Posted by Tony Tremble

I viewed Piotr's video at sharpness 9. Well, if the noise on the green roof is seen as being acceptable then good luck!!

TT

Tony, nobody ever said it was acceptable! My point was to show it doesn't get "linearly" worse with the sharpness increasing from the "3" setting.

I'm still seriously considering to revert back to the A1, where such artefacts are simply not present (at least I haven't noticed them on my unit during testing). The picture is very "quiet", but IMHO it lacks the vividity that the V1 at sharpness 9 produces. Now that I have to make my decision, I'd be very grateful if you posted a clip from your A1 with colors as vivid and natural as those in my V1E clip, edges eqally sharp but not overenhanced - and still NO noise whatsoever. If you can do that, many of those still on the fence would benefit. Thanks!

PS. Of course, do not mimick my bad panning and zooming in progressive, as the clips I posted were intended to take the V1E's 25p mode to the limits; the stuttering is very excessive. The A1 seems more forgiving also in this respect.

[Piotr Wozniacki]

Originally Posted by Tony Tremble

There should not be the necessity to endure all these crazy workarounds and I can't believe it has been suggested to render to 720P. The whole selling point for the V1 was 1080P!!!
TT

Also true, Tony. If I encoded and posted my clip in 720p as well, it was only to prove the point the artefacts are still there.