HPX371 Noise Issue?
 

Playing around with an HPX381 and getting something odd.
Every time I do a move there is massive noise in the background. The first frame grab here is when the shot is static and the second one 2 frames later as it starts to move.
This doesn't look right to me, any ideas?
Steve

Which scene file and which gamma are you using? What is your pedestal set at?

Dan

I stuck in the BBC settings, so CineLike D gamma, Cinelike matrix, Master Ped +3.
Steve

Just looking at stills is always awkward, but it looks like the camera is using quite a bit of temporal electronic noise reduction to try to help mask the noise off the chips. It seems to be doing a reasonable job on static subjects, but is confused by movement (not surprisingly) - I assume it is relying on comparing static subjects on a frame-by-frame basis. What gain setting were you on?

I should have said David, it was -3db gain.
Same codec as the Varicam and never saw anything even remotely like that.
Steve

Here's a quick clip, when the gull's head clears frame look at the background foliage. This test clip is pretty much how it looks natively too, so it's not extra .mov or web compression.
Looks the same played straight from camera to TV via HDSDI.
Steve

Hey Steve,
What frame rate are you shooting at? Shutter Speed? Looks like lack of dynamic resolution is blurring the background so much that the codec is a little overwhelmed but not sure the noise increases based on my computer screen.
I do remember people complaining about noise in the dark areas with the HPX-300 and I have seen noise on my LCD Laptop screen from my HPX-500 so I always think Panasonic has more noise in the blacks than Sony at least on the lower end units.

Steve:

You are aware that Cinelike D is the noisiest gamma setting, right? I would shoot the same scene with the same settings, but shoot one pass with Cinelike D and one with BPress. Compare and let us know what you think.

The 371 should look pretty clean for a 1/3" chip. Barry Green did some evaluation when the 370 was released and if I recall, the 370 was a hair cleaner than the EX1 in some settings and just a shade less clean in others, but that is very clean, I describe the EX1 as "sterile" it is so clean. I don't think you have a defective camera, I think you just may be using the wrong settings to obtain the cleanest image.

Dan

It's 1080/25P AVC Intra 100, 1/50th sec shutter speed.
I don't think it's noise as such as you'd see it the same on the still as well as moving image, it definitely looks like compression but I would have thought that the codec was the strongest thing on the 371. I've certainly never seen this on any other camera I've used, and I've used a lot.
Steve

This seems like the most logical answer. The movement is uncovering what the chips are actually putting out while the static frame gets all of the attention from the noise reduction. Which if true does not paint the camera in too favorable of a light. Nor does it look like CMOS has been much of a step forward if you ever need to move the camera.

Here's another cropped area.
The first shot is on static, the second is about 3 frames later as I start to pan.
Steve

I don't think that's really relevant here - the issue is not how noisy the camera is in itself, but how the noise increases on areas of movement. A different gamma setting may make a difference to the overall noise level, but is unlikely to affect what Steve is seeing.
The question Steve's findings raise is how is it achieving that cleanness? The 300 was slated for poor noise figures, the 370 was brought out as the answer. I still can't think of a more likely answer than what I previously suggested - that electronic noise reduction was incorporated. And Steves examples seem to indicate it falls apart under certain circumstances.

Steve, have you still got the camera? If so, are you able to compare the AVC-Intra recorded output with the live output? (Or record the latter on something like a nonoFlash?) I find it hard to believe it's a codec issue, and if the live/recorded feeds show the same effect it would rule it out - would back up my theory about it being due to the front end.

The other thing I'd try is seeing how it behaves in low light with gain. Any noise reduction circuitry may not work as well then as at low gain settings.

Agree with all that David.

You can actually see it happening in the viewfinder, so I guess, yes that eliminates the codec maybe.

I questioned after Andy Shipsides' test exactly what was happening. Panny talked about this brand new sensor, but how much better could it be than the one they'd put in the 300 six months earlier? His tests showed less noise but also less sharpness as if it was largely software at work.

There is a very strange phenomena under discussion elsewhere where part of the image shifts while the rest stays put, very weird. Again it seems as if it's software working its magic but causing something odd to happen that it didn't do in the lab.

Steve

It would be interesting to compare live SDI output to a high quality, shaded monitor to AVCINTRA recording of the same. I agree with David, it would be good to try to troubleshoot the issue by isolating whether or not this is coming from the imagers/processing or the recorder end of it. Do you have a good relationship with Panasonic Europe/U.K.? What do they say about this? Your post is the first I have read of this particular issue with the 371.

Dan

It's not a pretty picture!
Here are a couple more grabs, there is some weird streaking going on here.
These are while moving the camera, and something is not keeping up. Look at the second shot and you've got multiple images of the plug holes.
These are crops and with levels boosted a bit just to make the problem a bit more visible, but it is very noticeable on the original moving footage and through the viewfinder while shooting.
Steve

Wow, those are really weird looking captures. I would return the camera to a Panasonic service provider, something is wrong with it.

Dan

Do you have a reference for that, Steve? If others have spotted it on other cameras, it sounds less like you've got a faulty unit, more like a fundamental problem with the camera design.
It's amazing sometimes what can get missed in the lab.

My favourite story is about the V2000 video cassette system, developed by Philips and Grundig to try to rival Betamax and VHS. (See Video 2000 - Wikipedia, the free encyclopedia .) According to wikipedia:
The story goes that many test tapes had passed between Philips and Grundig, with various line up recordings - unfortunately, not one of them had anything which showed up the lip sync problem! It was only ever brought to light when a Philips machine was bought by somebody whose friend had a Grundig. They started to share tapes and thought - "that's odd......"

Digressions aside, I agree with Dan. You need to take the matter up with Panasonic.

It's a different though possibly related effect being discussed here AG-HPX371E jello issue.... - DVXuser.com -- The online community for filmmaking
Steve

I managed to get a quick look at a 371 yesterday - too quick to be scientific about any testing, but long enough to draw some conclusions.

Firstly, I was able to get an effect which matches something like Steve is noticing, and the fact that others on the link Steve provided above are also seeing similar problems seems to prove Steves isn't a faulty unit - it's something that affects all 371s.

Secondly, I was able to see it via the HD-SDI output, so am sure it is nothing to do with the AVC-Intra codec - this doesn't surprise me at all.

I note that other people on Steves link have come to the same conclusion I did. The front end doesn't seem to be inherently less free of noise than the 301, in spite of what panasonic claim. It's achieving it's final output via electronic noise reduction, and whilst that may improve matters most of the time, sometimes it looks very odd. The worst effect I saw was whilst panning across a textured wall - the appearance varied with pan speed, and at certain critical speeds there seemed to be a low-level "explosion" of noise.

I would worry about buying this camera, never be quite sure when something would jump out to spoil things, even if it looked fine most of the time. I note in one of the posts one person actually swapped his 371 back for a 301 - although the noise level was worse, he felt it more predictable. I can sympathise with that.

David, in the link I provided there was actually something a bit different, although most likely related. It was a shifting of certain parts of the image when the camera was still, rather than the moving effects I've seen.
I'm sure you'll agree that when you see "even if it looked fine most of the time" that's just not anything like good enough for a camera with a full Pro label like the 300 series.
Steve

No, it’s not good enough. I’ve now had the chance to look more closely at the link you supplied, and also the one it references - HPX371e small Jello effect? - DVXuser.com -- The online community for filmmaking . I was quite surprised to see the subject was first mentioned over two months ago, and quite a few people seem to have seen some related effect. Also notable is the deafening silence from Panasonic – Barry Green said on the 19th May that “I looked at it and have no idea what that can be. I've sent the thread on to the Panasonic product manager to look at.”

That was two months ago. Maybe it’s time you posted again to that forum asking for an update, Steve?

There is quite an irony about it. Panasonic make quite a lot about the 371 using AVC-Intra and the I-frame nature of it, stressing how that means there’s no chance of codec performance being at all affected by movement. Not much use when even slight camera movements can cause bad effects in the front end!

It seems that most of the time, on normal pictures, no problem may be seen. But another question needs to be asked if it is indeed performing noise reduction. That can make a picture look cleaner – but destroy subtle low level detail at the same time. That may not be seen on the raw picture – but can severely affect performance if you try to push the images in post. A point I’ve made on another thread is that there is no point in 10 bit recording if the noise level swamps the least significant bits. There’s even less point in 10 bit recording if any noise reduction has destroyed the very slight changes that 10 bit could capture, but 8 bit couldn’t.

There has been a response from Panny on one of those threads, yesterday. Says they are aware of it and working on it, that's about it.
Steve

David,

On another forum, Steve Cooperman, P2 manager for the U.S. posted that there is a problem with all 370's and a firmware update is due at the end of July. It's clear that Panasonic is behind the curve a bit vs. Sony with CMOS development. Just another reason why I'm happy to be shooting with CCD's most often. I had a client recently reject my EX1 and swap to an HPX170 due to some flicker artifacts that weren't present with the 170.

My advice for those who are having issues with 300's or 370's is to find a used HPX2000 w/Intra board. Not easy, but they're out there occasionally. Obviously, a 2000 isn't a full raster native chip set, but it has a lot of other things going for it.

Regarding 8-bit vs. 10-bit, your characterization of " the very slight changes 10-bit could capture", I don't agree. 10-bit has 4X as many gray steps as 8-bit, 1024 vs. 256, same as HDCAM SR.

Jeff Regan
Shooting Star Video

Jeff, correct me if I'm wrong, but the 2000 is a full raster camera, ie 1280x720 full raster, at least when recording AVC Intra.
I think you're probably right about Panny not quite being up with Sony on the CMOS front, and wanting to keep up they've used software trickery to make up the distance, but it's backfired.
Steve

Steve,

Yes, full raster 720 chip set when using Intra. I should have written full raster 1080X1920 camera. Panasonic seems to have done a good job with their 4/3" CMOS DSLR's and VDSLR's, although there will have to be a lot of massaging if the AF100 is to be free of aliasing artifacts and excessive skew.

Let's see what the 370/371 firmware update looks like. EX1 and 3 have been out too long for Panasonic to be having these issues with the 300 successor.

Jeff Regan
Shooting Star Video

OK, found it. He says:
Firstly, it does not occur "infrequently", and it is easily visible on normal pictures, not just peering at a frame grab. After Steve first alerted me to it in this thread, I was able to demonstrate the effect within ten minutes of getting my hands on a 371, and within fifteen minutes consistently show it to an extent that was visually disturbing. All that involved was panning across a painted wall with some furniture in front - hardly an uncommon scenario. Likewise, it didn't involve frame grabs, just looking at a normal TV at a normal distance.

We'll have to wait and see what the firmware upgrade promised brings. But it was to reduce the high noise levels of the 300/301 that the 370/371 was introduced, so there must be a huge question mark over whether Panasonics technicians can solve this problem without reverting to the high noise levels of the 300/301 this was intended to cure. I think it would be extremely unwise to buy one of these cameras until an acceptable fix has been demonstrated.

It also raises the question of why such a fault was not spotted in the lab before it went on general sale? Or were they aware of it, but just hoped everybody would say how much quieter it was, and nobody notice the new fault?
That's true in a recording sense, but 10bit and 1024 steps will only show an improvement IF the source is good enough to distinguish more than 256 discrete grey steps in the first place. A noisy camera will not be able to resolve 256 grey scale steps discretely, and electronic noise reduction is almost certain to have the same effect. Recording a noisy camera into a 10 bit recording format is a little like pouring a pint into a gallon container - you still only have a pint. Just having the gallon container doesn't give you more than a pint of water.

And 10 bit recording will only give you "the full gallon" of quality if the front end of the camera is up to it.

Quote "t also raises the question of why such a fault was not spotted in the lab before it went on general sale? Or were they aware of it, but just hoped everybody would say how much quieter it was, and nobody notice the new fault?"

Hmmm., doesn't it just! Surely if that's the case, on this, a professional product, then they've really shot themselves in the foot. Or it could be that we're all just barking up the wrong tree, we'll have to give them the benefit of the doubt.
Agree 100% with everything in your last post David.

Jeff, yes the full raster 720 thing was me being a bit pedantic - it's one of my pet hates people confusing "full raster" with 1920x1080 when that's not what it means (not you Jeff, I know you know!)
Steve

David,

Yes, in another thread you brought up the noise in the front end of the camera making 10-bit acquisition moot. I just don't want people to think that 10-bit is always a "very slight change" over 8-bit. My editor showed me graphic evidence of 10-bit AVC-Intra 100 over 8-bit DVCPRO HD using my HPX2700. Much more range in post for grading and color correction before seeing banding and added noise. 4:4:4 color space record option isn't the only reason HDCAM SR is used. I almost always shoot at -3db gain with my 2700, as I did with my HDX900, to enable the recording of the cleanest signal possible.

AVC-Intra 100 is a big step over DVCPRO HD due to full sample recording using square pixels, twice as efficient in compression and 10-bit depth. 10-bit recording was not available in a one-piece camcorder until Panasonic came out with the HPX2000 in early 2007, and then the HPX3000, 2700 and 3700 as well as HPX300 and 371. Sony's 10-bit camera solution is HDCAM SR using tape and large, six figure cameras.

Panasonic's putting 10-bit into camcorders for under $20K is a breakthrough. Now they need to get their CMOS act together to make full use of it for under $10K camcorders.

Jeff Regan
Shooting Star Video

This has been an elightening thread. The HPX-300/370 has not been the best of launches for Panasonic.

I was watching "Expidition Great White" on Discovery last night and the images from most of the cameras were just beautiful. A lot of detail, great color range and accuracy. I did a quick internet search to try and discover the cameras being used and found an artcle mentioning the HPX-2000 as the main cameras. I would guess by the look they were shooting 720p60.

1080p is often the buzz but to my eyes, 720p deilvery is plenty for now and some time to come unless you are aiming for the theatre production.

You've misread what I posted - I didn't say the above but rather "the very slight changes that 10 bit could capture, but 8 bit couldn’t". The inference is that those "very slight changes" (in grey scale level) may be imperceptible in normal viewing - but desirable for extensive post-processing. Assuming they exist - and aren't masked by noise or camera processing.

And I was quite clear in that other thread [ http://www.dvinfo.net/forum/panasoni...ml#post1547088 ] that 10 bit recording was desirable for high end cameras. The evidence is that it is far less relevant (if at all) for lesser models. In the other thread I quoted from the EBUs tests - "The 8-bit bit-depth is sufficient for mainstream programmes, but 10-bit bit-depth is preferred for high-end acquisition." I consider the HPX371 a very long way from "high-end acquisition" - especially after what's been learnt in this thread - and hence the recording being 8 or 10 bit is totally irrelevant.
Maybe so, but the 2700 is far more expensive than the 371, so what does that prove? The 2700 is about 3-4x more than the 371 in the UK - and that doesn't include any lens! And as said before, how do you know what improvement is down to 10 bit, how much down to no sub-sampling of luminance and chrominance, and how much to just generally better compression? AVC-Intra 100 is better than DVCPro-HD, yes, but that's for a lot more reasons than bit-depth.

In the cold light of day I find Steve Coopermans posting even less comforting than it seemed last night.

Maybe it's just careless wording, but he says that "We plan to have a free firmware upgrade at the end of July", then "At that time, we'll have further information". He never actually states that the firmware upgrade will solve the problem, though it's what everybody is assuming. "Further information" is nothing like "the firmware upgrade which will solve the problem", is it? Perhaps he'd like to clarify the matter directly on this forum?

I've also just been shown another instance of the fault occuring, this time on test raw chromakey footage, daylight and 0dB gain. Yet again, moving objects show a noise trail, and there seem to be random bursts of occasional noise on the background. Just what you want for chromakey? And that's at 0dB, what will it be like at 12 dB or more? Steve Coopermans assertions that the problem "occurred infrequently and was not very noticeable" just don't seem to agree with what I, Steve, and others are finding in reality.

When the 371 was first released, Panasonics claim was that it's 1/3" chips were "Rivalling the image quality and sensitivity of ½” imagers". This saga has done nothing to substantiate those claims.

David,

I agree with most of your assertions, broadly speaking. AVC-Intra 100 offers more than 10-bit depth vs. 8-bit depth with DVCPRO HD, as I posted, ie; no 3/4 subsampling, more efficient compression. lower noise floor.

My point is that 10-bit is considered high-end, yet Panasonic has made it attainable at a price point that nobody else can touch(save RED One with 12-bit). An HPX2700 body is $19,950 with trade-in, B-stock HPX2000's can be had for $15-16K USD. Add a lens and viewfinder to a 2700, and you're at 3X the cost of a 370, but still $70K lower than Sony's only 10-bit format, HDCAM SR.

I agree that visually, looking at camera original footage, 10-bit is hard to discern over 8-bit(especially with an 8-bit display device), but so it 4:4:4 color space vs. 4:2:2. The computers know the difference, and we both agree 10-bit(as well as 4:4:4) can pay dividends in post.

Because Panasonic keeps messing up their CMOS models doesn't mean that AVC-Intra 100 isn't the best codec available for the money now for 2/3" CCD cameras and hopefully at some point for CMOS cameras as well. Meanwhile, Sony handicaps their under $20K cameras with a 35Mbps, 8-bit, Long GOP, 4:2:0 codec and their under $100K cameras with 8-bit, 50Mbps codecs.

Anybody can buy a $20K Pansonic camera body that is 10-bit, but they can't do that with Sony. I don't want to make it seem like 10-bit is THE reason to buy a camera, but it's not a trifle either.

Jeff Regan
Shooting Star Video

Jeff,

Actually I'm pretty sure that the EX1 does ten significant bits out of the HD-SDI. It's recording it that's the problem.

Also, it seems that transcoding 8-bit to 10-bit in post will give you more latitude than if you stayed in 10-bit.

But I do of course agree, take ten bits if you can get it.

Peter,

You're correct, the EX cameras have 10-bit HD SDI uncompressed outputs.

I don't agree that transcoding from 8-bit to 10-bit vs. acquiring and staying in 10-bit will offer more latitude,
although editing on a 10-bit time line with something like Pro Res can be beneficial vs. staying in 8-bit.

Jeff Regan
Shooting Star Video

Jeff, sorry about the typo :(. I meant "... than if you stayed in 8-bit."


P.S. For some reason I can't edit the post.

Adam Wilt just published his review of the HPX370 and addresses the noise on movement issue:

ProVideo Coalition.com: Camera Log by Adam Wilt | Founder | Pro Cameras, HDV Camera, HD Camera, Sony, Panasonic, JVC, RED, Video Camera Reviews

Jeff Regan
Shooting Star Video

Thanks Jeff, interesting read.
One thing he comments on is using the Synchro Scan buttons to change frame rates and it didn't work properly but started to work once he'd fiddled about with it. From what I saw it only works when the shutter switch is in the Off position, so he probably switched it off and then it worked.
Steve

------------------------------------------------------------
P2 Asset Support System (PASS) Newsletter
This is distributed to PASS members
August 5, 2010 / Issue 32
------------------------------------------------------------
News for P2 products
I. [Update] AG-HPX370, AG-HPX300 series
II. New CAC files for AJ-HPX3700 series etc.
------------------------------------------------------------
I. Updated: AG-HPX370 ,AG-HPX300 series
------------------------------------------------------------
AG-HPX370 series
1. Two types of P.A.P.Filters
You can choose two kinds of P.A.P. filters to improve image quality.
2. Applicable Lens models for CAC function were added.
These CAC files require updating the firmware of AG-HPX370 series.

Latest version software of AG-HPX370 seires
https://eww.pavc.panasonic.co.jp/pro...pghpx370_e.htm

AG-HPX300 series
Applicable Lens models for CAC function were added.
These CAC files require updating the firmware of AG-HPX300 series.

Latest version software of AG-HPX300 seires
https://eww.pavc.panasonic.co.jp/pro...pghpx300_e.htm

*"P2 Status logger" is available.
Log-in -> "Download P2 Status logger"
P2 Asset Support System | Broadcast and Professional AV Systems

------------------------------------------------------------
II.New CAC files
------------------------------------------------------------
Chromatic Aberration Compensation (CAC) is to minimize
neighboring blur, it automatically compensates the
blur that is caused mainly by lens chromatic aberration.

The CAC function is automatically activated when a lens in the CAC database is connected.
The CAC database can be updated by importing CAC files, which are
available in PASS.

The following CAC files for AJ-HPX3700, AJ-HPX2700,AJ-HPX3000 series are newly available
For Fujinon HA22X7.3BERM-M58
For Canon KJ22ex7.6B IASE / IRSE

The following CAC files for AG-HPX370 , AG-HPX300 series are newly available
For Canon KT17ex4.3B IRSD PS12

Learn more details and download CAC files:
PASS portal P2 Asset Support System | Broadcast and Professional AV Systems
Log-in > "Library" > "Utility software" > CAC Files (for P2 Camerarecoder)


So is this what we were waiting for, or is it a HPX300 emulator like somebody mentioned on another board???

Yes, Niels, that's how it appears.
As you know the problems with the HPX300 were mainly high noise and low sensitivity. "Solved" by the miraculous HPX370. But it turns out at the expense of these issues we've been talking about.
"Solution", turn off the processing which "solved" the HPX300's noise issues, which leaves you, as you say, with an HPX300.
Huh?
So you can a) A 300 with noise, b) a 370 with little noise but weird ghosting problems, or c) something from Sony with low noise and no ghosting!
Steve

Though I suppose option c still comes with poor form factor, although Visual Impact in the UK has now taken delivery of the PMW320, so that's that out of the way too.
Steve

Well ridiculous right? Maybe we can have some sort of explanation from somebody from Panasonic???