DV Info Net

DV Info Net (https://www.dvinfo.net/forum/)
-   Open DV Discussion (https://www.dvinfo.net/forum/open-dv-discussion/)
-   -   Camcorder comparisons? (https://www.dvinfo.net/forum/open-dv-discussion/19604-camcorder-comparisons.html)

Michael Wisniewski January 10th, 2004 09:42 PM

Camcorder comparisons?
 
What kind of tests would you recommend for camcorder comparisons ... how would you setup the tests?

I was thinking we could come up with a list of tests that are rigorous and repeatable. Anyone could run the tests on their camcorder and post the results online so we can compare the results.

Example tests: 16:9, low light, image clarity/sharpness, color fidelity, etc.

Frank Granovski January 10th, 2004 10:35 PM

This site shows new cams with their basic specs. Plus links to pic comparisons.

http://www.grandeye.com.hk/etx/900/index.html
Quote:

Example tests: 16:9, low light, image clarity/sharpness, color fidelity, etc.
[list=1][*]16:9 - only some cams have good 16:9 - and we know which ones these are[*]low light - you would have to set lux/test guidelines[*]sharpness - we would need wave-form monitors[*]color - I for one am color blind. I'll be happy to oblige, though. :-))[/list=1]

Marco Leavitt January 10th, 2004 10:37 PM

I think this is a great idea. The biggest problem obviously is the inability to shoot the same subject under the same lighting conditions. Shooting resolution charts really wouldn't accomplish much. If everyone had light meters, I guess we could all contribute shots of a room with the exact same level of ambient light at a various levels for one test. That might tell something. I have a GL1 and would be glad to participate if other people are interested. Maybe we could come up with an object that everybody could buy locally and shoot under tungsten bulbs and daylight. Or else we could mail an object back and forth, but there are clear problems with that. For one thing, it would take forever. Given the number of retail chains out there, surely there's some kind of colorgul widget we could all find locally.

Frank Granovski January 10th, 2004 10:41 PM

How about the strength test? Drop 'er cam from 1 foot and see if she still works; then from 2 feet, etc. :-))

PS: and don't exclude these fine 3 tests: heat test, cold test and 'er humidity test. ;-))

Glenn Chan January 10th, 2004 11:17 PM

http://babelfish.altavista.com/babel....html&lp=ja_en has lots of camcorder comparisons.

Shooting things which we know what they're supposed to look like might work. Trees, people, etc.

Michael Wisniewski January 11th, 2004 12:23 AM

Quote:

only some cams have good 16:9 - and we know which ones these are
What's a good way to show the quality of the 16:9 mode? Would having standardized shots work? Maybe use a close up with a medium shot etc. Maybe add in some fast panning or bright lights to see how it behaves? making it repeatable would be the goal so that we could compare the same shot/video from camcorder to camcorder
Quote:

low light - you would have to set lux/test guidelines
How do you set this up? Is there an common instrument that measures lux?
Quote:

sharpness - we would need wave-form monitors
okay i'll have to research this first
Quote:

I for one am color blind
That settles it your in charge of the color tests

Michael Wisniewski January 11th, 2004 12:25 AM

Some more tests:

Handling blown out highlights
Grain at different light levels

Glenn Chan January 11th, 2004 01:16 AM

measuring sharpness would kind of only require one person with a wave-form monitor, since we can transit DV stills or short clips over the internet with no loss. However, I don't think a wave-form monitor is necessary as long as the *methodology* is consistent. This would give meaningful, comparable results. You could just hook your camera up to a computer and view on a computer monitor, which have excellent resolution.

However there is a small problem with sharpness tests in that it can be affected by the Kell factor. You have to pan your camera in sub-pixel increments if you want to measure lines of resolution off a resolution test chart (like the EIAJ one). You also have to frame the test chart very carefully, and print it out at resonably high resolution. However you only need to do all that if you want really accurate results.

Bryan Beasleigh January 11th, 2004 02:48 AM

Michael, you'll drive yourself nuts. It's easier said then done.

When I was researching which camera i should buy I made a test tape and took it with me to any camera store i went to and shot as much comparison footage as possible.

If a store had 3 cameras that I was considering I'd tape footage of all 3 , same subject same light. if the store had 2 i'd do both. it's hit and miss.

Frank Granovski January 11th, 2004 02:52 AM

Yup. That's the way to do it: take a fresh Fuji miniDV tape with you to the cam shop, then take test footage of everything. The salesperson will love ya. :-))

Jeff Donald January 11th, 2004 07:12 AM

As someone who views images all day long, hundreds of images, sometimes thousands, in my role as a photography teacher, let me say you're missing the most important part. The subjective nature of our medium and finished work makes picking cameras extremely difficult when comparing performance specs.

I have looked at cameras and used cameras that are supposed to have very sharp images, but they don't look as sharp to me as other cameras (which are supposed to be less sharp). The same with color and low light and many other aspects of camera performance. I've had clients say what great low light images they got with their camera. But to me the image was worse than many. The image was very noisy, but the client liked the greater amount of color and the noise didn't really bother him. I would have rejected his camera based on the noise, but he loved it based on the color.

Other subjective features weigh heavily in our selection and use of cameras. I will sometimes pick a lower end camera to shoot with because it will have better performance in key areas. I've shot with cameras that might have better focusing capability, better viewfinder, better audio, over a camera that might have a slightly better picture.

A project I worked on several years ago gave me the opportunity to shoot with two different cameras. I choose the model with a slightly lower quality image, but a much better viewfinder. Why? The project called for a lot of run-n-gun style shots. It wouldn't be possible to use an external monitor and probably little opportunity for retakes. The better viewfinder meant I could make sure the image was in focus (sharp as possible) and maximize the potential for using the footage. The camera with the potentially better image wouldn't have mattered if a lot of the scenes were not in critical focus. Besides in most cases you can improve the image in post.

I really think that in picking a camera each user should prioritize their needs in a camera and try to match the camera to their needs. Shopping and buying the camera with the best specs can lead to serious disappointment. Everyone's needs are going to be different but you could establish some categories such as, low light, high contrast, etc. that might be helpful. But the difficulty with this method is how the comparison is done. For example, one camera might do good in low light situations right out of the box, with no user adjustment. But another camera gives only fair results in low light, right out of the box. But if the second camera is adjusted properly it gives a superior low light image to the first camera. So how do you evaluate and rate those cameras? Do you test and rate them as if you're a novice user and know little about your camera (having never read the manual) or do you test them setup and tweaked to the max? If you test and rate the tweaked version the novice may be disappointed that he can't get the results the testers get.

Many cameras today are so very close in performance that the differences don't matter, or can easily be adjusted via in camera menus or corrected in post. Then how should a camera be picked? At the very least Bryan and Frank's method will beat any chart of performance specs. But if you ask any of the really experienced shooters, Chris Hurd, Charles Pappert, Wayne Orr, Bill Pryor, many of this forums wranglers and dozens of others here I've missed, about how to pick a camera, they'll tell you to get to a big show or convention. Go to NAB, CES, DV East, DV West, PMA, state PPA conventions and the dozens of state and regional shows and compare models. Pick up the cameras and play with them. That's what they are there for. You wouldn't buy a car without a test drive, why would you buy a camera without shooting with it?

I know many people's arguments are they live in a rural area and can't make it to the big shows in the east or west. Then consider many of the state and regional shows sponsored by organizations such as the PP of A, or various wedding organizations. You won't find Sony or Canon there but the bigger regional dealers will be there with cameras for you to compare. If you're considering a bigger ticket camera, say $3,000 and up then you really need to go to a bigger show, if you can't find the cameras locally. It's a couple hundred dollars for a plane ticket and another hundred or so for a room and meals. Compare that to spending $4000 or $5000 on the wrong camera. The money spent may delay your purchase a month or two, but nothing like the frustration you'll feel when you discover that you spent $4000 on the wrong camera.

Ben Wiens January 11th, 2004 12:57 PM

Why I often rely on only test results
 
I totally support Jeff Donald's suggestion that hands on tests are very desirable. Unfortunately it's not always practical in my experience. When I was in the market for my first camcorder recently, the camera store I had insurance credit with did not have any of the models I was interested in on display. Actually nether did most of the other stores. In my experience, when I go to trade shows, they just happen to not have the particular model I am interested in. As I live in Canada, attending tradeshows in the US is also bloody expensive, we have to pay for things at our 2nd tier world dollars. That is why I think tests, and user feedback on forums such as this are highly desirable, even though I agree we should make every attempt to test these things in person.

Michael Wisniewski January 11th, 2004 05:54 PM

Quote:

Originally posted by Bryan Beasleigh : Michael, you'll drive yourself nuts. It's easier said then done.
  • You're right, but I figure it's the lesser of two weevils.

    Either way, it's driving me nuts.


Michael Wisniewski January 11th, 2004 06:52 PM

The tests I'm proposing are just supposed to be part of the process of buying a camcorder. I'm assuming people will do their own hands on tests **and there's not much we could do if they didn't

Right now though the process can be so subjective - ask about low light and you'll get 10 different answers. A pro will say you "have" to go with the VX2000 if you want low light, and then an amateur will say the Optura Xi's low light is "great" ... and they'll both be right.

So I'd love to have comparable video tests so we can decide what's acceptable. People could also run the tests on their own camcorder to see how it compares.

FYI: The results would be a video file/s taken in similar lighting condtions using similar settings (auto or manual). Anyway, it looks like I'm going to have to do more research and come up some suggestions before this proceeds further.



Michael Wisniewski January 11th, 2004 07:22 PM

Frank - I'll ask B&H Photo about the "drop" test ;-p

I was also thinking of a spill, smudge, little dirty fingers, big fat finger, and cold wet nose test.


All times are GMT -6. The time now is 11:46 PM.

DV Info Net -- Real Names, Real People, Real Info!
1998-2024 The Digital Video Information Network