dpreview Test

[ho_co]

Advertisement (gone after registration)

Gee, I thought Alan and I were the only ones left reading this thread. I'm glad to see there are still others as well!

 

Alan:

1) You say "I have nothing against the test being repeated with an IR filter if you think it will make a difference. Is that what you are saying?"

 

No, I'm not saying "reshoot with IR-cut filter because I think it will make a difference." I'm saying we won't know whether it makes a difference until we shoot with filter.

 

2) Thanks for getting more info on CRI. The data you show go exactly to my point: CRI is based on visual representation of a small number of specific colors. That's a great way to go for human eyes, and that's the purpose of the system--these people want to sell us home illumination. But notice that on all the diagrams you reproduce, the red end of the visible spectrum isn't nearing zero. That is, all the diagrams imply a fair IR component.

 

There's no reason for a CRI definition to include IR specification, since that's outside our visual capability.

 

But for an organization doing a test with an IR-sensitive camera NOT to take the possible IR content of its light source into account is dumb. And for that organization then to announce that IR would have made no difference because "the M8's IR sensitivity is only an issue with certain man-made materials" is inane. (And as blakley repeated, we still haven't any information on the lights dpreview actually used.)

 

Alan, I'm glad for the information that you and others have assembled here. You're helping us progress toward understanding the issues involved.

 

We've seen good arguments here and in the previous thread that what we see as disappointing results from the M8 are not likely due to IR, internal image processing, or the use of JPG in the test. But unless we can control for each of those factors in turn, we have no way to know why dpreview got these results.

 

I've got no stake in the matter: To me, this kind of pseudo-scientific test is useless except in its broad summary, viz, "on a scale of 1 to 6, I personally rate this camera X."

 

If we're curious why the M8 didn't show better than it did, we need facts, not speculation.

 

For example, someone could hire out the dpreview studio for a day and shoot the same charts with dpreview's lights and in daylight, with and without IR filter, JPG and RAW, with a second sample of the same lens they are using, both at the aperture they used and at a wider aperture, etc etc. (For completeness' sake, the test should include use of older lamps and new ones, and be repeated in all combinations of lens, aperture, filter etc.)

 

And no, I don't think that's going to happen.

 

>>We need solid information before we can get anywhere with speculation.<< And I think we're all getting better informed.

 

Once again: I'm curious how dpreview got these results, but I don't worry about it. They got those results and I'd like to know why, but the M8 is a superb performer in most people's experience, including my own. And I'm happy with that.

[ho_co]

BTW: just another couple niggles.

 

1) They used a Summicron 50, right? Which version? It shouldn't make any difference at the aperture they were using, but lenses of different generations behave differently. The same holds for lenses of other brands, I'm sure. Were the tests performed, for example, with a 1956 Summicron and a 2001 Nikkor, or the other way around? Possibly comparing apples and oranges?

 

Correct me if I'm wrong, but most Japanese 50/1.8 lenses today are generally regarded as not performing as well as their 50/1.4 counterparts. At the aperture dpreview uses for these tests, that shouldn't make any difference, but I'd like to know more about why they chose to use the slower lens.

 

2) I'd also like to get more information on why they have chosen to stop the lenses down to the neighborhood of f/9. We know that the M8 tends to perform best up to about f/5.6. I doubt that's atypical, but I don't know. How did dpreview choose f/9? Even the number is strange to my ear, but I'm probably older than the combined ages of all their competent people. I think most of us would speak of f/5.6 or f/8 or f/11, but few of us IMHO would say, "Now for this shot I really feel f/9 is the correct aperture for the mood I want."

 

3) They used JPG, right? How were the processing parameters set on the various cameras? A number of testers feel Nikon's default settings are somewhat oversharpened. Leica chooses a lower contrast than others.

 

That is, were all the cameras set to their default sharpness, saturation, contrast settings? That's logical, but not fair. Instead IMHO, one should adjust all the cameras to give comparable results in some given situation before running the test. But that would be time-consuming and in some cases next to impossible since not all manufacturers offer the same range of adjustments. For example, on the D200 Nikon offers settings for "sharpness, contrast, color reproduction, saturation, and hue" (D200 instructions, p 45); and that is in addition to choice of color spaces sRGB or Adobe RGB; choice of Color Modes I, II or III (which are never explained to my satisfaction); et al.

 

 

These aren't complaints. I'm just pointing out that there is a wide range of variables involved in a test like this. None of these points will alter the outcome of the test we're talking about, but they do bear on the fact that there are too many loose ends to consider the dpreview tests definitive at any level of detail. I respect their conclusions, but I doubt there is much rigor in the individual repetitive tests.

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[fotografr]

Gee, I thought Alan and I were the only ones left reading this thread. I'm glad to see there are still others as well!

 

 

Well, there's one other anyway. The rest of us nodded off during the math segment.

[ho_co]

<g>

[AlanG]

 

These aren't complaints. I'm just pointing out that there is a wide range of variables involved in a test like this. None of these points will alter the outcome of the test we're talking about, but they do bear on the fact that there are too many loose ends to consider the dpreview tests definitive at any level of detail. I respect their conclusions, but I doubt there is much rigor in the individual repetitive tests.

 

Most (not all) of your questions about their methodolgy are answered in the reviews themselves. (See the M8 review from July 2007, the recent DP-1 and Canon 450 reviews.) e.g:

 

"For direct comparisons we always use sharp prime lenses stopped down, typically to F9 for 35 mm lenses. Here we have used the Nikkor 50 mm F1.8 (it's sharper than the F1.4 at F9) on the D60 and D40, the Canon EF 50 mm F1.4 on the EOS 450D and the Summicron-M 50 mm F2.0 on the Leica M8."

 

"Sharpness and Detail

I won't linger on the M8's internal JPEG engine as I will be talking about that a little more later in this review. Sufficient to say that you can see the significant improvement shooting RAW and using either Capture One LE or Adobe Camera RAW to convert the images. As noted above the optimum conversion (for clean low ISO images) is to set Noise Suppression to zero which delivers very good detail and texture, arguably better than Adobe Camera RAW, which to be fair does produce good results in 'default' mode. "

 

Leica M8 Review: 12. Software:

[ho_co]

Alan, if you feel those citations answer my questions, good. I'm not going to run through my points again to see how many are answered, but at a glance I'd say maybe one question is answered, and it's a derivative one. On that count, I'm not sure, but isn't "sharper" a subjective term in optics, not a technically defined one?

 

 

I said before that if you like the dpreview reviews, I'm happy for you.

 

And I think all I've done since then (besides getting some math instruction) is point out the level at which I think the dpreview reviews are competent, and why I question them at levels deeper than that.

[AlanG]

Advertisement (gone after registration)

Alan, if you feel those citations answer my questions, good. I'm not going to run through my points again to see how many are answered, but at a glance I'd say maybe one question is answered, and it's a derivative one...

 

I suggested that if you thoroughly re-read the reviews, some of your questions may be answered to your satisfaction. I know I can't do that for you.