dpreview Test

[t024484]

Advertisement (gone after registration)

Like I said before, I liked the testing with the coloured resolution sheets, not the test itself.

I am still looking forward to see the same test in raw with UV/IR filter, against top contenders from C&N.

First thing I will try to do, is to repeat the tests with my own M8 and see how it differs from dpreview.

I have the original b&w resolution chart. It should be possible to Photoshop it in the various colours, after having shot it in small parts and merged them together

I will probably have to print it in a much larger format, to keep the resolution of the printed chart as high as the photographic original. Because of that, the test pictures will only have to be taken from a greater distance.

 

Hans

[jaapv]

Good idea, Hans. We'll see if your efforts bring a blush of shame to DpReview tester's cheeks

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[tompkins135]

the M8 was doing very well against a 10 Mb D60 and a 14 Mb Foveon sensor

 

But the Foveon eqiupped camera costs about $750 with it's 28mm lens and the Nikon D60 roughly the same, how much for an M8 similarly equipped.. A mere snip at around $8000!

[marknorton]

The use of jpegs was confirmed in another thread on this subject. We know that Leica's s/w engineers might need to take a look into the M8 jpeg, but the fact that the test used jpeg's makes it pretty unreliable IMHO anyway

 

As Jaap says, if this is a test to differentiate sensors, it disappoints because the variation in JPEG quality between cameras will mask what's important. I'd reserve judgment until I could see properly exposed and processed DNGs but test results like these, taken at face value, do Leica no favours.

 

That said, we know Leica JPEG quality is not especially good and we've covered before the simplicity which would come from making it a DNG only camera with just a rough JPEG for chimping. After all, no-one can complain about JPEG quality if you don't actually produce any.

 

I expect Leica is up against a wall with a shortage of CPU cycles - just not enough of them to do more to improve the JPEG; up the clock speed and you hit battery life.

[jaapv]

That is the next upgrade then: New motherboard, new CPU, extra firmware, higher capacity battery...:D

[scott kirkpatrick]

Look at the sensor comparison issues from DPReview's point of view for a moment -- they've already bought themselves a lot of work, and concluded for their audiences that the extra step of standardizing the raw --> jpeg transformation wasn't worth the further effort. And if they did, say, run everything through Lightroom or ACR only, (which I gather can't handle Foveon, the subject of that particular review), there would be howls of unfairness from those who feel that only some other tool, Aperture for example, properly reflects the qualities of their favorite image box.

 

scott

[jaapv]

Advertisement (gone after registration)

Look at the sensor comparison issues from DPReview's point of view for a moment -- they've already bought themselves a lot of work, and concluded for their audiences that the extra step of standardizing the raw --> jpeg transformation wasn't worth the further effort. And if they did, say, run everything through Lightroom or ACR only, (which I gather can't handle Foveon, the subject of that particular review), there would be howls of unfairness from those who feel that only some other tool, Aperture for example, properly reflects the qualities of their favorite image box.

 

scott

 

Hmm. I see what you mean, Scott, but I do disagree. In my book, if it is "good enough", it isn't good enough. If you can't or won't do a valid test, don't publicize it with the voice of authority.

[Guest jimmy pro]

If you can't or won't do a valid test, don't publicize it with the voice of authority.

 

Ah but that'd shut down 85% of the internet

 

To periphrase Groucho Marks, I have nothing but confidence in dpreview....and not very much of that either.

[jaapv]

Ah but that'd shut down 85% of the internet

 

To periphrase Groucho Marks, I have nothing but confidence in dpreview....and not very much of that either.

 

:D

[AlanG]

Like I said before, I liked the testing with the coloured resolution sheets, not the test itself.

I am still looking forward to see the same test in raw with UV/IR filter, against top contenders from C&N.

First thing I will try to do, is to repeat the tests with my own M8 and see how it differs from dpreview.

I have the original b&w resolution chart. It should be possible to Photoshop it in the various colours, after having shot it in small parts and merged them together

I will probably have to print it in a much larger format, to keep the resolution of the printed chart as high as the photographic original. Because of that, the test pictures will only have to be taken from a greater distance.

 

Hans

 

This should be interesting. Consider first simply comparing red/blue resolution on jpegs vs/raw. If that doesn't change the results, the DPReview tests will be valid.

 

The IR issue in respect to this test is irrelevant because the amount of light in general photography is many times greater than the typical amount of IR radiation on the scene. With daylight simulating fluorescent lamps, IR is virtually nil.

[AlanG]

Hmm. I see what you mean, Scott, but I do disagree. In my book, if it is "good enough", it isn't good enough. If you can't or won't do a valid test, don't publicize it with the voice of authority.

 

It think it is a valid test. They are testing the jpegs. They never said they were testing the raw files. It is not DPReview's fault if one brand's in-camera jpegs are not processed very well.

 

When DPReview reviewed the M8 they did compare in camera jpeg vs. raw, and the raw resolution test photo showed more detail. I don't know how to extrapolate this to the red/blue test, but I bet the M8 will do better on that test in raw than it did in jpeg.

 

Leica M8 Review: 15. Photographic tests:

 

Leica M8 Review: 18. Compared to...:

[diogenis]

Alan,

Their test must be valid, otherwise they risk a lot of their credibility.

The fishy tactic is still there though, in how on earth do they even consider comparing a consumer digital camera with a professional one like the Leica. It's like comparing a Fiat Punto and a Ferrari, cheering "heeey, look at fuel consumption!!! "This, will ultimately lead some to believe that no matter what you buy, both cameras are identical, bar the price of course..

 

..And for Leica this is a lesson to finally improve its jpeg routines.

[ho_co]

... They did not us an IR filter, thus there was no extra piece of glass to affect the optical path on the Leica. The fluorescent photo lights they use do not put out IR. Besides, even if IR could affect the image focus, the red channel would show blurring of the black and white test target. But it didn't.

...

They use those lenses at f9 because they are trying to compare the cameras and not the lenses. ...

Alan--

Again, good research, but please post a link for this information. It's meaningless if we can't verify it.

 

Again, how do you set a 50 Summicron to f/9? The half-stop between f/8 and f/11 is f/10 (actually f/9.8) by my calculation. (Using the 1/3 stop detents of a Zeiss lens, they'd come up with f/9.2 and f/10.3).

 

Considering the colors in the test charts, they're pretty, but they're not standard. I read few lens tests because most seem (like this one) to be mere demonstrations of "Look at what we can do." Is moving to color charts a general trend? I thought b/w lens charts saw all the colors the imaging substrate could see? What is changed? (I'm not saying dpreview isn't onto something, just that I don't know what it is.)

 

dpreview doesn't show the vertical or 45° resolution test bands either, though a lens often differs in the three directions and sensors always do as I understand the matter.

The IR issue in respect to this test is irrelevant because the amount of light in general photography is many times greater than the typical amount of IR radiation on the scene. With daylight simulating fluorescent lamps, IR is virtually nil.

Arguable. Regarding IR in 'general photography,' see the LFI articles a few months back on determining best visual focus with IR-pass filter. We also know the M8 is rather UV-sensitive, so UV should be eliminated as well. Is it also 'virtually nil' with these lamps? (I assume you mean "daylight-simulating fluorescent lamps," since daylight doesn't simulate fluorescent lamps. ) In general, we need to be careful with generalizations: Since we can't see it, how do we know how much IR is present?

... They are testing the jpegs. They never said they were testing the raw files. It is not DPReview's fault if one brand's in-camera jpegs are not processed very well. ...

Good point. If they started RAWs, we'd be having this discussion over which RAW processor they used. But we also know that Leica has made specific choices in regard to their JPG colors (see KammaGamma Articles Leica M8 Colors).

 

Their test must be valid, otherwise they risk a lot of their credibility.

As long as people see them as credible, they're happy. Some people here accept the test, others don't.

 

 

IMHO, the test is completely invalid. It tests what it tests and shows what it shows, but can't lead to conclusions about the equipment tested.

 

As I said above, how good are your M8 files? Are the dpreview tests illustrative of your results? QED.

[AlanG]

Alan,

Their test must be valid, otherwise they risk a lot of their credibility.

The fishy tactic is still there though, in how on earth do they even consider comparing a consumer digital camera with a professional one like the Leica. It's like comparing a Fiat Punto and a Ferrari, cheering "heeey, look at fuel consumption!!! "This, will ultimately lead some to believe that no matter what you buy, both cameras are identical, bar the price of course...

 

You are entitled to your opinion, but I didn't form such a conclusion from reading the review. Anyone considering either camera should at least objectively study the full review of that model. They only compared the DP-1 with the M8, D40, D60 on this one aspect.

 

The main part of the DP-1 review compared it to the Canon 450, the Nikon D60 and the Ricoh GRII. The full review of the M8 compared it with the 5D, D200 and LC-1. And even then, the comparisons are limited to these imaging tests, and are not overall comparisons.

 

Additionally, it is my impression that a number of the people considering the DP-1 do want to see how it compares with an M8.

[AlanG]

Alan--

Again, good research, but please post a link for this information. It's meaningless if we can't verify it.

 

Again, how do you set a 50 Summicron to f/9? The half-stop between f/8 and f/11 is f/10 (actually f/9.8) by my calculation. (Using the 1/3 stop detents of a Zeiss lens, they'd come up with f/9.2 and f/10.3)...

 

I really think you are splitting hairs with the f 9 vs. f9.2, etc. Virtually nothing in photography is that accurate. Nor does it need to be for a valid test. They just had to come close. But perhaps they have a way to measure it. I bet that whether the Leica lens was set anywhere between f8-f11 it would out resolve the sensor. Are you saying that a Nikon 50 f1.8 will out resolve a 50mm Summicron anywhere between f8-f11?

 

"...dpreview doesn't show the vertical or 45° resolution test bands either, though a lens often differs in the three directions and sensors always do as I understand the matter."

 

The full size charts show this. You can download them and look at them. You can split the r,g,b channels and study that also.

 

I don't think I can put any more time into this. You ought to read this thread and if that doesn't help, contact DPReview and ask them directly:

 

http://www.l-camera-forum.com/leica-forum/leica-m8-forum/54228-dp1-review-dcr-compares-leica-m8.html

[ho_co]

I really think you are splitting hairs with the f 9 vs. f9.2, etc. ...

I said you can't set a Summicron to f/9, but you can set it to f/10. If dpreview says they can set it to f/9, I'd like to know how.

... I don't think I can put any more time into this. ...

Nor I. If you find the test meaningful, good for you. I've simply stated some reasons I don't.

 

Thanks for the link to the earlier thread.

[ho_co]

If the dpreview test were comparing lenses, we would ask that equivalent focal lengths be used. Perhaps that isn’t necessary since we are supposedly comparing sensors.

 

Since the M8’s sensor is more sensitive to IR (and possibly to UV) than the others in the test, the only way to compare sensors fairly is to use a UV/IR cut filter on the M8.

 

If the “daylight fluorescent” tubes used by dpreview are supposed to emit neither UV nor IR, I would want to know the brand and model of the tubes in question, and to see a spectral diagram of their output to verify the claim.

 

Below is Phillip Askey's response to my question of lighting and IR reflectivity.

 

John

Not sure what you're driving at but the M8's IR sensitivity is only an issue with certain man-made materials, the color test chart doesn't reflect enough IR to be an issue. The IR filter simply would have made NO difference in this test. We used our normal daylight simulation softboxes which use high CRI bulbs at about 5000 Kelvin.

Does that satisfy?

 

He is wrong in saying that “IR sensitivity is only an issue with certain man-made materials.”

 

Definitions I find of high-CRI bulbs don’t say anything about IR, and specify that UV must be present. We know both of these affect the M8 sensor.

 

In other words, the people who performed the test aren't well informed about the M8 and we still don't know anything about the IR spectrum of their lamps. Solution: Use the UV/IR filter. Leica feels it is important enough to give away two with every camera sold.

 

 

On my monitor, in the yellow-blue, yellow-magenta, magenta-green and black-white M8 samples shown, the M8 frequency lines are still well resolved at the left of the swatch. That is, the area shown still had room to move to the left. I see that Michael Hußmann pointed this out as well in the earlier thread.

 

Michael also refutes the argument that “It’s due to JPG,” and raises serious questions about the matter of IR. Nonetheless, I’d like to see results with the filter.

 

 

The performance of the DP1 in this test is excellent, that of the M8 very good and that of the D60 certainly in the neighborhood of the M8’s, and possibly better. From this comparison, I think the most we can say is that all three cameras are in the same league for the type of test done. And this set of tests isn’t representative of how I use a camera.

 

That is, the test compares apples and oranges. Strictly numeric tests measure what they are set up to measure, but the tests must be meaningful. I find the most useful part of dpreview’s tests to be the tester’s summary, because there the experience of the user comes into play.

 

As many here and in the earlier thread have said, these tests don’t accurately represent the M8. It would be nice to know why.

[jaapv]

Arguable. Regarding IR in 'general photography,' see the LFI articles a few months back on determining best visual focus with IR-pass filter. We also know the M8 is rather UV-sensitive, so UV should be eliminated as well. Is it also 'virtually nil' with these lamps? (I assume you mean "daylight-simulating fluorescent lamps," since daylight doesn't simulate fluorescent lamps. ) In general, we need to be careful with generalizations.

 

UV is not present; The Summicron does not transmit UV by design.

[ho_co]

Thanks, Jaap.

[Advertisement]

Hallo,

Look here: Leica M

[diogenis]

You are entitled to your opinion, but I didn't form such a conclusion from reading the review. Anyone considering either camera should at least objectively study the full review of that model. They only compared the DP-1 with the M8, D40, D60 on this one aspect.

 

The main part of the DP-1 review compared it to the Canon 450, the Nikon D60 and the Ricoh GRII. The full review of the M8 compared it with the 5D, D200 and LC-1. And even then, the comparisons are limited to these imaging tests, and are not overall comparisons.

 

Additionally, it is my impression that a number of the people considering the DP-1 do want to see how it compares with an M8.

 

DP tried a test out of the ordinary, which was probably led somehow by Sigma to demonstrate "ultra sharp, detailed images with full color single pixel resolution" as they said.

Result of this test is that "The DP1 produces a very good resolution (for its pixel count) on our standard black and white resolution test chart but it only becomes apparent on the color test chart how well the camera resolves color detail"

Next finding is that the M8 fails completely with the blue/red test and some(?) others due to blurring,

and their conclusion is that they want a sensor following Sigma's design of 3 layers only with say, 10mp per layer now...

Well I don't like how m8's inconsistency is shown in this test as there is no explanation given, and also, there is no connection given of this test with reality, so everyone can form their own conclusions... Go with the DP1! yaay...