DXO Review

[mjh]

Advertisement (gone after registration)

So who is going to explain to me what 68.6 actually means in practice? Or is it just another number which has technical meaning but no correlation with the real world:D.

The individual measurements are usually valid and the results are interesting, even when they may not adequately reflect the quality you see in an image. But when you aggregate all the various measurements to a single number it becomes largely meaningless.

[pgk]

I think that that a realistic assessment of DxO numbers is that they appear to be accurate for the specific things they measure, but they don't measure everything.

You are probably right, but, I used to measure MTF curves for lenses. The problem was always explaining what they meant. As a simplistic option imagine simply ascribing a single number as a overview of a lenses' MTF curve (say based on the area under the curve - bigger is better). Would this actually tell you anything about the performance:confused:?

[zlatkob]

This may be true in some cases e.g. noise but can you really see it all the time, with your eyes? Or only in lab conditions?

 

Well, not all of the time, but in an 8x10" or A4 sized print of an ISO 2500 photo, for example. A DxO Low-Light ISO score of 900 vs. 1800 will be noticeable with your eyes. On the other hand, it is a technical measurement of a technical parameter, so it may not make any difference for the meaning or impact of the photo.

[adan]

Michael, I think by "smeared RAW" Jerry is referring to the fact the M9 applies in-camera noise-reduction, as DxO (and Sean Reid) both have noted. Pixel-peeping, I see small areas in M9 images that are "bald" - nothing there to sharpen even though adjoining areas have detail and/or noise that sharpens up nicely.

 

Image: 200% pixel peep of M9 ISO 800 image, NO noise reduction applied in post-processing - there are areas where the luminance has been so smoothed that there is nothing for sharpening algorithms to work on. I hope the jpegging process doesn't make it invisible.

 

I'd love it if Leica would add control of this NR to the menu - sometimes smooth and noiseless is good - sometimes I'd rather turn it off and get maximum detail everywhere even if the noise increases.

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

[stunsworth]

Andy, I thought all digital cameras applied noise reduction to RAW images.

[adan]

Well, the M9 appears to apply more than the M8 - and the top-end SLRs usually have a control in the menus to turn it off, or at least set it lower or higher.

 

As is the case with the Nikon D3s Nikon D3S Review: 10. Menus: Digital Photography Review - scroll down in the list of shooting menus to find:

 

High ISO NR:

High

Normal

Low

Off (default setting)

 

or the Canon 1DmkIV: Scoll down to Custom Function II-2: http://www.dpreview.com/reviews/canoneos1dmarkIV/page12.asp

 

High ISO speed Noise reduction:

Standard

Low

Strong

Disable

 

BTW: fans of CMOS sensors might want to check these settings, try ISO 3200 with NR tuned "off" - and see what their CMOS sensor delivers "naked". It might be educational.

[snaggs]

Advertisement (gone after registration)

and the top-end SLRs usually have a control in the menus to turn it off, or at least set it lower or higher.

 

BTW: fans of CMOS sensors might want to check these settings, try ISO 3200 with NR tuned "off" - and see what their CMOS sensor delivers "naked". It might be educational.

 

All CMOS sensors have noise control built in, the menu noise control is additional software based filters. Thats part of what the DIGIC III chips etc do. The advantage of CMOS is the data is read per pixel allowing more flexibility in this base level noise control.

 

OFF, is never OFF.

 

Daniel.

[Jamie Roberts]

{snipped}

M9 colors (LR RAW) are shifted. Even when I set white ballance carefully and manually. They may be more pleasing to viewer, no doubts. Like more contrast photo is more pleasing to many people too, but it doesn't mean it is natural.

 

{snipped}.

 

I wish everyone with CCD cameras (M8. M9, DMR and so on) would stop using LR till they figure out how to do colour.

 

And "natural colour?" Are we going to start that one? I'll tell you right now: there isn't a Nikon or Canon or Leica made right now that does "natural" colour.

 

I will say the Kodak sensors I've used in Leicas, with the right post processing, save me a lot of time getting skin tones and other colours correct compared with other cameras.

 

As for DxO--well, when they start talking about sensor DR (as opposed to system DR--which is all I care about) they always lose me and it's been a mystery to me--probably because they're just measuring the sensor and ignoring the rest of the image chain.

 

In the real wold, my M8 more than matches my 5d in terms of what I think of as dynamic range at ISO 640 and under (and that means about ISO 800 for the 5d). Yes, the 5d has a stop on the M8 at least. The D3 has two stops.

 

I'm sure, however, viewing even a large print at normal distances a D3s, an M9 a 5d2 would all look pretty darned similar at ISO 1000.

 

Anyway, while I can't prove this, I do think the way DxO measures DR, while perfectly valid as a test, doesn't tell the whole story.

 

But maybe Nikon's colour and DR at higher ISOs gets messed up AFTER the sensor?

[Jamie Roberts]

All CMOS sensors have noise control built in, the menu noise control is additional software based filters. Thats part of what the DIGIC III chips etc do. The advantage of CMOS is the data is read per pixel allowing more flexibility in this base level noise control.

 

OFF, is never OFF.

 

Daniel.

 

 

CCD vs. CMOS

 

In fact, on-board noise reduction may be a reason why DxO always measures CMOS noise lower; again, it may not be. I have no idea what DxO means by "we measure the sensor DR only" when it comes to CMOS sensors.

[mjh]

Michael, I think by "smeared RAW" Jerry is referring to the fact the M9 applies in-camera noise-reduction, as DxO (and Sean Reid) both have noted. Pixel-peeping, I see small areas in M9 images that are "bald" - nothing there to sharpen even though adjoining areas have detail and/or noise that sharpens up nicely.

The auto-correlation that DxO has found is very slight (0.1 or less it seems). To me it doesn’t even look like a serious attempt at noise reduction but more like it was the side-effect of something else. Frankly I don’t see how this result would explain the “bald” areas in the image.

[mjh]

I thought all digital cameras applied noise reduction to RAW images.

As a rule, cameras with CMOS sensors do; some basic noise reduction circuitry is usually even integrated onto the sensor. This low-level noise reduction cannot be turned off. But that doesn’t imply that resolution and sharpness are compromised. Most of the basic noise suppression is merely intended to compensate for the inherent non-uniformity of the CMOS sensor pixels and thus doesn’t introduce a correlation with neighbouring pixels.

[Printmaker]

I'm going to sell my M9 after this reading.:D

Cheers,

Ario

 

How much? Black or gun metal grey?

[Jamie Roberts]

As a rule, cameras with CMOS sensors do; some basic noise reduction circuitry is usually even integrated onto the sensor. This low-level noise reduction cannot be turned off. But that doesn’t imply that resolution and sharpness are compromised. Most of the basic noise suppression is merely intended to compensate for the inherent non-uniformity of the CMOS sensor pixels and thus doesn’t introduce a correlation with neighbouring pixels.

 

Michael, wouldn't the basic noise suppression increase the amount of DR read by DxO "off the sensor"? Wouldn't a CCD design typically optimize after the (analog?) signal is read off the chip? Is that why DxO tries to "interpret" the amount of NR the M9 is adding--to equal the playing field in some way?

 

Aside from doing a very basic (and perhaps apples to oranges) comparison, how could they compare the two results without always favouring CMOS chips? I see, for example, that the Nikon D3x has almost as much DR as the large Phase chips. Can that really be correct when it comes to real world use of the cameras?

 

I'm just curious... however, since even the DxO results essentially say the M9 has between 10 and 13 stops of DR across its ISO range, and I truly can't see needing more than that to print anyway, and I'm more convinced by the colour output from the Kodak CCDs, I don't think I'll cancel my M9 order....

[diogenis]

This is just a meaningless comparison, not a "review" but comparison between different cameras. It might show some (questionable) results when you compare dSLRs or RFs but not when you compare them together.

As an example: the D3x gets a better score than the D3s, so what? D3s is older than D3x?

The same absolute difference exists between M8 and M9, but the later costs almost twice as much and you can't even buy it. So what? Can you compare them?

And what about all the newcomers that eventually dump bulky dSLRs and convert to the RF concept. Are they wrong or what?

DxO did a sincere comparison, by checking the sensor of M9 and that is all. Fine by them. Let's just move on a bit shall we?

[pgk]

But when you aggregate all the various measurements to a single number it becomes largely meaningless.

My thoughts entirely Michael:D. I don't understand the attempt to deal with complex data by condensing it down into a far too simplistic single figure. Unfortunately doing this simply fuels the X is 'better' than Y debates that rage all too often here.

[mjh]

Michael, wouldn't the basic noise suppression increase the amount of DR read by DxO "off the sensor"? Wouldn't a CCD design typically optimize after the (analog?) signal is read off the chip? Is that why DxO tries to "interpret" the amount of NR the M9 is adding--to equal the playing field in some way?

What DxO is actually reviewing is the performance of sensor pixels as read out by a certain camera. Their measurements cover the basic sensor performance, plus stuff like the read-out clock speed, signal amplification, and quantization. They don’t care about resolution, internal image processing, metering, white balance, and all the other things that would contribute to the overall camera performance. Whenever their measurements based on the data in raw files do not reflect the actual performance of the sensor pixels, they try to reconstruct the original data, as explained in the “raw half-cooked” article on their website. This doesn’t remove all the noise-suppression applied, though. The output of CMOS sensors would normally be marred with a lot of fixed-pattern noise that needs to be removed as early in the process as possible (by the sensor itself), and the effects of this kind of noise reduction will not be stripped from the results by DxO. Fixed-pattern noise, while not necessarily being white noise, can usually be removed without resorting to averaging of neighbouring pixels and its removal doesn’t introduce correlations between pixels. DxO’s auto-correlation test wouldn’t reveal this kind of noise reduction taking place, even when it obviously is.

 

But is it fair? One might argue that it is: CMOS sensors wouldn’t stand a chance in such tests otherwise, and as long as the removal of fixed-pattern noise doesn’t compromise resolution, it should not be held against the sensor. Still, CMOS sensors are at a slight advantage in these reviews, especially since the removal of fixed-pattern noise, even when it doesn’t reduce the resolution in the spatial domain, does reduce the resolution of tonal values.

[preston]

The DxO numbers are meaningful, not just technical. If the M9 produced higher DxO numbers, we could SEE the practical effects as lower noise in high ISO photos and as greater dynamic range. And if that were to happen, everyone would congratulate Leica for achieving such great DxO results.

I would tend to agree, but in practice I've had trouble finding a clear correspondence between DxO dynamic range results and my photographic testing.

 

In one test I shot a very wide-range subject (dark interior plus bright window view) with a 5D2. I shot a matched series at ISO 1600, 800, and 400, with the exposure set to barely hold the highlights in the raw files (using Camera Raw).

 

Based on the DxO charts, I was looking to see the benefits of a couple of stops of extra dynamic range in the shadows in the 400 image. But in practice, looking into the deep shadows I could see only a little more apparent captured tonal range than in the 1600 file.

 

Of course the whole 400 image was cleaner, and yes, the shadows looked much less noisy. But in terms of how dark a portion of the scene could be recorded by the camera such that I could see tone separation, I couldn't see all that much difference.

 

It's conceivable that flare from the lens (28mm f1.8, at f8) cut the shadow range. But I am guessing that part of the issue may be the correlation of the signal-to-noise measurement definition with human visual perception. When I think "greater dynamic range" I expect to see tonal separation in deeper shadows (or highlights, if the shadows are held constant). But he DxO definition of "greater dynamic range" might instead be more like "less noise in the deep shadows", not "deeper shadows". They are both good but they don't look the same the eye.

[Jamie Roberts]

What DxO is actually reviewing is the performance of sensor pixels as read out by a certain camera. Their measurements cover the basic sensor performance, plus stuff like the read-out clock speed, signal amplification, and quantization. They don’t care about resolution, internal image processing, metering, white balance, and all the other things that would contribute to the overall camera performance. Whenever their measurements based on the data in raw files do not reflect the actual performance of the sensor pixels, they try to reconstruct the original data, as explained in the “raw half-cooked” article on their website. This doesn’t remove all the noise-suppression applied, though. The output of CMOS sensors would normally be marred with a lot of fixed-pattern noise that needs to be removed as early in the process as possible (by the sensor itself), and the effects of this kind of noise reduction will not be stripped from the results by DxO. Fixed-pattern noise, while not necessarily being white noise, can usually be removed without resorting to averaging of neighbouring pixels and its removal doesn’t introduce correlations between pixels. DxO’s auto-correlation test wouldn’t reveal this kind of noise reduction taking place, even when it obviously is.

 

But is it fair? One might argue that it is: CMOS sensors wouldn’t stand a chance in such tests otherwise, and as long as the removal of fixed-pattern noise doesn’t compromise resolution, it should not be held against the sensor. Still, CMOS sensors are at a slight advantage in these reviews, especially since the removal of fixed-pattern noise, even when it doesn’t reduce the resolution in the spatial domain, does reduce the resolution of tonal values.

 

Thanks Michael--that's exactly what I was looking for! Cheers!

[diogenis]

DxO review reminds me of that TV show: "does it blend?" Where some dudes add in a powerful blender gadgets and time them until they blend

Actually I would love to see a comparison between a Canon, Nikon and Leica in the blender.

 

So, Does it blend?

[Advertisement]

Hallo,

Look here: Leica M

[AlanG]

DxO review reminds me of that TV show: "does it blend?" Where some dudes add in a powerful blender gadgets and time them until they blend

Actually I would love to see a comparison between a Canon, Nikon and Leica in the blender.

 

So, Does it blend?

 

Perhaps some here don't fully understand what DXO is actually testing for.

 

This is their explanation:

 

DxOMark Sensor metrics and photospace

 

it is up to you to determine if any of this will matter in your photographs. For instance, I am happy with a lot of images I get with a cheap p&s at ISO 400.