DXO Mark Leica M10 Score

[jaapv]

Advertisement (gone after registration)

You still don't get it - most of what is perceived as " noise performance"  has nothing to do with the sensor and everything with the level of noise reduction in the camera firmware chosen by the camera maker. For instance, Nikons will often have "better" noise performance than Sony cameras - with an identical sensor. Just because Sony chooses a different compromise. Leica chooses to perform as little noise reduction as possible, leaving it to the user in post-processing.

It all has very little to do with "sensor quality" or " cutting edge" The sensors themselves are mostly limited by the properties of light itself nowadays - and often identical in differently branded cameras.

 

This means that High-ISO noise cannot be a useful parameter for judging a sensor. It is very useful for marketing purposes, though...

 

I agree that it is indeed a sad situation that those dimly lit churches did not get photographed properly before the advent of the newest generation of sensors, due to the miserable ISO performance of film and older digital cameras...

 

I'll repeat it here: except for specialists, and they know to choose their tools and are not hung up on brand, bad light makes for a bad photograph in the vast majority of cases.

[Luke_Miller]

........ bad light makes for a bad photograph in the vast majority of cases.

 

That has been my experience.  My Nikon D4 has near state-of-the-art high ISO performance, but in very low light I still add flash.

[Hannes Lummes]

You still don't get it - most of what is perceived as " noise performance"  has nothing to do with the sensor and everything with the level of noise reduction in the camera firmware chosen by the camera maker. For instance, Nikons will often have "better" noise performance than Sony cameras - with an identical sensor. Just because Sony chooses a different compromise. Leica chooses to perform as little noise reduction as possible, leaving it to the user in post-processing.

It all has very little to do with "sensor quality" or " cutting edge" The sensors themselves are mostly limited by the properties of light itself nowadays - and often identical in differently branded cameras.

 

This means that High-ISO noise cannot be a useful parameter for judging a sensor. It is very useful for marketing purposes, though...

 

I agree that it is indeed a sad situation that those dimly lit churches did not get photographed properly before the advent of the newest generation of sensors, due to the miserable ISO performance of film and older digital cameras...

 

I'll repeat it here: except for specialists, and they know to choose their tools and are not hung up on brand, bad light makes for a bad photograph in the vast majority of cases.

Note Jaap that this was my first post in this thread, so "you still don't understand" is quite funny.

 

I actually understand very well the meaning of in camera noise reduction, and therefore always very carefully compare what kind of information is available in low key parts in a RAW file or picture. If you paid any attention to the pictures posted to this thread you could see that Sony was able to register the patterns of the wall in the high ISO examples whereas Leica was not.

 

You resorting to sarcasm is unnecessary and not constructive. I don't carry a tripod and in many places it is not even allowed. Other people do. And did. I hope you feel smart and vindicated though.

 

If you don't need low light performance, that is fine. Implying that I simply don't understand, or am not skilled enough is just ad hominem. Please be a bit more analytical. I am fully capable of noticing that some of my photos with M240 are hard to edit because of the nasty banding and green cast in the shadows. There is simply not enough usable info always.

Could you at least answer to my very central question about the ray angle and sensor design problem. Would be interesting. If you know.

 

 

 

Edited January 5, 2018 by Hannes Lummes

[Chaemono]

55 Otus on the α7R III vs. Noctilux on the M10 now for color and overall rendering. I agree on the details on the wall in the Sony pictures above. Here, the Otus seems to blur some details on the back of blue radio in the out-of-focus area into oblivion.

 

Less compressed JPEGs here: https://www.smugmug.com/gallery/n-Jfdr66/

 

α7R III + 55 Otus out of the box (AWB by the camera, Adobe profile), slightly cropped

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

ISO 320 f/2.8 @1/100 sec.

 

 

M10 + Noctilux out of the box (AWB by the camera, Adobe profile), slightly cropped

ISO 320 f/2.8 @1/90 sec.

Edited January 6, 2018 by Chaemono

[Chaemono]

And now with WB adjusted to try to remove tint and to match.

 

Less compressed JPEGs here: https://www.smugmug.com/gallery/n-Jfdr66/

 

α7R III + 55 Otus slightly cropped, Exposure +0.35, Shadows +100, Clarity +10, Saturation +5

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

ISO 320 f/2.8 @1/100 sec.

 

 

M10 + Noctilux slightly cropped, Huelight Standard+ V160, Exposure +0.10, Shadows +100, Clarity +10, Saturation +2

ISO 320 f/2.8 @1/90 sec.

Edited January 6, 2018 by Chaemono

[Chaemono]

And now the usual suspect for color and with the shadows lifted by +100. The 55 Otus is the lens the α7R III sensor needs. The problem is it's a bit unwieldy  .

 

Less compressed JPEGs here: https://www.smugmug....llery/n-Jfdr66/

 

α7R III + 55 Otus from out of the box ARW (AWB by the camera, Adobe profile), slightly cropped

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

ISO 1600 f/2.8 @1/60 sec.

 

 

M10 + Noctilux from out of the box DNG (AWB by the camera, Adobe profile), slightly straightened and cropped

ISO 1600 f/2.8 @1/60 sec.

Edited January 6, 2018 by Chaemono

[Chaemono]

Advertisement (gone after registration)

Less compressed JPEGs here: https://www.smugmug....llery/n-Jfdr66/

 

α7R III, WB adjusted to try to remove tint and to match M10, Shadows +100, Clarity +10, NR +30, Sharpening +40

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

ISO 1600 f/2.8 @1/60 sec.

 

 

M10, Huelight Standard+ V160, reds and oranges slightly tweaked, WB adjusted, Shadows +100, Clarity +10, NR +30, Sharpening +40

ISO 1600 f/2.8 @1/60 sec.

[Chaemono]

α7R III + 55 Otus vs. M10 + Noctilux. Here one won't even be able to tell the difference unless one crops a lot. It's better to support this Sony combo by the lens when walking around in order to avoid bending the lens mount IMO. All Leica has to do next year with the SL2 is top the M10 sensor a bit and offer it with 30 MPx-plus and they are in business. It would be interesting to compare α7R III + 55 Otus vs. SL2 + 50 Summilux-SL next year.

 

Less compressed JPEGs here: https://www.smugmug.com/gallery/n-Jfdr66/

 

α7R III from ARW with AWB by the camera and Adobe profile, not touched in LR

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

ISO 200 f/2.8 @1/500 sec.

 

M10 from DNG with AWB by the camera and Adobe profile, not touched in LR

ISO 200 f/2.8 @1/500 sec.

[Chaemono]

And now processed to try to match. 

 

Less compressed JPEGs here: https://www.smugmug.com/gallery/n-Jfdr66/

 

α7R III

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

ISO 200 f/2.8 @1/500 sec.

 

 

M10

ISO 200 f/2.8 @1/500 sec.

[Rollin]

As Alex U. noted in #84, I'm surprised to see Leica maybe cooking the ISO rating on the M10. This has been known about Fuji for awhile - their ISO's are off about 1.5 stops from others. Fuji claims they use a different ISO measurement - cynics say it is a way for them to make their noise performance look better, the menu says ISO 3200, but really more like 1200, ya your noise will be less.

 

When looking on DXO I noticed the M10's ISO was lot less than others (screenshot below), and the SL did not show this so can not say Leica is using a different measurement too. Click on Measurements to see it in any of their comparisons, and when you look at the details on noise performance at SNR 18% tab and hover over the dot on the graph you'll see the M10's graph is worse than it looks at first glance. Today I did a quick check of this with my M10 and Sony a6300 (I sold all my FF Sony's to fund the M10 so all I have on hand) and found DXO was accurate finding M10 was about .5-.8 a stop under. I did it both using the same lens on both cameras and with equivalent focal lengths.

 

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

[adan]

Yep, or as I mentioned in Post #25: https://www.l-camera-forum.com/topic/279882-dxo-mark-leica-m10-score/?p=3419176

 

Except at ISO 100.

 

Worth knowing if one uses a hand-held meter or "sunny 16" guesstimation of exposure. OTOH some people shoot Tri-X at 200 or Fuji ACROS 100 at 50, so....

[Chaemono]

On the from ARW and from DNG just-opened-in-LR-and-not-touched Santa JPEGs above at ISO 1600 for both cameras but now with the 55 Otus on the Sony and not some crap lens that is not able to capture the available light properly, the M10 shows less noise in the out-of-focus areas than the α7R III to my eyes. I can post the crops and maybe I will. And we’ll do ISO 6400 and ISO 10000 for both cameras, post the crops here and a link to less compressed JPEGs and compare noise in the out-of-focus areas. I’ll do this over the next few weeks.

Edited January 7, 2018 by Chaemono

[pgk]

 I'm surprised to see Leica maybe cooking the ISO rating on the M10

Try looking at the digital ISO info here: https://en.wikipedia.org/wiki/Film_speed

 

Any deviation from known 'standards' could risk breaching 'trading standards' law in countries like the UK as it could be considered as misleading .....

[adan]

Interesting. DxOMark describes their testing methodology here, including for ISO. https://www.dxomark.com/dxomark-camera-sensor-testing-protocol-and-scores/

 

They use the Saturation-based speed standard. From pgk's link above: In rough outline, at what exposure the sensor will saturate and start to clip or blow highlights. More precisely, the ISO that will produce a saturation of 12.7% (pixel wells filled to 12.7% of capacity) when exposed to an 18% gray card - plus a factor of root-2 (¹/₂ stop) to allow headroom for specular highlights. And, BTW, assumes a ~3-stop brightness range above "gray-card-gray" 12.7% + 1 stop = 25%, plus 1 stop = 50%, plus 1 stop = 100% (saturated - highlights about to blow).

 

If I am reading all that correctly, then DxOmark is saying in effect that the M10 sensor has a stop (or so) less "headroom" than the other cameras compared - ISO "400" behaves as though you gave it an ISO 200 exposure and overfilled the pixels?

 

Now, another perfectly fine, ISO-approved "ISO speed" measurement standard is Noise-based speed. Which I won't try to explain technically, since I don't exactly understand its details myself. But among other things uses a subjective standard of what constitutes "excellent" or "acceptable" amounts of noise. More precisely, a S/N ratio of 40:1 is considered "excellent" and a S/N ratio of 10:1 is considered "acceptable," and while those S/N ratios can be objectively measured, the assignment of "excellent" or "acceptable" to those particular values was a subjective choice based on viewing images. YMMV

 

There is yet another approved means of determining ISO, which is more intuitive to photographers, but only applies to 8-bit files (i.e. jpegs, not raw/.dng). Standard output sensitivity. And that is, the speed which renders a gray card as 118 on the 0-255 8-bit tonal scale - in the sRGB color space (gamma 2.2). Set your camera to "400," expose for "400" and if a gray card comes out the correct brightness of 118 in sRGB, then the camera's 400 really is 400. Can't be used for raw because there is no fixed "mid-point brightness" that applies to all cameras or sensors, which may capture in 12 or 14 bits, or something else.

 

Of note: if the measured NBS ISO is higher than the SBS ISO, the noise-based ISO is the one that should be used, rounded down to the nearest standard ISO (1/3rd-stop, e.g. 160, 200, 250, 320, 400....).

 

(Which, by the way, is bad news for perfectionists who want to use the "exact" base ISO for max. DR, with zero "pull" or "push". The ISO says the speeds must be rounded to the nearest 1/3rd stop).

 

So it is altogether possible that Leica (legitimately) used the NBS (or even the SOS for jpegs) instead of the SBS, and got a different result than DxO. And labelled their ISO settings accordingly.

[Chaemono]

Quickly two after processing them to try to match. The M10 at a cooked Noctilux ISO 6400 and the α7R III at a cooked 55 Otus ISO 6400. Will post how they looked as DNG and ARW later but just on Exposure, Highlights, and Shadows adjustments: M10/α7R III Exposure +.15/+.15, Highlights -65/-60, Shadows +70/+70. The lenses make such a difference and the 55 Otus brings the best out of the Sony sensor. I'll do more but have to check the Sony mount first.

 

Less compressed JPEGs here: https://www.smugmug.com/gallery/n-Jfdr66/

 

α7R III at a cooked 55 Otus ISO 6400

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

 

M10 at a cooked Noctilux ISO 6400

[Chaemono]

Mount is starting to bend. I’ll switch to the Sony 55/1.8 Sonnar FE next. Let’s see what cooked ISO rating we get out of that.

[adan]

Chaemono, can you explain again exactly what you are trying to demonstrate? And how it relates to the DxO report on the M10 (in which lenses were not a factor)?

 

I feel like I am getting "carpet-bombed" with pictures that don't show me any significant differences at all. Which may be the point, but 3-4 examples would be enough to do that.

[pico]

I'm glad that I am too old and feeble to submit work to today's technically oriented photo editors. Stuck in film metrics, ISO/ASA 200 to 1600 appeals to me.

 

Besides, "Gentlemen do not photograph at night".

[david strachan]

I admire your tenacity Chaemono, but like Adan do not see much difference between the comparisons...except all the M10 images have a bit more "pop".

[Advertisement]

Hallo,

Look here: Leica M

[adan]

Inspired by pgk's wiki link in post 153, I decided to do a little ISO testing of my own. The setup is shown below - a Gretag/Macbeth ColorChecker (for its medium gray squares) and two Tiffen (Kodak-style) color-reference targets either side of the Gretag gray squares, since I am not sure that either of Gretag's squares conform to "18% reflectance."

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

 

The setting was outdoors, direct low-winter sunlight at 5280 feet elevation, no clouds.

 

The scene was metered 5 different ways, for a "concensus" reading. M10 meter off gray patch - M10 meter, whole scene - Sekonic L-308S reflected reading off gray patch, Sekonic reflected reading whole scene - Sekonic incident reading. Set to ISO 200, the M10 and Sekonic reflected read the lighter ColorChecker gray patch at 1/500 @ f/13 - the overall scene readings and the incident reading gave me 1/500 @ f/11. I used the latter exposure for 200, and adjusted it accordingly at other ISOs (e.g. ISO 3200 was 1/4000th @ f/16). I shot .dng + jpg, using a Leica 75mm Summarit f/2.4. All M10 jpeg settings were set to "standard."

 

(a side note on metering - interesting how closely my M10 meter and my Sekonic "synced" in similar usage).

 

First test - a Standard Output Sensitivity test using jpeg images. The basis of the SOS test is that if an ISO is accurate, it will render an 18% gray card with a brightness of 118 on the standard 8-bit scale of 0-255, in a .jpg in the sRGB color space. The results are tabulated below as: ISO set on the M10 - measured value of Tiffen scale left, Gretag darker gray, Gretag lighter gray, and Tiffen scale right.

 

100     109    77    119    116

200     115    78    120    120

1600   148   101   149    143

3200   137   101   149    145

 

Put simply, the M10's ISOs are pretty much correct, or if anything, higher than the nominal ISO - IF one is aiming to get an 18% gray correct (i.e. according to this particular test). Only ISO 100 is darker than the nominal exposure should be.

 

Key points: 1) the darker Gretag patch is obviously an outlier - darker than 18% 2) the M10 .jpg at ISO 200 conforms very closely to the SOS standard, at worst 3/255s (1.2%) darker than nominal. 3) the higher ISOs are substantially brighter (higher effective ISO - perhaps 2000 and 4000) than nominal. 4) ISO 100 was the "darkest." 5) those results are almost the reverse of the DxO results, using a different test measuring Saturation-based speed. And 6) they apply directly only to jpg output, which most of us do not use.

 

Therefore, I moved on to my own "invention" - a quasi-Standard Output Sensitivity test. Opening the .dngs shot simultaneously with the test .jpgs used above, with Adobe Camera Raw, as an sRGB 8-bit file to match the jpgs. Call it the "qSOS_srgb" test. ACR exposure settings were all set to "0," Leica M10 profile was used (presumably the same one the camera uses for creating .jpgs in-camera). No recovery of highlights or shadows. The results:

 

100     109    77    119    116

200     115    78    120    120

1600   145   100   149    151

3200   139   100   149    147

 

Virtually identical to the results from the in-camera .jpgs

 

Just for grins, since I normally use the Adobe 1998 working space, I repeated that trial opening the same .dngs into that space. Call it the qSOS_A1998 test. Results:

 

100     108    78    118    115

200     118    80    121    123

1600   144   101   148    149

3200   137   101   147    146

 

Now, my quasi-SOS tests are not officially sanctioned by the ISO. The SOS is not to be used with raw files, because it can't be compared across cameras of different bit depths (118 in 8-bit may or may not be any particular value in 12/14/16-bit). Still, I found it interesting that the M10's jpegs do meet or surpass the SOS for ISO in jpegs, and the .dng files track the jpgs very closely. In my next attempt, I am going to try to at least approximate the idea of the Saturation-Based Speed test that DxOMark uses for measuring ISO. (More or less, finding the actual ISO that avoids clipping highlights, rather than reproducing am 18% gray).

 

For reference here is a closeup of my measuring areas for the data listed above, across the four different gray samples. I used Photoshop's eyedropper set to a radius of 11 to avoid too many spurious readings of speckles or noise.