The M9 has indeed one stop advantage over the M8

[t024484]

Advertisement (gone after registration)

I had rather mixed feelings because of all sorts of conflicting reports, generated with Imatest or whatever other test, in which the M9 was compared against the M8 with respect to noise performance and Dynamic range, at various ISO settings.

Some of the tests even coming to the conclusion that the DR of the M8 was superior to the M9.

Because Leica mentioned that the M9 had 1 stop less noise as the M8, and because noise has always been an issue since the introduction of the M8 was what motivated me to take a deep breath.

 

The M9 has an elevated black level, showing because of that the full noise content, fluctuating around the mean value in positive and negative direction, and can be represented as a Gaussian curve.

That makes it very easy to calculate in isolation the Standard Deviation of the Read Noise (RN) and Quantization Noise (QN) from a complete black image, for all ISO values.

 

The M8 on the other hand, generates files where the black level has already been subtracted, making it more difficult to calculate the RN and QN, because the left part of the Gaussian curve is cut.

 

In the past, I have tried in a rather simple way to calculate the standard deviation of the RN and QN of the M8, assuming that only negative values of the noise were cut, exactly at the point where the Gaussian curve would have had its maximum.

In that case, the noise has only half of its energy, or 1/Sqrt (2) of its rms noise voltage.

This assumption proved to be quite wrong in the investigation I performed now.

 

I collected for the M8 and for the M9 images with the following details:

For ISO 160, ISO 640 and ISO 2500, images of a white wall exposed as:

Fully black

¼ exposure

½ exposure

¾ exposure

Full exposure, just below clipping.

 

I isolated the R, Gr, Gb and B pixels from the fifteen M8 and the fifteen M9 Raw images, and calculated the average exposure value (EV), the standard deviation (SD) and the SNR (Signal to Noise=EV/SD) for all 4 * 30 = 120 different data sets.

 

As a next step, I compared all calculated data against a computer model that I had build, to validate the results, and to refine my model.

The result of this all is that my computer model is now accurate to +/- 0.1 EV or +/- 1/10 of a stop for the two cameras that I used.

All physical elements are taken care off in my model, being RN, QN, Photon Shot Noise, Pixel Response Non Uniformity (PNRU) Noise and the Compression Noise that the M8 adds.

 

All things being equal between the two cameras, apart from the amount of pixels, there are two important differences.

The M9 can produce uncompressed files where the M8 cannot, and there happens to be an important difference in RN.

 

Compression noise has a measurable contribution up to ISO 640, but Read noise is playing an important role over the whole ISO range.

After having spent so much time in measuring, I can now confirm that the M9 has 41% less RN as the M8 has, which has a noticeable contribution in noise reduction looking at the graphs below.

 

For those waiting for M9 software updates to reduce noise, I can only say that software noise reduction leads to a decrease in resolution and possible degradation of the Image. In order to get a better High ISO performance, the right way to go is to decrease the RN.

 

The fact that the D3X performs so much better at high ISO has everything to do with a RN value that is 1/3 of the M9, and much less with software. The D3X has a large amount of read amplifiers, all working at a much lower speed, and therefore generating much less noise because of reduced bandwidth.

 

But for the time being, a 41% in RN reduction plus having uncompressed files is already a good step in the direction. The positive effect of both is that over the whole range of ISO 160 to ISO 1250 for the M8, the M9 has the same noise figure when the ISO is set 2/3 stop higher.

In combination with the fact that when the same composition is printed on the same size of paper, this will give the M9 another 26% [=Sqrt(10/18)] or 0.4 Stops in over-all noise reduction over the M8.

 

When not considering the printing advantage, the gain of the M9 is 2/3 Stops in ISO over the M8 as the pictures below are showing. The two graphs are for all different ISO settings almost exactly covering each other.

 

When setting the M9 at twice the ISO value of the M8, a gap of less then 0.4 Stops is created between the two curves, being the value that is gained at printing the same composition on the same size op paper.

 

Conclusion

 

In total adding the 3 positive effects, being lower RN, uncompressed Raw files and more pixels available for printing on the same printing area, contribute to 1 Stop in ISO setting, fully confirming Leica’s statement.

 

ISO 160 for the M8, is equal in noise to ISO 320 for the M9. This goes on all the way to ISO 1250 for the M8 having the same noise as ISO 2500 for the M9.

I restricted myself to the Raw images, the effectiveness of Raw processors is left untouched.

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

 

 

As an addition I have added a graph with the noise characteristics of the M8 and the M9 this time both at the same ISO 640 to show what a big step forward the lower read noise brings.

 

 

 

PS I thank Chris Tribble for his highly estimated contribution, in sending me various M9 Images

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[sean_reid]

I had rather mixed feelings because of all sorts of conflicting reports, generated with Imatest or whatever other test, in which the M9 was compared against the M8 with respect to noise performance and Dynamic range, at various ISO settings.

Some of the tests even coming to the conclusion that the DR of the M8 was superior to the M9.

Because Leica mentioned that the M9 had 1 stop less noise as the M8, and because noise has always been an issue since the introduction of the M8 was what motivated me to take a deep breath.

 

The M9 has an elevated black level, showing because of that the full noise content, fluctuating around the mean value in positive and negative direction, and can be represented as a Gaussian curve.

That makes it very easy to calculate in isolation the Standard Deviation of the Read Noise (RN) and Quantization Noise (QN) from a complete black image, for all ISO values.

 

The M8 on the other hand, generates files where the black level has already been subtracted, making it more difficult to calculate the RN and QN, because the left part of the Gaussian curve is cut.

 

In the past, I have tried in a rather simple way to calculate the standard deviation of the RN and QN of the M8, assuming that only negative values of the noise were cut, exactly at the point where the Gaussian curve would have had its maximum.

In that case, the noise has only half of its energy, or 1/Sqrt (2) of its rms noise voltage.

This assumption proved to be quite wrong in the investigation I performed now.

 

I collected for the M8 and for the M9 images with the following details:

For ISO 160, ISO 640 and ISO 2500, images of a white wall exposed as:

Fully black

¼ exposure

½ exposure

¾ exposure

Full exposure, just below clipping.

 

I isolated the R, Gr, Gb and B pixels from the fifteen M8 and the fifteen M9 Raw images, and calculated the average exposure value (EV), the standard deviation (SD) and the SNR (Signal to Noise=EV/SD) for all 4 * 30 = 120 different data sets.

 

As a next step, I compared all calculated data against a computer model that I had build, to validate the results, and to refine my model.

The result of this all is that my computer model is now accurate to +/- 0.1 EV or +/- 1/10 of a stop for the two cameras that I used.

All physical elements are taken care off in my model, being RN, QN, Photon Shot Noise, Pixel Response Non Uniformity (PNRU) Noise and the Compression Noise that the M8 adds.

 

All things being equal between the two cameras, apart from the amount of pixels, there are two important differences.

The M9 can produce uncompressed files where the M8 cannot, and there happens to be an important difference in RN.

 

Compression noise has a measurable contribution up to ISO 640, but Read noise is playing an important role over the whole ISO range.

After having spent so much time in measuring, I can now confirm that the M9 has 41% less RN as the M8 has, which has a noticeable contribution in noise reduction looking at the graphs below.

 

For those waiting for M9 software updates to reduce noise, I can only say that software noise reduction leads to a decrease in resolution and possible degradation of the Image. In order to get a better High ISO performance, the right way to go is to decrease the RN.

 

The fact that the D3X performs so much better at high ISO has everything to do with a RN value that is 1/3 of the M9, and much less with software. The D3X has a large amount of read amplifiers, all working at a much lower speed, and therefore generating much less noise because of reduced bandwidth.

 

But for the time being, a 41% in RN reduction plus having uncompressed files is already a good step in the direction. The positive effect of both is that over the whole range of ISO 160 to ISO 1250 for the M8, the M9 has the same noise figure when the ISO is set 2/3 stop higher.

In combination with the fact that when the same composition is printed on the same size of paper, this will give the M9 another 26% [=Sqrt(10/18)] or 0.4 Stops in over-all noise reduction over the M8.

 

When not considering the printing advantage, the gain of the M9 is 2/3 Stops in ISO over the M8 as the pictures below are showing. The two graphs are for all different ISO settings almost exactly covering each other.

 

When setting the M9 at twice the ISO value of the M8, a gap of less then 0.4 Stops is created between the two curves, being the value that is gained at printing the same composition on the same size op paper.

 

Conclusion

 

In total adding the 3 positive effects, being lower RN, uncompressed Raw files and more pixels available for printing on the same printing area, contribute to 1 Stop in ISO setting, fully confirming Leica’s statement.

 

ISO 160 for the M8, is equal in noise to ISO 320 for the M9. This goes on all the way to ISO 1250 for the M8 having the same noise as ISO 2500 for the M9.

I restricted myself to the Raw images, the effectiveness of Raw processors is left untouched.

 

[ATTACH]168658[/ATTACH]

 

 

As an addition I have added a graph with the noise characteristics of the M8 and the M9 this time both at the same ISO 640 to show what a big step forward the lower read noise brings.

 

[ATTACH]168659[/ATTACH]

 

 

PS I thank Chris Tribble for his highly estimated contribution, in sending me various M9 Images

 

You arrived at your results via a very different method but your results, as you know, are consistent with what I observed in my testing. Thanks for your work.

 

Cheers,

 

Sean

[tgm]

Thanks a lot ,

these are very interesting findings, it is not so clear to me why the new test gives much more solid results, so, I do not really understand what was wrong in the first test. Anyway, I am willing to believe the second new series of tests which confirms Leica's statement that the M9 has almost one stop advantage in noise level over the M8.

 

 

On the other hand, it puts the question, why other reviewer for example Erwin Puts come to the opposite conclusion. Do you have an explanation?

 

Thomas

[rickp13]

thank you for investing so much time and effort in this study. i will watch your thread with great interest.

 

greetings from hamburg

 

rick

[perb]

In order to get a better High ISO performance, the right way to go is to decrease the RN.

 

The fact that the D3X performs so much better at high ISO has everything to do with a RN value that is 1/3 of the M9, and much less with software. The D3X has a large amount of read amplifiers, all working at a much lower speed, and therefore generating much less noise because of reduced bandwidth.

 

This is very interesting , Hans. As Sean said, it matches his observations, and it also matches what S2 project manager Stephan Shulz said in response to question #2:

 

http://www.l-camera-forum.com/leica-forum/leica-s2-forum/95115-press-release-s2-technical-specs-7.html#post1002971

 

I wonder if it would be technically feasible to have a new option in future firmware for the M8/M9, and if so, would Leica be interested?

 

Myself, I hardly ever shoot many fps, rather, I do one shot here and another there, so if I could set my camera to slow/low-noise I would probably have that setting on at all times.

 

Could Leica work more with Kodak on read noise for the future M10?

 

Regards

Per

[sean_reid]

This is very interesting , Hans. As Sean said, it matches his observations, and it also matches what S2 project manager Stephan Shulz said in response to question #2:

 

http://www.l-camera-forum.com/leica-forum/leica-s2-forum/95115-press-release-s2-technical-specs-7.html#post1002971

 

Regards

Per

 

But, of course, in that discussion Stephan was talking about possible increases in the S2 - a different kettle of fish.

 

Cheers,

 

Sean

[rosuna]

Advertisement (gone after registration)

After having spent so much time in measuring, I can now confirm that the M9 has 41% less RN as the M8 has, which has a noticeable contribution in noise reduction looking at the graphs below.

 

For those waiting for M9 software updates to reduce noise, I can only say that software noise reduction leads to a decrease in resolution and possible degradation of the Image. In order to get a better High ISO performance, the right way to go is to decrease the RN.

 

The fact that the D3X performs so much better at high ISO has everything to do with a RN value that is 1/3 of the M9, and much less with software. The D3X has a large amount of read amplifiers, all working at a much lower speed, and therefore generating much less noise because of reduced bandwidth.

 

But for the time being, a 41% in RN reduction plus having uncompressed files is already a good step in the direction. The positive effect of both is that over the whole range of ISO 160 to ISO 1250 for the M8, the M9 has the same noise figure when the ISO is set 2/3 stop higher.

 

Really interesting conclusions...

 

The alternatives for Leica in reducing the noise levels are sensors with lower read noise, and this means more amplifiers al lower speeds.

 

Perb asked:

 

This is very interesting , Hans. As Sean said, it matches his observations, and it also matches what S2 project manager Stephan Shulz said in response to question #2:

 

http://www.l-camera-forum.com/leica-forum/leica-s2-forum/95115-press-release-s2-technical-specs-7.html#post1002971

 

I wonder if it would be technically feasible to have a new option in future firmware for the M8/M9, and if so, would Leica be interested?

 

This is:

 

higher ISO can also be achieved in two other ways. The first is to slow down the read speed of the sensor output from 24 Mhz to 12 Mhz. The signal has less noise, but the frame rate would drop by half.

 

I don't used the 2 fps frame rate anyway...

 

Many thanks t024484!

.

[dalippe]

You arrived at your results via a very different method but your results, as you know, are consistent with what I observed in my testing. Thanks for your work.

 

Cheers,

 

Sean

 

Hi Sean,

 

You arrived at the same conclusion about the final print, which I suppose is all that matters in the end. But there also seem to be some interesting disagreements (or perhaps differences if "disagreements" sounds you two are stuck in a debate, which clearly you're not). OP seems to be saying at even at base ISO (before you believed any software smoothing came into effect), there is better S/N *at the pixel level*. I might have missed recent updates, but my memory is that you saw no advantage at the pixel level and attributed all of the gain in S/N to downsampling the M9 and/or upsampling the M8.

 

David

[BerndReini]

I arrived at the same conclusions without any scientific tests after a day with a demo M9 simply shooting what I usually shoot and looking at the pictures. I admit that the way I shared my unscientific results with other forum members was a little provocative, but boy, could I have used these formal results to back me up back then.

 

For what it's worth, when I go shooting at nighttime, I now limit my Auto ISO settings to 1/60s and ISO 1000. I used to set my ISO to 320 with the M8 and quickly switch to 640 in a pinch. On that note, I do wish that the Auto ISO could be limited to 1/250 for daylight street photography and maybe with half-steps from 1/60th down. I know this is asking a lot but a half stop makes a big difference at night and so does the difference between 1/45s and 1/30s as far as sharpness for hand-holding people photos with a 50mm.

[hammam]

...

 

ISO 160 for the M8, is equal in noise to ISO 320 for the M9. This goes on all the way to ISO 1250 for the M8 having the same noise as ISO 2500 for the M9...

 

This part, I think I understood.

 

But thanks for the time and effort.

[thrice]

Perhaps Leica would be better served sending you a camera to test Hans? rather than Herr Puts?

 

After all, how many users concerned about high iso and DR are shooting in jpeg.

 

I understood enough of your testing from my brief electrical engineering background, it looks solid and comprehensive. You seem to have a very good understanding of the imaging/processing characteristics of the camera which is great! Leica should offer you a job

[diogenis]

You arrived at your results via a very different method but your results, as you know, are consistent with what I observed in my testing. Thanks for your work.

 

Cheers,

 

Sean

Sean, if I can remember, your observations were also 1 stop difference but the reasoning was that this was exhibiting due to more information from the larger sensor printed on paper. However Hans with his research proved a real 1stop difference on sensor level. I believe that your reasoning still stands, so shouldn't the outcome be then 2 stops instead of one (better sensor + larger sensor) on printed photos???

In short question is: the effect of noise diminishes when more pixels are involved or not?

[GMB]

This part, I think I understood.

 

But thanks for the time and effort.

 

Me too

[Guest roey]

Sean, if I can remember, your observations were also 1 stop difference but the reasoning was that this was exhibiting due to more information from the larger sensor printed on paper. However Hans with his research proved a real 1stop difference on sensor level. I believe that your reasoning still stands, so shouldn't the outcome be then 2 stops instead of one (better sensor + larger sensor) on printed photos???

In short question is: the effect of noise diminishes when more pixels are involved or not?

 

Hans' explanations are quite elaborate, so I guess it is easily missed -- but he came to the conclusion that the 1 stop improvement is a result of both lower sensor noise *and* more pixels:

 

In total adding the 3 positive effects, being lower RN, uncompressed Raw files and more pixels available for printing on the same printing area, contribute to 1 Stop in ISO setting, fully confirming Leica’s statement.

[chris_tribble]

In total adding the 3 positive effects, being lower RN, uncompressed Raw files and more pixels available for printing on the same printing area, contribute to 1 Stop in ISO setting, fully confirming Leica’s statement.

 

MANY thanks for the hard work on this (and I'll send white wall shots you asked for as soon as I'm near a daylight lit white wall - problem is it's late October in London and I'm tied to a computer for the next day or two!).

[lars_bergquist]

Profuse thanks for this well reasoned piece of scientific method. It comes as a great relief from all the club-thumping asseveration, with accompanying Neanderthalian vocalizations, that we have been hitherto treated to as a 'discussion' of this matter.

 

A little bit of peer review would also be nice of course.

 

The un-reconstructed old man from the Age of Evidence

[guidomo]

Great piece, thanks for this.

 

Just one point of caution I would like to make. For myself one major reason for getting the M9 is the wider FOV when compared to the M8, helping at the short focal length end.

 

Now, you say that part of the 1 stop advantage of the M9 over the M8 is due to increased resolution. That of course is only true if one compares images with the same field of view, i.e. uses a longer lens on the M9. If one uses the same focal length, enjoying the wider FOV, the effective resolution of M8 and M9 is the same, and the 1 stop advantage might not be a full stop.

 

This leads me to conclude that the two perceived advantages of the M9, namely wider FOV (or "getting rid of the crop") and the 1 stop advantage are somewhat double counting of the same thing. Not something you have said in your report, but some people may be thinking they get both advantages on top of each other.

[Leicaiste]

If one uses the same focal length, enjoying the wider FOV, the effective resolution of M8 and M9 is the same, and the 1 stop advantage might not be a full stop.

.

 

Sorry, but who will do that ?

 

I didn't want the FF for more field of view with the same lens, but to be able to use the lenses for what they were designed for.

 

Using again the 35/2 when I was using the 28/2 on the M8 etc.

 

And by the way, taking full advantage of the resolution.

[Advertisement]

Hallo,

Look here: Leica M

[guidomo]

I didn't want the FF for more field of view with the same lens, but to be able to use the lenses for what they were designed for.

 

Isn't that the same thing?

[carstenw]

No, one looks for more field of view with the same lens, the other seeks to use the lenses for the field of views they were designed for, i.e. the same field of view with a different lens, compared to the M8.