Jamie Roberts Posted September 11, 2009 Share #1 Posted September 11, 2009 Advertisement (gone after registration) Hey--just a question out there for everyone...especially those with the camera... Is Lightroom mangling the M9 DNGs when it imports them? Or are there differences between M9's own compression scheme that are showing up in color? I'm asking because of a couple of things...the main one being it's so hard to test without actually having the camera to hand... 1) Chris Tribble's posted DNGs (which are M9 compressed, right Chris?) which admittedly were taken in poor light, have been somehow "massaged" by Lightroom so that when I take them into C1 they are not recognised by C1 and you have to manually apply the M9 profile. When I do that, however, the color is absolutely horrid. Skin tones especially cannot be corrected. There is literally diagrammatic color separation in what should be a smooth tone. 2) Similarly, but much less severely, I've seen some yellow splotchiness in some M9 compressed DNGs in C1 especially at high ISOs (and in tungsten? Might have been fluorescent). Subtle, but there. Will have to check firmware, but I think it's been production level shots. 3) I just downloaded an "untouched" uncompressed DNG and though it has mixed light and high ISO (2500) (and therefore an impossible color balance) it doesn't exhibit anything like blotchiness. Looks like daylight and fluorescent to my eyes) @ Tim--I think that's where your "banding" issues are coming from... So is it Lightroom or the compression scheme? The light source? Or some combo of everything? Link to post Share on other sites More sharing options...
Advertisement Posted September 11, 2009 Posted September 11, 2009 Hi Jamie Roberts, Take a look here Is Lightoom mangling M9 images? Or is there a compression issue?. I'm sure you'll find what you were looking for!
sandymc Posted September 11, 2009 Share #2 Posted September 11, 2009 Jamie, This may or may not be related, but I've gotten some very odd results with M9 images converted via CornerFix and then loaded into C1. I say odd because I can't quite work out what's happening, and it's only one of about 10 images I have. In this case it shows as a strange kind of overexposure rather than bad color. C1 not liking any image that isn't exactly as the camera wrote it is an old story, but this is a step stranger than usual - normally C1 just throws up over any image in a format it doesn't like; this time its just some images. Both Aperture and LR load the same image fine though....and C1 loads the original image fine. Regards, Sandy Link to post Share on other sites More sharing options...
chris_tribble Posted September 11, 2009 Share #3 Posted September 11, 2009 Is Lightroom mangling the M9 DNGs when it imports them? Or are there differences between M9's own compression scheme that are showing up in color? Jamie - it may be because I EXPORTED the DNG from LR rather than copying the original (compressed DNG). I'm currently uploading two files - the original problem as a straight copy of the original - here: http://dl.getdropbox.com/u/862415/L1000026-2.DNG + an uncompressed image taken in bright sunlight this morning (pulled 80) here: http://dl.getdropbox.com/u/862415/L1000295.DNG Be interesting to hear what you're able to do with them! Link to post Share on other sites More sharing options...
sandymc Posted September 11, 2009 Share #4 Posted September 11, 2009 Ok, I just found something that could explain what you're seeing. Sorry - technical content here: WARNING: PRELIMINARY AND SUBJCT TO CONFIRMATION - I just worked this out in the last 10 minutes, and may be missing something(!) The M9 DNG's use the same compression technique as the old M8. However, the M9 DNGs also set the black level, in some case to quite a high value - I have one image here that's set to 500 (white is 16383), another that's 165. However, the compression scheme is pre-black level adjustment. Net result is, while the M8 compressed always gave you 256 levels, the M9 compressed will give you less real levels post black level adjustment. In the case of a black level of 500, you're down to 212 effective levels. AKA, you could quite possibly see banding/color issues on an M9 using compressed DNGs. IMHO. Could that be what you're seeing? Sandy Link to post Share on other sites More sharing options...
chris_tribble Posted September 11, 2009 Share #5 Posted September 11, 2009 Sandy - sounds plausible but I'm not the expert here... Glad I've ordered some large SDHC cards as I'd pretty well decided I'd not use the compressed DNGs - but feedback to Leica is going to be important if you're right as they may want to change the algorithm... Ah, the joy of early adoption! Link to post Share on other sites More sharing options...
sandymc Posted September 11, 2009 Share #6 Posted September 11, 2009 Chris, Yes, well, interested to see what Jamie says - he can see by eye what takes me piles of numbers(!) Sandy Link to post Share on other sites More sharing options...
Jamie Roberts Posted September 11, 2009 Author Share #7 Posted September 11, 2009 Advertisement (gone after registration) Chris-- So much better! These are essentially straight out of the box with a WB correction not even my normal M8 curves. Welcome, dear visitor! As registered member you'd see an image here… Simply register for free here – We are always happy to welcome new members! They look pretty good. FWIW, I do think you're undoing some of the uncompressed goodness pushing to ISO 80... some of the facial higlights (speculars) are a little gone (just a little). But that's bright morning sun, right? If so, it's looking really good. @ Sandy-- I think Leica and Kodak have really tried to get a more film-like set of "curves" in the M9 DNG. Compression in the shadows (getting a black point) (and less contrast in the midtones and highlights) is one way to do that. Still, I'd hope they wouldn't be quite as aggressive as you've laid out in your note! And the funny thing is, the color blotchiness has been in mid-tones or highlights.... Link to post Share on other sites Simply register for free here – We are always happy to welcome new members! They look pretty good. FWIW, I do think you're undoing some of the uncompressed goodness pushing to ISO 80... some of the facial higlights (speculars) are a little gone (just a little). But that's bright morning sun, right? If so, it's looking really good. @ Sandy-- I think Leica and Kodak have really tried to get a more film-like set of "curves" in the M9 DNG. Compression in the shadows (getting a black point) (and less contrast in the midtones and highlights) is one way to do that. Still, I'd hope they wouldn't be quite as aggressive as you've laid out in your note! And the funny thing is, the color blotchiness has been in mid-tones or highlights.... ' data-webShareUrl='https://www.l-camera-forum.com/topic/96362-is-lightoom-mangling-m9-images-or-is-there-a-compression-issue/?do=findComment&comment=1029906'>More sharing options...
sandymc Posted September 11, 2009 Share #8 Posted September 11, 2009 Just FYI, I've checked. I would not recommend using compressed DNGs on the M9 with the current firmware. But of course, your mileage may vary. Sandy Link to post Share on other sites More sharing options...
scott kirkpatrick Posted September 11, 2009 Share #9 Posted September 11, 2009 Jamie and Chris -- I saw the un-LRed file come across and downloaded a copy, since your shot of the three people sipping wine was my example for which the path through LR to neutral dng to generic M9 dng gave red faces that we dare not show here... I'll attach a crop of how bad it looks in neutral dng, straight from LR. I've tweaked the uncorrupted shot to keep a bit more color in their faces, but shifting the color temp from 2800 to 3000 and the offset from +12 magenta to +1 or 2. Jamie seems to have lightened the shot a bit, and I should, too, but there is definitely now room to work with these. scott so first one -- bad !!! LR forced it into neutral DNG profile second one -- corrected using only WB and magenta/green offset with M9 generic profile. Welcome, dear visitor! As registered member you'd see an image here… Simply register for free here – We are always happy to welcome new members! Link to post Share on other sites Simply register for free here – We are always happy to welcome new members! ' data-webShareUrl='https://www.l-camera-forum.com/topic/96362-is-lightoom-mangling-m9-images-or-is-there-a-compression-issue/?do=findComment&comment=1030161'>More sharing options...
Jamie Roberts Posted September 11, 2009 Author Share #10 Posted September 11, 2009 Just FYI, I've checked. I would not recommend using compressed DNGs on the M9 with the current firmware. But of course, your mileage may vary. Sandy Wow Sandy--how much clipping are we talking about? Is there any benefit in levels gained for highlights, or are they just setting the blackpoint arbitrarily?! I thought there was something weird about the shadows... though as I said, the blotchiness is something else... Link to post Share on other sites More sharing options...
sandymc Posted September 11, 2009 Share #11 Posted September 11, 2009 Wow Sandy--how much clipping are we talking about? Is there any benefit in levels gained for highlights, or are they just setting the blackpoint arbitrarily?! I thought there was something weird about the shadows... though as I said, the blotchiness is something else... Jamie, The black level varies from image to image among the samples I have, and I have too few to be able to see a pattern. It may be, e.g., that the black level is high at high ISO, low at low, or that this is camera by camera, or is set on the fly by looking at dark pixels. So the loss would vary. My guess is however that it will vary by camera and ISO. The worst impact, I would think, would indeed probably be to shadow detail and shadow noise, but it could manifest all over as banding/posterization effects in severe cases. In effect, its a loss of dynamic range. The good news however is that even if I'm right (a) you can just use uncompressed DNGs and ( Leica should easily be able to fix the issue with a firmware update. Regards, Sandy Link to post Share on other sites More sharing options...
chris_tribble Posted September 12, 2009 Share #12 Posted September 12, 2009 Sandy / Jaimie - if your analysis is correct (and it sounds very plausible) is it worth passing directly to Stefan Daniel? I know that Leica watch the forum, but it would be sensible for them to get the message out that it's way better to use uncompressed DNGs for the moment. The relative expense in image storage and processing overhead is worthwhile compared to the expense of throwing away image data or, worse, damaging images by going for the compressed version. I for one will stick with uncompressed DNGs for the time being. Many thanks for the cogent explanations. Link to post Share on other sites More sharing options...
sandymc Posted September 12, 2009 Share #13 Posted September 12, 2009 Chris, Yes, I think it would be worth passing to Leica. It is possible that there is a well thought out engineering explanation for why they are doing things this way - sometimes you get forced into odd compromises by quirks of the sensor or DSP processor or whatever. But the most likely explanation is that whoever wrote the code just took the old M8 compression code (works fine on its own!) and the new "non-zero black level" image in the M9 (works fine on its own!), and stuck them together without quite thinking through the end result. Regards, Sandy Link to post Share on other sites More sharing options...
sandymc Posted September 12, 2009 Share #14 Posted September 12, 2009 BTW, re the "red faces" issue - I'm pretty sure its as result of LR by default performing lossless compression on images, and C1 not being able to read data that is losslessly compressed correctly. I could duplicate the problem using LR in that way, and when I ran the resulting "red faces" image through CornerFix configured to uncompress the file, C1 reads the uncompressed version of the "red faces" file correctly. Sandy Link to post Share on other sites More sharing options...
t024484 Posted September 12, 2009 Share #15 Posted September 12, 2009 Chris, Yes, I think it would be worth passing to Leica. It is possible that there is a well thought out engineering explanation for why they are doing things this way - sometimes you get forced into odd compromises by quirks of the sensor or DSP processor or whatever. But the most likely explanation is that whoever wrote the code just took the old M8 compression code (works fine on its own!) and the new "non-zero black level" image in the M9 (works fine on its own!), and stuck them together without quite thinking through the end result. Regards, Sandy Could it be that Leica has added a DC voltage to the sensors ouput amplifier before digitizing the signal. Noise, Dynamic Range and Bit Depth in Digital SLRs -- page 2 "On Canon DSLR's, there is an alternative method of measuring the read noise. Canon applies an offset or bias to the signal from the sensor; a constant voltage is added to the signal from the sensor before the signal is quantized in the ADC. Although voltage fluctuations can be either positive or negative, a fluctuation of negative amount will get clipped to zero upon quantization, since the output of the ADC is a non-negative integer raw value. By adding a bias or offset voltage, the full histogram of noise is preserved, since a small negative fluctuation from the offset value is still a positive number: When a bias is not applied (as for instance with Nikon cameras), negative voltage fluctuations are clipped to zero, making efforts to analyze and remove various forms of read noise more difficult. An example of this clipping is shown in figure 11." This could in some way explain why the M9 has less noise as the m8 has. Hans Link to post Share on other sites More sharing options...
sandymc Posted September 12, 2009 Share #16 Posted September 12, 2009 Could it be that Leica has added a DC voltage to the sensors ouput amplifier before digitizing the signal. Noise, Dynamic Range and Bit Depth in Digital SLRs -- page 2 "On Canon DSLR's, there is an alternative method of measuring the read noise. Canon applies an offset or bias to the signal from the sensor; a constant voltage is added to the signal from the sensor before the signal is quantized in the ADC. Although voltage fluctuations can be either positive or negative, a fluctuation of negative amount will get clipped to zero upon quantization, since the output of the ADC is a non-negative integer raw value. By adding a bias or offset voltage, the full histogram of noise is preserved, since a small negative fluctuation from the offset value is still a positive number: When a bias is not applied (as for instance with Nikon cameras), negative voltage fluctuations are clipped to zero, making efforts to analyze and remove various forms of read noise more difficult. An example of this clipping is shown in figure 11." This could in some way explain why the M9 has less noise as the m8 has. Hans It certainly could explain why the M9 has variable black levels, while the M8 had zero black levels. Although this is a CCD sensor rather than the CMOS sensor the Canons use, and CMOS sensors inherently have very different noise characteristics to CCD ones. Sandy Link to post Share on other sites More sharing options...
scott kirkpatrick Posted September 12, 2009 Share #17 Posted September 12, 2009 It certainly could explain why the M9 has variable black levels, while the M8 had zero black levels. Although this is a CCD sensor rather than the CMOS sensor the Canons use, and CMOS sensors inherently have very different noise characteristics to CCD ones. Sandy Notice that the M9 adjusts ISO in one-third stop increments, which is easier to do in analog (with a voltage offset) than in software. Full stop increments, as in the M8, can be done by dropping the low order bits. scott Link to post Share on other sites More sharing options...
t024484 Posted September 12, 2009 Share #18 Posted September 12, 2009 Notice that the M9 adjusts ISO in one-third stop increments, which is easier to do in analog (with a voltage offset) than in software. Full stop increments, as in the M8, can be done by dropping the low order bits. scott If a second amplifier has been used after the on board amplifier of the sensor, like Nikon an Canon are doing, this second amplifier can be used for the intermediate steps of 1/3 ISO increments, and at the same provide a signal to the a/d converter with a very low impedance, making it less susceptible for noise injection. Wether or not a second amplifier has been used, can be measured quite easily, by taking pictures with a shutter time of 1/4000 and with the lenscap on. By taking pictures with ISO values starting at ISO 160 and going upwards with 1/3 ISO incerments, and by analysing the noise in the DNG file, one has immediate answer to the questions: 1)Has a DC level been applied before digitizing 2) Are the 1/3 increments in ISO made before or after digitizing 3) What is the read noise of the sensor, and the noise of the A/D converter. I did this for the M8, and found that no DC was applied, that the Read Noise was 2 to 3 times higher compared to high end DSLR's, but that the A/D noise was very low. So the Achilles heel of the noise in the M8 lies in the poor sensor signal being digitized. By using a second amplifier, and by applying a DC voltage, a big improvement could have been made in reducing the noise. Hans Link to post Share on other sites More sharing options...
t024484 Posted September 12, 2009 Share #19 Posted September 12, 2009 If someone can provide me with M9 DNG's shot as described above for ISO 160, 200,250,320, 640 and 1250, I will perform the necessary test and report the results back in this forum. Hans Link to post Share on other sites More sharing options...
scott kirkpatrick Posted September 12, 2009 Share #20 Posted September 12, 2009 If someone can provide me with M9 DNG's shot as described above for ISO 160, 200,250,320, 640 and 1250, I will perform the necessary test and report the results back in this forum. Hans I'll do it, but won't have an M9 for a few more weeks. Sandyu McGuffog would also be interested in such a set of files and in the analysis you propose. Anybody who already has their M9 willing to help? scott Link to post Share on other sites More sharing options...
Recommended Posts
Archived
This topic is now archived and is closed to further replies.