M8raw2dng - Tool to convert Leica M8 RAW files to DNG

[Erik Gunst Lund]

Advertisement (gone after registration)

Thanks

[williamgm]

Arvid, this is a little off-topic but would the same thing be possible for files from the Leica Digilux 2 ? I'm not even sure if it would work, but given the popularity of this camera (I bought mine earlier this year) any way to obtain more quality from an old camera/sensor would be welcome!

 

Thanks,

Geoff

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[menos I M6]

Hello Dirk,

 

I encounter something similar in some situations with my M8 so I have already some ideas how to address this.

1.) DNG has the option to include defective pixels so that the raw converters can interpolate them (this would increase the filesize a little bit though)

2.) interpolate the Pixels directly in the data during conversion. (i.e. fix the raw data) (this changes the raw data though)

 

Main problem right now is: how to create a sensor mapping? Manually would be easy but tedious (though you only have to do it once), automatically is a lot of programming effort, error handling, testing... I can't decide on how to do it.

 

But be assured that there will be a solution as I myself have this issue...

…

Cheers, Arvid

 

Hey Arvid, I think a manual solution, where one can set a manual matrix with a GUI would be great. As I understand it, pixel defects occur and stay rather stable, so an automated solution would not be preferred.

[Erik Gunst Lund]

The new Mac version lacks info on the white balance.

 

Looks really good in ACR6, as soon as you set the wb,

 

If you are into boosting shadows two stops are possible at ISO 160.

 

The other raw converters don't like the latest DNG files...

[bjornthun]

I've tried it today, running win7, 64-bit with Lightroom 5 and CaptureOne v.7.

 

It works with both converters, but LR5 seems to work much better than C1 v.7.

 

The noise pattern looks much better than with the 8-bit dng files the camera produces. Probably the 16-bit file format allows the camera to store the 14-bit sensor readout in a better way than is the case with 8-bit files. Essentially the noise performance suffers when using the 8-bit files.

 

After converting the .RAW files to dng the information on the lens used is lost at least in LR5. Is it possible to retain that in the converted dng files?

 

Arvid, thanks for making this excellent software available!

[Erik Gunst Lund]

8 bit is 256 values, or shades of gray

16 bit is 64.000 values, or shades of gray

 

A huge difference when the converters does the number crunching in the dark values...

[bjornthun]

Advertisement (gone after registration)

8 bit is 256 values, or shades of gray

16 bit is 64.000 values, or shades of gray

 

A huge difference when the converters does the number crunching in the dark values...

 

And also a big difference when the camera writes the sensor readout to the file, assuming it uses the full 14 bits that are read and stores those properly in the 16-bits of file space for each pixel.

 

Arvid do you know if the 16-bit files actually makes the camera use the approx 65536 for storing 14-bit or 16384 levels of information. Are the 16384 possible levels of information from a 14-bit sensor readout actually stored to the file or is it "dumbed down" to 8-bit even though 16-bit storage is used per pixel in the RAW file.

[bla]

Arvid do you know if the 16-bit files actually makes the camera use the approx 65536 for storing 14-bit or 16384 levels of information. Are the 16384 possible levels of information from a 14-bit sensor readout actually stored to the file or is it "dumbed down" to 8-bit even though 16-bit storage is used per pixel in the RAW file.

 

Hello Bjorn,

 

if that was the case the whole effort would be in vain, wouldn't it? Of course does the 16bit file contain the 14bit sensor readout values, i.e. 0 to 16383.

 

So we don't use all 65535 available values, but still...

 

@Erik

The missing white balance surprises me. I will check on that. I guess it might have something to do with my change of the camera Model. Same explanation for Bjorns missing Lens information. I will have a quick look on my files and tell you my findings in a minute.

 

BTW nice that you like the image quality even though it's still rather unpolished...

 

Edit:

@Erik: tried to open the latest DNG with c1pro v7 on mac. Worked fine.

@Bjorn: I too have LR5 and the lens data shows in Metadata (that is if it's 6-bit coded I see Lens name, focal length and aperture). Do you have the latest version of m8raw2dng from my page? there was some messup in older 1.05 version that I fixed today.

 

Greetings,

Arvid

Edited November 17, 2013 by bla

[bjornthun]

Hello Bjorn,

 

if that was the case the whole effort would be in vain, wouldn't it? Of course does the 16bit file contain the 14bit sensor readout values, i.e. 0 to 16383.

 

So we don't use all 65535 available values, but still...

 

@Erik

The missing white balance surprises me. I will check on that. I guess it might have something to do with my change of the camera Model. Same explanation for Bjorns missing Lens information. I will have a quick look on my files and tell you my findings in a minute.

 

BTW nice that you like the image quality even though it's still rather unpolished...

 

Greetings,

Arvid

 

Then it works the way I thought it did! Thanks for confirming. :)

 

Then my next theory: The Leica M9 gained much of its' improved noise performance from just using 16-bit raw files. After all this is an important part of the signal processing.

[bla]

Then my next theory: The Leica M9 gained much of its' improved noise performance from just using 16-bit raw files. After all this is an important part of the signal processing.

 

As far as I know the m9 uses the same 14bit data in 16bit container... And in normal compressed mode (which many people use) the same 8bit compression scheme the m8 uses, too. I guess better noise performance comes from the different sensor. It just got better over time...

[bjornthun]

As far as I know the m9 uses the same 14bit data in 16bit container... And in normal compressed mode (which many people use) the same 8bit compression scheme the m8 uses, too. I guess better noise performance comes from the different sensor. It just got better over time...

An improved sensor would explain better noise characteristics even with the 8-bit compressed files in the M9. Then Lightroom is capable of using 16-bit files from the M8 to do the noise reduction with a visually even better result than with 8-bit M8 files. Then it all makes sense to me.

[AncientCityPhoto]

I tried out the software! Very interesting. I am using Lightroom 5 and it reads the DNG files just fine. Without setting a black point, ISO2500 files are amazingly clean. They are also magenta colored. With the black values command set, I get files very similar to the standard DNG file. Once the magenta slider is set to the green side, and the profile is on "camera standard" the colors very similar. Still, noise at ISO2500 is better...cleaner and more natural looking.

My exif data seems all there. Aperture, lens, camera, etc...

 

However...now I see the sensor issues! I see a whole like of "dead pixels?" A big black line going about 1/4 up the image on the left side bottom! Amazing how this doesn't show up on the standard DNG files.

 

Why couldn't Leica just offer up this data to begin with as the standard DNG?

[bjornthun]

I've shot test images on all ISOs in available light both in the normal mode and in service mode, and I will check them out later in LR and compare.

 

I too wonder why Leica hasn't offered this themselves, but my suspicion is the increased write times for 20MB files.

 

What are the files with ending .bia? Can they be deleted?

[menos I M6]

@AncientCityPhoto - I have a similar issue when using the .RAW conversion with my M8.2.

 

I hope, Arvid (who also indicated, he has similar issues with dead pixels) can find a solution to fix dead pixels.

 

I think, as now more people are able to give feedback on his amazing software, more issues show up, which needs fixing ;-)

 

The prospect of the much nicer files though is immense. As others have commented, the converted .RAW files look a lot better at higher ISO levels, than native M8 DNG files.

 

@ Arvid - although I understand, that having a separate colour profile for the different .RAW files, though using a unique camera manufacturer name, I honestly would prefer to retain the original camera name and deal with the colours in post processing (or in my case, not at all, as I shoot mostly B&W).

[AncientCityPhoto]

For black and white the files are great. The biggest issue with actual usability at the moment I believe is the hot and dead pixels. The RAW fiels are obviously not running some of Leicas clean-up job. Whatever Leica does to their files is pretty darn good. If we can get that to run on these RAW files we are in serious business! Especially at 2500 ISO.

[bla]

Hello and thanks for your comments,

 

I will not quote all of your posts but try to address the issues generally.

 

I'll create a parameter for the model in exif so everyone can decide for themselves. I just noticed when using LR5 it gives you one additional color profile to choose from, namely embedded. I can see no other drawbacks, but I can change this.

 

Dead pixels, hot pixels can and will be fixed so I advise you to not delete the RAW+JPG pairs yet.

 

As of now there have not been any surprising issues for me that I wasn't aware of. I was mainly focusing on the color cast but seems like some sort of pixel fixing is on your wishlist.

I have two main issues here:

1.) anyone got an idea on how to acquire a test image where the defects nicely and reliably show?

2.) I do not like the idea of having to create a GUI driven pixel fixer because that'd be a whole project of its own, especially as I'm supporting Windows, Mac & Linux. SO i guess it'll be something like "open the DNG in a viewer, look for coordinates of defect pixels and put them in a file named like your m8 serial or something similar.

Cool thing would be you wouldn't have to send the camera to Solms if a new defect appears...

But as I said I couldn't take a photo yet where it really shows so advice is needed.

 

I am surprised that you are happy with ISO results as I exclusively shoot at 160. I have to write the article to explain things...

 

Last I want to mention that Brian at leicaplace.com has done some similar tests and was also quite content. I invite you to post images where an improvement is seen as there are many sceptics that have to see proofs rather than testing it for themselves...

Especially since the acquisition of an image is a little more complicated. Btw does the longer write time bother you?

 

last: keep the bia file please for now. it's the black image acquisition and is taken for shutter speeds longer than 1/15s or so. I don't use it yet. If you want to explore the content, rename the bia to raw and run it through the converter (using the same jpg)

 

Have fun and share the information!

 

Arvid

[Erik Gunst Lund]

The .bia file is where you would see your hot/dead pixels.

[bla]

The .bia file is where you would see your hot/dead pixels.

 

Hi Erik,

 

this is not working unfortunately. You can try for yourself with earlier mentioned method. (renaming the BIA and converting)

 

@Geoff:

Sorry, man, I totally forgot to answer your question! I have no idea about the digilux models. If you get them to show a service menu and if they allow to write some kind of raw data, I can have a look at the files. But as it is more or less a Panasonic camera I am in doubt. Feel free though to contact me if you find something. Maybe start from here.

 

Greetings,

Arvid

[Erik Gunst Lund]

Too bad, would have been too easy

[Advertisement]

Hallo,

Look here: Leica M

[bjornthun]

When I use the -b and -c commands, the make and model of the lens is displayed in Lightroom 5.

 

I have now looked at my files shot at 1250 and 2500 ISO and they look much better than what I'm used to from the M8. It seems that we can forget about colour noise and actually just work on luminance noise when doing noise reduction in post at 1250 and 2500 ISO. Using "Camera standard" instead of "Adobe standard" improves the colours a lot. Sometimes I find it helpful to also move the tint slide to the left in "Camera profile".

 

I can see one vertical line in the images at 2500 ISO.

 

I'm starting to think that with current RAW developers the M9 or M-E should be useable at 2500 ISO.