fenykepesz Posted June 16, 2019 Share #1 Posted June 16, 2019 Advertisement (gone after registration) as i am waiting to get hold of my new M10-D i am preparing my digital workflow to be ready for the new style RAW DNG files. in general, my workflow uses linux and a combination of Python 3.x and GIMP to process all my pics, film or digital. as i don't use OS-X or Windows, please don't suggest solutions for these platform, i will not be able to make use of those. - first question is about reading data with rawpy and libraw - does anybody have experience with such for the M10-D style RAW/DNG files ? - second question is about the lens distortion & vignetting corrections and if they are applied in the camera itself or to be done outside with a separate software. ideally i wouldn't have to bother with such conversions as they would just complicate my workflow, so i expect them to happen in the camera. is that the case for RAW/DNG files ? - last question is about all the other possible corrections and to what extent they are or are not present in the DNG/RAW files of the M10-D data, as compared to the JPEG files the M10 produces ? any input greatly appreciated ! bye, pisti Quote Link to post Share on other sites More sharing options...
Advertisement Posted June 16, 2019 Posted June 16, 2019 Hi fenykepesz, Take a look here reading M10-D RAW/DNG files with linux/python/gimp. I'm sure you'll find what you were looking for!
pico Posted June 16, 2019 Share #2 Posted June 16, 2019 3 minutes ago, fenykepesz said: i don't use OS-X or Windows, please don't suggest solutions for these platform I look forward to the responses. 3 Quote Link to post Share on other sites More sharing options...
jaapv Posted June 16, 2019 Share #3 Posted June 16, 2019 AFAIK no M camera makes distortion corrections. Quote Link to post Share on other sites More sharing options...
dkmoore Posted June 16, 2019 Share #4 Posted June 16, 2019 1 hour ago, fenykepesz said: as i am waiting to get hold of my new M10-D i am preparing my digital workflow to be ready for the new style RAW DNG files. in general, my workflow uses linux and a combination of Python 3.x and GIMP to process all my pics, film or digital. as i don't use OS-X or Windows, please don't suggest solutions for these platform, i will not be able to make use of those. - first question is about reading data with rawpy and libraw - does anybody have experience with such for the M10-D style RAW/DNG files ? - second question is about the lens distortion & vignetting corrections and if they are applied in the camera itself or to be done outside with a separate software. ideally i wouldn't have to bother with such conversions as they would just complicate my workflow, so i expect them to happen in the camera. is that the case for RAW/DNG files ? - last question is about all the other possible corrections and to what extent they are or are not present in the DNG/RAW files of the M10-D data, as compared to the JPEG files the M10 produces ? any input greatly appreciated ! bye, pisti All but a few words of this was gibberish to me but I agree with the few I did understand: use OS-X or Windows 1 Quote Link to post Share on other sites More sharing options...
frame-it Posted June 16, 2019 Share #5 Posted June 16, 2019 (edited) 1 hour ago, fenykepesz said: as i am waiting to get hold of my new M10-D i am preparing my digital workflow to be ready for the new style RAW DNG files. in general, my workflow uses linux and a combination of Python 3.x and GIMP to process all my pics, film or digital. as i don't use OS-X or Windows, please don't suggest solutions for these platform, i will not be able to make use of those. - first question is about reading data with rawpy and libraw - does anybody have experience with such for the M10-D style RAW/DNG files ? - second question is about the lens distortion & vignetting corrections and if they are applied in the camera itself or to be done outside with a separate software. ideally i wouldn't have to bother with such conversions as they would just complicate my workflow, so i expect them to happen in the camera. is that the case for RAW/DNG files ? - last question is about all the other possible corrections and to what extent they are or are not present in the DNG/RAW files of the M10-D data, as compared to the JPEG files the M10 produces ? any input greatly appreciated ! bye, pisti latest libraw supports the m10d, im not sure how the lens profiles corrections would work in Gimp..the lensfun database might help IF it even has Leica M lenses , perhaps open the dng in rawtherapee on linux , enable or disable what you need and export directly to gimp? https://sourceforge.net/projects/dcptool/ might help to check whats in the Leica DNg, and what can be stripped. Edited June 16, 2019 by frame-it Quote Link to post Share on other sites More sharing options...
helohe Posted June 16, 2019 Share #6 Posted June 16, 2019 Did you try Darktable (https://www.darktable.org/) or Rawtherapee (https://rawtherapee.com/) ? Quote Link to post Share on other sites More sharing options...
fenykepesz Posted June 16, 2019 Author Share #7 Posted June 16, 2019 Advertisement (gone after registration) that's a very good start, thank you, frame-it and helohe ! i wasn't familiar with these software. let's see if they also allow a command line driven processing of photographic data which would allow me to include them for certain functionalities in my workflow without major hassle. you should hear more in a month time from now, after some travelling and after i set up a new machine and coded & tuned the new workflow. Quote Link to post Share on other sites More sharing options...
adan Posted June 16, 2019 Share #8 Posted June 16, 2019 Digital Ms apply some correction for vignetting in the camera. Especially the "color vignetting" or "purple edges" caused by rangefinder lenses wider than 35mm on digital sensors, but also optical vignetting overall. It is not stored as a "profile" in metadata, but is actual permanent changes to the pixel brightness values. At higher ISOs, the changes and corrections are reduced to avoid excessive brightening and the associated amplification of noise. Thus it is "OS-and-application-agnostic" - it will already be there regardless of what is used for post-processing. The camera must know which specific lens and approximately which aperture was used to do this, thus the requirement for the 6-bit coding of lenses (or their selection and identification via the menus (tricky with the M10-D)), and the secondary external light-sensor on the front near the shutter dial, which compares actual scene brightness to the brightness coming though the lens to estimate f/stop. Quote Link to post Share on other sites More sharing options...
dkmoore Posted June 16, 2019 Share #9 Posted June 16, 2019 3 hours ago, fenykepesz said: that's a very good start, thank you, frame-it and helohe ! i wasn't familiar with these software. let's see if they also allow a command line driven processing of photographic data which would allow me to include them for certain functionalities in my workflow without major hassle. you should hear more in a month time from now, after some travelling and after i set up a new machine and coded & tuned the new workflow. Serious question and not sarcasm/anything else. It seems you have very special needs for your photo editing. A level that is far more sophisticated than I. Can you explain what the advantage or uses are of photo editing to this degree of control and detail? Quote Link to post Share on other sites More sharing options...
pico Posted June 16, 2019 Share #10 Posted June 16, 2019 38 minutes ago, dkmoore said: Serious question and not sarcasm/anything else. It seems you have very special needs for your photo editing. A level that is far more sophisticated than I. Can you explain what the advantage or uses are of photo editing to this degree of control and detail? I realize you were not asking me, but I wonder if the OP is doing anything not possible outside of Linux. Before retiring I did some command-line work with Photoshop underneath Windows & Mac OS. One advantage was that it was necessary for unattended batch processing. It worked very well even over a network. More info if there is any interest. 1 Quote Link to post Share on other sites More sharing options...
fenykepesz Posted June 16, 2019 Author Share #11 Posted June 16, 2019 my gibberish is actually much less sophisticated than it may sound : over the years i had to use linux i became quite familiar with, and also needed to program a lot. ultimately i wrote my own photo-managing software fm.py, kind of a database system, which makes use of other software such as GIMP (the Photoshop for linux). so, i need to find solutions for the following scenarios, smoothly embedded in my workflow with fm.py, all running on one single platform, linux in my case : - 1) import the M10-D's RAW/DNG files using rawpy(?), convert to TIFF, store in database and on linux filesystem. - 2) find ways to correct for the lens' geometric distortions, vignetting and chromatic aberrations, preferentially in Python, or, less ideal, with programs invoked via system calls by my fm.py script. - 3) c2g : like many others, i started photography with FP, HP and Tmax films, and still intend to convert some color photos to b&w, but i am reluctant to do such manually with desaturation and similar processes. now, i am absolutely thrilled by GIMP's 'neuronal' data-driven c2g algorithm and i am looking for ways to invoke its code from within my fm.py code if possible. below some links about this not so young technique - but if some of you are aware of other non-manual c2g techniques than i will be more than happy to hear about those ! https://jcornuz.wordpress.com/2009/05/30/could-this-be-the-ultimate-black-and-white-converter/ http://buettner.to/blog/index.php/2012/07/24/more-c2g-experiments/ PS: i am not against 'manual' processing, actually the M10-D forces me to do such manual work which i love, but for the conversion of color to grey i prefer something non-hypothesis-driven, independent of my mind and mood. it's like using an old-style colorfilter that does all the job, and there is nothing to change about after the shot is done. that's why i considered getting a monochrome camera but then only as an M10-M (which may or may not come to the market). 2 Quote Link to post Share on other sites More sharing options...
dkmoore Posted June 16, 2019 Share #12 Posted June 16, 2019 (edited) 4 hours ago, fenykepesz said: my gibberish is actually much less sophisticated than it may sound : over the years i had to use linux i became quite familiar with, and also needed to program a lot. ultimately i wrote my own photo-managing software fm.py, kind of a database system, which makes use of other software such as GIMP (the Photoshop for linux). so, i need to find solutions for the following scenarios, smoothly embedded in my workflow with fm.py, all running on one single platform, linux in my case : - 1) import the M10-D's RAW/DNG files using rawpy(?), convert to TIFF, store in database and on linux filesystem. - 2) find ways to correct for the lens' geometric distortions, vignetting and chromatic aberrations, preferentially in Python, or, less ideal, with programs invoked via system calls by my fm.py script. - 3) c2g : like many others, i started photography with FP, HP and Tmax films, and still intend to convert some color photos to b&w, but i am reluctant to do such manually with desaturation and similar processes. now, i am absolutely thrilled by GIMP's 'neuronal' data-driven c2g algorithm and i am looking for ways to invoke its code from within my fm.py code if possible. below some links about this not so young technique - but if some of you are aware of other non-manual c2g techniques than i will be more than happy to hear about those ! https://jcornuz.wordpress.com/2009/05/30/could-this-be-the-ultimate-black-and-white-converter/ http://buettner.to/blog/index.php/2012/07/24/more-c2g-experiments/ PS: i am not against 'manual' processing, actually the M10-D forces me to do such manual work which i love, but for the conversion of color to grey i prefer something non-hypothesis-driven, independent of my mind and mood. it's like using an old-style colorfilter that does all the job, and there is nothing to change about after the shot is done. that's why i considered getting a monochrome camera but then only as an M10-M (which may or may not come to the market). Thanks for the response and explanation. That is certainly a lot of work but if you want/need the control than I can see why it makes sense. I read the article and I could see myself going down that path someday when I have more time. Also, from Pico's response, it seems like you can process batches much quicker using these tools. Although the multiple systems seems like it would slow things down unless you only need to set up once and then it all runs automatically. Would it kill my MacBook Pro or iMac (speed wise)? Edited June 17, 2019 by dkmoore Quote Link to post Share on other sites More sharing options...
fenykepesz Posted June 17, 2019 Author Share #13 Posted June 17, 2019 (edited) you know, dkmoore and pico, for me it's not so much about batch processing than rather about avoiding having to switch between OSs and software and using one single platform, one software and so on. i am aware (my son tells me so too!) that on other platforms this all is easily achieved but since i am married to this linux thingy, i am left doing things myself. regarding article, dkmoore, did you mean the old c2g links ? if yes, i show here a first very simple low-quality color picture i created with my 9 year old Nokia N900 phone... Welcome, dear visitor! As registered member you'd see an image here… Simply register for free here – We are always happy to welcome new members! ...and a second picture processed with Gimp's c2g tool plus some minimal adjusting of the contrast curve. i am not sure how i would come to this result without crazy much manual work on the entire picture and on local details using PS, Gimp or whatever. i am really keen to know if there are similar or more advanced c2g functions out there. this current one implemented in GIMP/GEGL is more than a decade old. perhaps this c2g topic should become a new thread, sorry about that... Edited June 17, 2019 by fenykepesz Quote Link to post Share on other sites Simply register for free here – We are always happy to welcome new members! ...and a second picture processed with Gimp's c2g tool plus some minimal adjusting of the contrast curve. i am not sure how i would come to this result without crazy much manual work on the entire picture and on local details using PS, Gimp or whatever. i am really keen to know if there are similar or more advanced c2g functions out there. this current one implemented in GIMP/GEGL is more than a decade old. perhaps this c2g topic should become a new thread, sorry about that... ' data-webShareUrl='https://www.l-camera-forum.com/topic/298347-reading-m10-d-rawdng-files-with-linuxpythongimp/?do=findComment&comment=3761006'>More sharing options...
dkmoore Posted June 17, 2019 Share #14 Posted June 17, 2019 (edited) I'm impressed with what you did here.. I do think you will find the images from the M10 far closer to the end result (stating the obvious). So, your results will be that much better. Editing took care of the haziness in the background very nicely and surprising detail in the shadows from an old phone image. With the understanding that this isn't a "show image" it would seem whatever you are doing with GIMP/GEGL works. You didn't ask this but in response to your comment wanting a single system I use primarily LR and Google NIX, which is essentially one system because the NIX apps are plugins. And the latest iteration of LR is significantly improved. But, this would force you to abandon LINUX which is a non-starter it seems. Edited June 17, 2019 by dkmoore Quote Link to post Share on other sites More sharing options...
pico Posted June 17, 2019 Share #15 Posted June 17, 2019 (edited) Mac OS is unix, very much BSD Unix. It has a command line used via the app called Terminal. $ cd .. $ pwd /Users $ ls Guest Shared pico molly $ cd Shared $ ls Adobe Library SC Info iMazing linhof-grips Capture One PDF Expert SharedScript.pl iMazing Mini molly Gemini 2 Pictures adi iSkysoft.plist $ Edited June 17, 2019 by pico 1 Quote Link to post Share on other sites More sharing options...
fenykepesz Posted June 18, 2019 Author Share #16 Posted June 18, 2019 you are perfectly right, pico, that's why scientists and alike can work with apples as they run under the hood unix. btw, i prefer emacs (over nano, pico, atom, vi, etc) and dkmoore, i didn't do much for that above picture, it's all to the credit of that c2g tool i mentioned ! now, after so much sidetracking, back to the M10-D - in some three weeks and some hands-on experience (i didn't see my new camera yet!) i will rephrase my questions about importing and processing its DNG/RAW files... Quote Link to post Share on other sites More sharing options...
pico Posted June 18, 2019 Share #17 Posted June 18, 2019 1 hour ago, fenykepesz said: you are perfectly right, pico, that's why scientists and alike can work with apples as they run under the hood unix. btw, i prefer emacs (over nano, pico, atom, vi, etc) Welcome to GNU Emacs, a part of the GNU operating system. Type C-l to begin editing. Get help C-h (Hold down CTRL and press h) Emacs manual C-h r Emacs tutorial C-h t Undo changes C-x u Buy manuals C-h C-m Exit Emacs C-x C-c Browse manuals C-h i Activate menubar F10 or ESC ` or M-` (`C-' means use the CTRL key. `M-' means use the Meta (or Alt) key. If you have no Meta key, you may instead type ESC followed by the character.) Quote Link to post Share on other sites More sharing options...
DaveSee Posted June 24, 2019 Share #18 Posted June 24, 2019 (edited) Greets! Please PM me as posting Linux processing may not serve many of us I've been processing Leica, Ricoh, Olympus, Canon & Nikon files with Linux for prolly 15 years... but not with the Linux tools you use. I'd like to help, and even learn from you too. Please see attached... Oh! & this could run on a Mac... Winders, not sure. Nothing wrong with Adobe & CaptureOne apps... I don't have the patience to sit & tweak individual images: guess I'm yet a film dinosaur... frame, shoot, soup & print... so, that's my Linux scripts' inspiration. Okay, occasionally I'll wade in on one, or three images to tweak in unison, Dave On 6/15/2019 at 5:31 PM, fenykepesz said: any input greatly appreciated ! bye, pisti Quote ...have attached my recent M10D Linux processing script... Sure, may be inefficient & krufty, but it works for me. m10dng-clean Edited June 24, 2019 by DaveSee added information & support Quote Link to post Share on other sites More sharing options...
Xavier Posted June 24, 2019 Share #19 Posted June 24, 2019 I do not have an M10-D, so I have no direct way of checking whether it works, but how about "dcraw" ? I did not find any mention of it in the thread and am a bit surprised; I used it to do batch processing from the command line about 10 years ago, and it seems to have kept adding new cameras since then. If yours is not supported, you may be able to find out how to add support for it (you have to look at the source code and tinker a bit) or maybe send a file to the author. Quote Link to post Share on other sites More sharing options...
fenykepesz Posted July 17, 2019 Author Share #20 Posted July 17, 2019 i summarize here briefly my experiences in setting up a data import and processing pipeline for Leica M10-D RAW data using linux and related opensource tools. it was actually pretty straight forward once i had the camera in my hands. 0) SD card : partitioning with fdisk with type 'b' for W95 FAT32 followed by mkfs.vfat. once i found out that the metal bottom part needs to be attached and locked before using the card i had no problem whatsoever with a 4GB and two 32GB cards. 1) data transfer : i simply move the DNG files from the SD card to my computer with the 'mv' command in a shell terminal. 2) library : libraw needs to be in place. LibRaw-0.19.3 works fine with the M10D DNG files. 3) reading and converting RAW/DNG : i use python 3.7.4 with rawpy 0.13.1 with imageio 2.5.0 which reads the approx 25-30MB large DNG files and saves them as TIFF in a few seconds. 4) reading DNG metadata : i use for that venerable ExifRead 2.1.2 which seems to do the job just fine in python3. 5) command line tools for DNG : 'display' does a good job (part of the ImageMagick package 6.9.10-53) for quick viewing. for glancing at the DNG metadata i use EXIF.py or exiv2. 6) image processing : my main tool for that is GIMP 2.10.4 but i admit i truly use just very simple and very few techniques or tools for manipulating my photos. 7) c2g, or color to gray conversion : that's a big topic i like to address separately... 😎photo and database management : i wrote for that my own tool called fm.py which outputs for example also the HTML pages for my photo websites. i did not address yet the question if the M10D itself does the lens specific distortion corrections for the DNG files, or if one needs to do that separately offline. that's it - i am pretty relieved as my whole M10D data 'processing pipeline' operates now in my compute environment of choice and i don't need to rely on other and potentially pricey software. but then again - my needs are pretty simple, i am just a hobby photographer. 1 Quote Link to post Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.