Secrets of the blue dot revealed! (and other M8 metadata)

[cbretteville]

Advertisement (gone after registration)

There has been a lot of speculation about the metadata stored in the M8 images as they come off the memory card. There have been a fair amount of chat around it here and I guess other places too. It is time to unveil what the blue dot does and what is stored in the unpublished maker notes as of version 1.110 of the M8 firmware.

 

Disclaimer: This work has not been endorsed by Leica, there can be no guarantees as to the correctness of the information. It can also change at any time.

 

The list is - drum roll:

 

Information stored in both file formats:

• G-Stop, the guesstimated aperture
  
  
• Blue dot brightness value (EV)
  
  
• TTL metered brightness (EV)
  
  
• Lens ID and frame selector position
  
  
• White balance preset
  
  
• Selected user profile
  
  
• Camera serial number
  
  
• System operating temperature (Centigrade)
  
  
• Colour temperature (Kelvin)
  
  
• Unique image ID
  
  
• CCD ID
  
  
• CCD circuit board ID
  
  
• Controller board ID
  
  
• M16C ID

Additional information in DNGs:

• UV/IR filter vignetting/cyan drift correction

Additional information in JPEGs:

• JPEG file quality
  
  
• As shot neutral, Red
  
  
• As shot neutral, Green
  
  
• As shot neutral, Blue

A lot more information can be found in my report, "Metadata in the Leica M8 image files". The report is the result of analysis done on a substantial number of image files captured by myself and others using M8 cameras. Program code has been written to do the actual decoding of the metadata and this output has been stared at a lot...

 

The first three sections of the paper should be of interest to most here. Parts four through nine are rather technical and will cause the eyes of a lot to slowly glaze over. Might be an excellent remedy if you're suffering from insomnia.

 

A first version of this report was sent to Leica in June. For general publication I wanted to include a discussion of the g-stop and its accuracy. That has been added - amongst a lot of other things. This version of the paper has, by the way, also been sent to Leica.

 

Why do this at all? Does it have any practical application? For most M8 shooters the answer is 'no – not directly'. I started this project because of the challenge of finding and deciphering all the hidden information in general and in particular because I wanted to see how accurate or inaccurate the g-stop data was. Others will hopefully take this information and put it to good use that will benefit the general M8 shooter.

 

As a first, Sandy McGuffog utilizes it in CornerFix starting with v0.9.1.5 (released at the same time as my paper is published). CF now has a feature that sets the aperture value in the public EXIF block to that of the g-stop. It also adds a LensName field to the XMP block with the name of the lens used if a shot was taken with a coded lens and lens detection was enabled.

 

I've got to thank Scott Kirkpatrick and Sandy McGuffog for their contributions. In particular Scott deserves credit for his help working with the drafts through both versions of this report.

 

By now you're wondering: where do I find this paper? Here: the m8 metadata project

 

Enjoy!

- Carl

[orjanf]

Thank you, impressive work and also very interesting.

A commented assembly code listing of the whole firmware to follow?

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[marknorton]

This is a very impressive piece of work and Carl deserves every credit for both doing an extensive "forensic" analysis of EXIF data and especially for sharing it with us. Thanks also to Sandy and Scott for their input.

 

I can imagine that Leica never thought anyone would get into it to this level of detail. I've been writing a DNG reader and Carl's information answers a number of "I've no idea what that is!" questions.

 

Interesting to see that the aperture guesstimation is actually not bad, better than I expected.

 

Thanks again Carl, along with Sandy's CornerFix and the work Scott did on sensor vignetting, three very interesting pieces of work, freely given for the benefit of the LUF community.

[peterv]

Thanks Carl (and Sandy and Scott)

 

Very interesting to read and very well explained!

 

Hope Leica 'adopts' this and mere mortal everyday users like me will be able to read the working aperture from the EXIF in the near future.

 

Well done sir!

[carstenw]

Great, I really look forward to reading and applying this!

[R.Costello]

Carl,

You have done all of us in this Forum a great service with this "Report".

Simply fascinating and very illuminating information.

Thank you

 

Richard

[Shootist]

Advertisement (gone after registration)

Thank you. Very interesting.

[rosuna]

Congratulations, and thanks.

[sean_reid]

Way to go Carl.

 

Cheers,

 

Sean

[barjohn]

I want to add my thanks for your hard work and efforts and to say "Well Done!!" Any chance we can get the complete decoder program that gives all of the maker's data. I certainly intend to batch process my images to add the information you add to the public EXIF file. It would be interesting to see how many variations there are in CCD, main board and CCD board and their serial numbers. It would also be interesting to see how they correlate to different issues we have seen here. Thanks again. Great job and excellent paper.

[scott kirkpatrick]

I want to add my thanks for your hard work and efforts and to say "Well Done!!" Any chance we can get the complete decoder program that gives all of the maker's data. I certainly intend to batch process my images to add the information you add to the public EXIF file. It would be interesting to see how many variations there are in CCD, main board and CCD board and their serial numbers. It would also be interesting to see how they correlate to different issues we have seen here. Thanks again. Great job and excellent paper.

 

Carl's probably asleep, so I'll answer this one. The fields that seem to be the component IDs are the most speculative piece of detective work. There are four fields, three of them always zero and the fourth of them apparently always equal to one in all of Carl's data. Since they didn't change, they weren't identifiable. A plausible guess by several people was that they could be the four components that are called out in the XML logic at the start of the M8 firmware. Of these, three have always been zero, and the fourth, the CCD-Board-ID, was 0, then 1 after the hardware recall, and just recently has been raised to 2. Carl didn't have any files from Leica M8s manufactured in the last month, but he did have a few which were taken on a pre-recall M8. And for those, the fourth field was 0. So that's the evidence to date.

 

As for code to read all this, watch for Sandy's next CornerFix release... It's open source.

 

scott

[barjohn]

Perhaps if those of us with a variety of serial numbers sent in an image with the DNG/EXIF data he could look to see if there are any differences.

 

I saw the source is on line but I haven't had a chance to look at it yet to see if it addresses those particular fields.

[sm23221]

Great job, interesting comprehensive study.

[Philinflash]

Excellent forensic examination!

 

Perhaps the blue dot only sees blue light (or has a heavy bias towards blue) and that contirbutes to the differences with a full-blown through-the-lens RGB image.

 

Is there any possibility of making the pdf file available at a higher resolution? I am sure you were trying to save file size but I have trouble reading some of the finer stuff.

 

Philip Kozloff

[philipotto]

Is there any possibility of making the pdf file available at a higher resolution? I am sure you were trying to save file size but I have trouble reading some of the finer stuff.

 

Philip Kozloff

 

Philip,

The text of the pdf is vector based: zoom in to whatever size is comfortable for your viewing.

[guidomo]

Great great work, many thanks!

 

I'll be very unscientific and conclude the following:

 

"When using Sandy's formula, the probability of the aperture estimate being within +/- 0.5 f-stops is around 80%. The probability of it being within +/-1 f-stop is almost 95%."

 

(from top figure on page 12.)

[sdai]

I'm sure next time Leica will figure out a way to encrypt the metadata like Nikon does to their NEF files. LOL

 

But seriously, Carl ... you should use these "unknown maker notes" to make some money, IMO these manual 6-bit code hand job to emulate officially supported Leica lenses sucks big time, you could buy a set of DXO analyzer or the Imatest software written by Norman Koran, map the characteristics of the ZM, CV lenses by yourself and inject your own code into a raw converter.

[c6gowin]

Very interesting. Thank you Carl, Sandy, and Scott for all the hard work. The paper is well written.

 

Out of curiosity, I ran Corner Fix in batch mode to update the EFIX data only (no lens profile loaded) for the set of 75mm Summarit lens resolution test photos I posted earlier today. The updated EFIX correctly identifies the Summarit lens but the reported G-stop is incorrect. The reported G-stop was 4.7 for an actual f-stop of 2.5 and incremented essentially relative to the actual f-stop as it was stopped down for the lens test. The G-stop inaccuracy for my test photos could be due to the 75mm focal length versus the theorized blue dot fov as discussed in the paper. Of course there are many other variables that could have caused the offset in G-stop versus f-stop.

[marknorton]

This comment at the start of the 1.10 UPD:

 

<!-- Update aller Komponenten nur moeglich, wenn CCD-Id = 0, CCD-Board-ID = 0..2, Controlboard-Id = 0

und M16C-ID = 0 -->

 

"Update of all components only possible if..."

 

shows that there is only one version of 3 of the 4 tracked hardware components; there are three versions of the CCB Board, 0 originally, 1 after the hardware recall and 2 (presumably) on most recent cameras after the undefined board change.

 

I sent Carl a very early image, before the camera went back to be upgraded, and all four fields are 0. All his images show a CCD Board ID of 1 and only those with the most recent cameras will likely show a CCD Board ID of 2.

[Advertisement]

Hallo,

Look here: Leica M

[Mark_Schretlen]

The updated EFIX correctly identifies the Summarit lens but the reported G-stop is incorrect. The reported G-stop was 4.7 for an actual f-stop of 2.5 and incremented essentially relative to the actual f-stop as it was stopped down for the lens test.

 

So when the Summarit is set at f/11, the G-stop estimates it at f/18? Even though it is off by almost two stops, I still think it is better than Leica's current method of reporting an f/11 f-stop as f/2.5 in the EXIF.