8bit DNG issue explained

[Jamie Roberts]

Advertisement (gone after registration)

{snipped}Yeah, the R glass is quite something, isn't it?{snipped}

 

...and it's a zoom no less!

 

And yes, though this is the m8 forum, I do love my R glass too

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[sandymc]

Good article. Two comments -

 

a) The actual formula that that Leica use does have a square root - its actually SQRT((14-bit data)*4)

 

For those confused by the seeming discrepancy between (1) the highlights being more compressed and (2) the prediction that underexposure will be more difficult to recover from, the explanation is that highlights have a LOT more levels to them in the first place. The "lightest stop" has 8192 levels, the next 4096, etc. So even if you are compressing the lightest stops by more that the darker, because you started with so many more levels you still end with more. Specifically, using the article's notation, stop 1 - 74 levels after compression, stop 6 - 14 levels after compression

 

Sandy

[wparsonsgisnet]

I thought the article was very illuminating.

 

It leads me to ask the same question we've been asking in other posts:

Can Leica change the software to save the dng without any compression?

 

We all *think* that an uncompressed file wil yield even better images than this packed/unpacked version. If this is all being done in software (maybe not so bad, if we can have options), then it would be nice if Leica would give us a menu choice for compression and no compression.

 

I would gladly give up battery life and accepts longer write times and bigger files on the SC cards, if this yielded better images.

 

In reading Max's article, am I correct in inferring that the processed (both compressed and then uncompressed) image will lose detail but not necessarily have reduced noise?

[sandymc]

In answer to the question re can Leica change the firmware to uncompressed - yes, no question they could. However, there could be a very substantial slow-down in the M8's operations. Currently, all you need to write an M8 DNG are 8-bit operations. As soon as you go to anything more than what Leica do now, even Nikon style 9 1/4 bit bit compression, you're into 16 bit writes, and potentially twice the time taken to write an image....

 

Sandy

[wparsonsgisnet]

Yup, I figured that. Twice the write time, half the card capacity, and like that.

 

If the image we end up with is better, it would be nice to have an option in the menu.

[rwfreund]

Actually, the Leica dng format increases a certain type of noise, that being quantization noise at higher luminance levels. The question, and I think their assertion, is that this increase in quantatization noise has negligible impact on real photographs. If this means the cusp of blown-out highlights, then I would tend to agree.

Digital sensor noise, however, is more prevalent, or more noticeable, at lower luminance levels since some of that noise, either sensor or amplifier, is constant and independant of luminance, thus it is a lower percentage of the sensor's digital output at high levels than at low levels. The "real" number of bits you get is a function of how many significant bits there are. Usually, this means how many bits are meaningful given acceptable noise thresholds. As the noise increases, as when you goose the sensor, effectively you reduce the number of significant bits.

All digitization introduces quantitization noise either linear or non-linear. a two bit code, for example, can express only four values, and input that is on the boundary between any of these values can be encoded either way. So, for our two bit sensor example, small random noise will look like a 25% error since it has only one of four values to chose from.

 

I think that the major damage done by Leica's choice of encoding, is not to our pictures, but to their press. There is a huge amount of FUD (Fear, Uncertainty, and Doubt), caused by their detractors over this issue.

-bob

[barjohn]

Advertisement (gone after registration)

I'm not convinced that it would take any more time to write the image (meaning from snapping the shutter to completimg the transfer to the SD card. The reason is that performing the SQRT and Multiplication functions in the processor are much more time comsuming than simple writes of an extra byte of data. This is especially true because each 14 bit value must be put in floating point format (assuming the computations are done in floating point) or if the processor is being asked to process these computations in decimal arithmetic they would be longer and slower routines but more accurate. Many register operations and memory operations (reads and writes) are required, all to process one 14 bit value as opposed to just writing an extra byte. You would pay a storage premium unless you used the type of lossless compression used by the Adobe DNG file.

[ho_co]

John--good point about the time needed just to write an extra byte as compared to floating point conversions etc.

 

But since the LUT stores just integers, don't you think maybe the M8 uses integer math throughout?

 

--HC

[scott kirkpatrick]

It's pretty clear that in both directions, encoding and decoding, the work is done by using an 8bit pixel value to point into a table of encoded values which are the result of the square root calculation. In the firmware, you can see some big tables which look like they do the encoding without any arithmetic, just one load and one store, and in the DNG file, you can see the LUT that provides the decoded values for 256 possible coded pixel values. So no arithmetic is needed.

 

scott

[marknorton]

I agree with Scott - the 12/14 bit values will be converted to 8 bit values using a look-up table - 16384 bytes of storage for 14 bit (nothing) - which is the only way they can deal with 10m pixels in a fraction of a second. Doing that many square roots in the same time is out of the question. Both this table - and the corresponding table put into the DNGs to reverse the transformation can be fine tuned at any time in the future to adjust the light/dark extremes.

 

Using this type of transform makes sense to me where there's a non-linear response (eyes and ears). If you think of CD, that uses linear quantisation and 50% of the available 65536 quantising values are used only for the loudest 6dB of sound, what a waste.

 

Much better to concentrate your available resolution on the lower levels, whether that's shadow detail in the M8 or a solo instrument in an orchestral piece.

[jaapv]

Please don't blow me out of the water for asking an imbecile question.

Can it be that the blue fringing around specular highlights is not sensor blooming but a posterization effect due to this non-linear compression?

[mjh]

I'm not convinced that it would take any more time to write the image (meaning from snapping the shutter to completimg the transfer to the SD card. The reason is that performing the SQRT and Multiplication functions in the processor are much more time comsuming than simple writes of an extra byte of data. This is especially true because each 14 bit value must be put in floating point format

There's no need to use floating-point format; simple 16 bit integer arithmetic suffices. Multiplying by 4, for example, ist equivalent to shifting by two bits to the left. Having said that, I agree with the suggestion that Leica is most likely using a simple look-up table to perform this computation. Spending a mere 16 KB for a LUT would make this calculation extremely fast.

[rwfreund]

Please don't blow me out of the water for asking an imbecile question.

Can it be that the blue fringing around specular highlights is not sensor blooming but a posterization effect due to this non-linear compression?

 

I really doubt it since this algorithm if it works the way we are all thinking has no bias toward one color or another, nor does it have any spatial bias, since it operates on the encoding of each pixel value.

 

That brings me to the question: Where is the Bayer interpolation performed?

 

-bob

[barjohn]

My point is that whether you use a lookup table or simple arithmetc, it is highly likely that it takes more time than writing a single additional byte. While I haven't looked up the characteristics of the processor and memory they are using, I would guess that it is not an 8 bit processor but either a 16 bit or 32 bit processor and memory. If it is either 16 bit or 32 bit it can write the two bytes or even 4 bytes at one time. If it has to take the 2 byte value, do a transform via a look up and then left shift it 4 bits, move it to a register, bring in another 2 bytes, left shift 4 bits, now combine with the first 8 bits to form two bytes and store, that has to take longer than just a read and store.

[scott kirkpatrick]

The shifting is taken care of within the lookup tables, and I am sure that whatever is the bus bandwidth through the Blackfin and to the SD card, it is fully utilized. Usually this kind of routine would be written to keep the bus busy, since that minimizes write time. Look at the number of pins on an SD card. I don't have one in my office right now, but I think there are only about 8. That may be the limiting point, if it means that all transfers are 8 bits at a time.

 

scott

[mjh]

My point is that whether you use a lookup table or simple arithmetc, it is highly likely that it takes more time than writing a single additional byte.

Not really. Table look-up is fast – just read the byte at some memory location plus an offset – and takes care of both the multiplication and the square root. Accessing dynamic RAM is obviously several times faster than writing to an EEPROM (the SD card). Cutting write time in half outweighs the small penalty of doing the table look-up.

[sandymc]

That brings me to the question: Where is the Bayer interpolation performed?

 

-bob

 

Bayer interpolation is performed in the RAW processor if you shoot raw, so C1 or Lightroom or whatever you choose. For JPGs, in the M8 firmware.

 

Sandy

[barjohn]

I was only thinking in terms of write to buffer memory since the SD card write occurs in the background while you are taking the next shot.

[Advertisement]

Hallo,

Look here: Leica M

[mjh]

I was only thinking in terms of write to buffer memory since the SD card write occurs in the background while you are taking the next shot.

Leica is trading one memory access for another, so there shouldn’t be a speed advantage either way. The buffer holds twice as many images with compression, though, so compressing to 8 bits still pays off.

[rosuna]

I think 8-bit compression is a solution too radical. Maybe uncompressed 16-bit files is too problematic for the M8 processor, and unnecessary. I would like to see implemented a middle solution, something like 10-bit non-linear compression. You get more degrees of freedom, and a safe margin. The Nikon's uncompressed NEF files from the D200 (12-bits A/D) weight 16MB (DCRP Review: Nikon D200). The compressed NEF is about half that size (Jeffrey Friedl’s Blog » A Qualitative Analysis of NEF Compression), this is, about 8MP. A 10-bit non-linear compression applied to Leica's DNG files would lead to a file size not much larger than 10MP. I don't care if the camera slow down a bit. It can be applied to the DMR too (the new firmware will ship soon).

 

You can see the effects of Nikon's compression here:

 

Jeffrey Friedl’s Blog » A Qualitative Analysis of NEF Compression

 

R.