8 bits versus 16 bits

[t024484]

Advertisement (gone after registration)

I started a new thread as a succesor to "the reality of Leica 8 bits", and wanted to show the first results of my SQRT compression as promised.

Jaap was so nice to send me several 16 bits pictures from his DMR.

 

With my first attempt, I compressed all 16 bits words as: y = INT(SQRT(x)+0,5)

Then I expanded back with: z = INT(y*y + 0.5).

Both calculations exactly like what Leica is doing for the M8.

I assume that the DMR has a 14 bit A/D converter like the M8 seems to have.

The DMR shifts the 14 bits to the 14 MSB positions within the 16 bit word, and the M8 lets them decompress in the 14 LSB positions, but this is meaningless.

 

I can very well see differences between the original and the compressed one, although I have to be carefull because not everything has been checked for the full 100%..

 

Look at the green side board of the wagon. The compressed one is more grainy.

Somewhere else in the picture is a red shield with white text. This red is also grainy compared to the even red original. The grain effect is caused because less steps are available to represent gradual colour changes.

 

 

First the original

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

 

and then the compressed one.

 

 

when I am ready with my logarithmic compression, I will try to make the pictures available in full size, ready for download.

 

Hans

[Guest jimmy pro]

Jaap was so nice to send me several 16 bits pictures from his DMR.

 

 

Both calculations exactly like what Leica is doing for the M8.

 

Just a question. Since Mr. (or is it Dr.) Jaap has both, why not just take the same shot with the M8 and DMR? At least that way nobody could point the finger at your methods as contributing to the affects. Maybe someone could borrow him an R-M adapter so he could use the same lens and that wouldn't be a factor to argue either.

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[jaapv]

It sure looks a lot like the difference I see between DMR and M8 shots....

I can do that, Jimmy, but I got a lot of flak for doing so in another thread. I can redo, using the same lens, as I have the adapter.

[mjh]

I assume that the DMR has a 14 bit A/D converter like the M8 seems to have.

Actually it’s a 16 bit converter, so you might want to get rid of the two least significant bits first (x = 4 * INT(x / 4)).

[marknorton]

In your expansion equation, since Y is an integer, the square of it is an integer so adding 0.5 and taking INT() is not necessary.

 

Your results are certainly very interesting and I think you should use the translation table in an M8 DNG to make it as close to the M8 as possible.

 

Nevertheless, a big difference between the images and unless you find a flaw in your working, seems to show that Leica were (as we would say in the UK) clutching at straws when they said there's no loss due to the compression.

 

Interesting work, I'll be interested in your further conclusions.

[sdai]

Just a question. Since Mr. (or is it Dr.) Jaap has both, why not just take the same shot with the M8 and DMR? At least that way nobody could point the finger at your methods as contributing to the affects. ...

 

Still, you'll be comparing apples to oranges because the sensor responses won't be the same even if you put them on the same settings, without even considering the downstream processing. Before you even bother to think about 8 bit or 16 bit the baselines are different.

 

The only valid test would be the one Michael has done before, that's to run 8bit firmware against 16bit firmware in the same camera.

[jaapv]

Advertisement (gone after registration)

I would say that this is a valid test, as the same file is used for both algorithms.

[sdai]

I would say that this is a valid test, as the same file is used for both algorithms.

 

I'm not so sure about that, Jaap. Because you don't have the sensor dump file, you only have the 16 bit DNG from the DMR and massage it into 8 bit with your algorithm, that's not the same as forming a 16bit DNG or 8 bit DNG from the sensor dump.

[t024484]

Still, you'll be comparing apples to oranges because the sensor responses won't be the same even if you put them on the same settings, without even considering the downstream processing. Before you even bother to think about 8 bit or 16 bit the baselines are different.

 

The only valid test would be the one Michael has done before, that's to run 8bit firmware against 16bit firmware in the same camera.

 

The thing that has been under investigation here, is simply to get an answer to the question: does compressing according to a SQRT produce visable negative side effects.

As a first indication, it looks as if the answer is yes.

 

I'm not so sure about that, Jaap. Because you don't have the sensor dump file, you only have the 16 bit DNG from the DMR and massage it into 8 bit with your algorithm, that's not the same as forming a 16bit DNG or 8 bit DNG from the sensor dump.

As far as I know, a DNG is a digitalized sensor dump. I do not understand what you mean with a sensor dump.

 

In your expansion equation, since Y is an integer, the square of it is an integer so adding 0.5 and taking INT() is not necessary

Mark you are right, this is still a leftover from my reconstruction of the 14 bit M8 file. I will remove it.

Actually it’s a 16 bit converter, so you might want to get rid of the two least significant bits first (x = 4 * INT(x / 4)).

I will certainly do that as another variant to the original DMR file.

[rosuna]

I always thought this was the case. I think a compression to 256 steps is too much. Nikon employs the same tonal compression, but to 10 bits instead of 8. It provides enough margin.

[luigi bertolotti]

I always thought this was the case. I think a compression to 256 steps is too much. Nikon employs the same tonal compression, but to 10 bits instead of 8. It provides enough margin.

 

After having reviewed this and the other related thread, I tend to agree... months ago, I remember a thread in which someone (don't remember who) said that more than 256 step is unuseful for human eye cannot resolve superior resolutions ... maybe this is true fro grayscale, but I start to think that with wide hue ranges in a certain pic, more than 8 bit DOES give some more... and I imagine that in a well processed print this can even be more evident. BTW, there must be some reason for pro MF digital backs, generally, all are based on a 16 bit raw.

[ammitsboel]

What are we looking at on the screen?

Since almost no monitor can truly show 16bit and the browser probably does not show higher than 8bit then what are we seeing?

Do the 16bit file look different because of they are showed somehow wrong on equipment that can't show 16bit? And is the reduction to 8bit dithered or just truncated?

 

From just looking at the two examples I agree with the grain, but the 16bit is more noisy/unclear and I wonder why?

[sdai]

As far as I know, a DNG is a digitalized sensor dump. I do not understand what you mean with a sensor dump.

 

What I mean is you are simply wasting your time here.

 

What you do is post-processing a DMR 16-bit DNG into a 8 bit DNG, this is different from forming a 16 bit DNG directly from a sensor dump and/or forming a 8 bit DNG directly from the sensor dump.

 

What do you want to prove? your compression algorithm is good or bad?

[jaapv]

I'm not so sure about that, Jaap. Because you don't have the sensor dump file, you only have the 16 bit DNG from the DMR and massage it into 8 bit with your algorithm, that's not the same as forming a 16bit DNG or 8 bit DNG from the sensor dump.

I hear what you are saying, but I think your argument invalidates Michael's position more than it does Rob's. After all, we cannot be sure that the sampling of the sensor dump is identical in a 8-bit camera to a 16-bit. In Rob's approach we can be sure that the only variable is the compression. Single-parameter experiments have the highest level of validity.

To summarize as I see it: your argument is: as compression takes place in DNG, we must use data as they are before DNG, i.e. the sensor dump. My argument is: this is basically correct, but in the worst case this may corrupt the experiment as we do not know what happens to the sensor dump and at best irrelevant as we only want to demonstrate the effects of compression, which happens within DNG.

[sdai]

My argument is: this is basically correct, but in the worst case this may corrupt the experiment as we do not know what happens to the sensor dump and at best irrelevant as we only want to demonstrate the effects of compression, which happens within DNG.

 

Put it this way, Jaap, you can open up a 16 bit TIFF in Photoshop and immediately save it as a 8-bit TIFF, and DNG is based on TIFF so what are you proving from this?

[GarethC]

You people are way smarter than me

 

I much prefer the thread on the plastic film on the bottom of the new camera.................

[chocy0924]

I am more interested in maliability of the tone curve between 8bit and 16 bit raw file. I believe 8bit and 16bit makes more difference in this regard. I really do wish M8 had an option to record 16bit file.

[diogenis]

After having reviewed this and the other related thread, I tend to agree... months ago, I remember a thread in which someone (don't remember who) said that more than 256 step is unuseful for human eye cannot resolve superior resolutions ... maybe this is true fro grayscale, but I start to think that with wide hue ranges in a certain pic, more than 8 bit DOES give some more... and I imagine that in a well processed print this can even be more evident. BTW, there must be some reason for pro MF digital backs, generally, all are based on a 16 bit raw.

 

Yea, the human eye is most accurate in green, then less in red and blue. And gradations in each color are in the range of 256. Combined all these 3 colors (RGB) give of course a range of millions of colors in the end, but the 2 photos presented do differ a lot. Either the procedure is wrong, or we are missing key elements to evaluate...

Maybe stick to B&W only? and redo the thing?

LFI stated that you strip the LSB of a 16b sensor signal, then x4 then you SQRT and look in that table..

[jaapv]

Put it this way, Jaap, you can open up a 16 bit TIFF in Photoshop and immediately save it as a 8-bit TIFF, and DNG is based on TIFF so what are you proving from this?

Rob is doing something different. He is applying the DNG lookup table to compress to 8 bits, just like the M8. The file is supposed to behave like a 12 bit file then.

[Advertisement]

Hallo,

Look here: Leica M

[fefe]

Put it this way, Jaap, you can open up a 16 bit TIFF in Photoshop and immediately save it as a 8-bit TIFF, and DNG is based on TIFF so what are you proving from this?

 

Then you would be looking at the effect of the algorithm used by photoshop to do 16->8bits with TIFF, here he is showing the impact of the algorithm used by Leica to go from 16->8 bits which is possibly different from the one used by PS to compress your TIFF (is PS method public ? Is it just stripping LSB ? Or using a SQRT as this ?). This would actually be a valid comparison to do, to see which ones creates the most artifacts.

 

All 16->8 bits conversions are not equal in term of visual impact, and all this is showing to me is that this method has a significant visual impact, but I'd like to see it compared to the "dumb" method of stripping LSB and eventually other existing converters.