Jump to content

The effects of compressing to 8 bits


Recommended Posts

Advertisement (gone after registration)

Some time ago,I performed some tests with uncompressed 14 bit files from a DMR, that where compressed to 8 bits and back to 14 bits to asses the effects of the Leica M8 compression and to find out if the compression algorithm used by Leica for the M8 was the best possible.

 

Now I aproached the matter from a mathematical side, and came to some conclusions that I would like to share.

 

Picture noise is composed of several elements, the most important being referred to as:

N0, being the sensor read noise ,

N1, noise of the analog circuits after the Sensor plus A/D converter

P, the shot noise and

W, the noise caused by gain difference between pixels

 

We have the factor g, expressing the relation between photons captured on the sensor, and the value S from the A/D converter.

Shot noise P" in photons is the square root of S", the total amount of captured photons, this is because of a poisson distribution of S".

Because of the factor g, after the sensor we get P=Sqrt(S/g), with P=P"/g and S=S"/g

 

Not going into too much detail, I calculated the signal to noise in EV or stops at ISO 160 as a function of luminosity in EV or stops.

For more detailed background, read Noise, Dynamic Range and Bit Depth in Digital SLRs

I did this without compressing to 8 bits, and also with the Leica M8 compression algorithm.

 

The compression that Leica uses before storing data on the SD card is: Multiply the 14 bit output with 4 to get 16 bits.

Then take the SQRT and the Integer of this number, to get 256 values between 0 and 256.

In the raw processor, this algorithm is reversed for decompression to 16 bits.

 

The values that I used where: N0=1,6, N1=5,6, both based on 14 bits.

g=10@Iso160, W=0,6%,

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

 

Horizontally is the Exposure in EV, 13 being the point where the sensor starts to saturate.

Vertically is the S/N in EV.

Dynamic range is the point where the Signal equals the noise, or S/N = 0.

In this Picture, the DR is 11.5 EV (13-1,5) which corresponds well with existing publications of the M8.

 

The difference between the two lines is, that the purple line leaves everything in 14 bits without compressing to 8 bits, and the blue line shows the effect of the extra noise caused by compression that Leica uses for the M8.

It can be well seen, that in the middle gray areas ( around 8 on the x axis) , about 1 full stop in S/N ist lost by compression, becoming less in the bright and in the dark areas.

It has no effect on the Dynamic Range of 11,5 Stops.

 

Hans

 

P.S. This picture is at ISO 160. At higher ISO values the lines are coming down, and the effect of compressing is getting lesser.

Edited by t024484
Link to post
Share on other sites

Isn't this just another way to say that the compression will bring some quantization noise, and that this additional noise will be more noticeable where other noise levels are low?

 

So, at low ISO with full 14(16) bit A/D and no compression the image may appear a little clearer and smoother? Perhaps something like what people say about the DMR?

 

Regards

Per

Link to post
Share on other sites

Although undoubtedly there is some loss of information, one full stop would seem high to me. And is inconsistent with the section on lossy NEF compression in the article you cite; Leica's square law compression is very similar to NEF compression in the mid-tones (although admittedly it has fewer levels), so I'm surprised that that a compression scheme that similar to one the referenced paper says has "a good margin of safety" could lose that much in the mid-tones.

 

Sandy

Link to post
Share on other sites

Although undoubtedly there is some loss of information, one full stop would seem high to me. And is inconsistent with the section on lossy NEF compression in the article you cite; Leica's square law compression is very similar to NEF compression in the mid-tones (although admittedly it has fewer levels), so I'm surprised that that a compression scheme that similar to one the referenced paper says has "a good margin of safety" could lose that much in the mid-tones.

 

Sandy

Hi Sandy,

 

You are right, I checked everything all over again, and I found a small error in my calculation.

Nikon has 689 levels versus 256 for the Leica, more than 2,5 times as many steps, making noise quite roughly 2,5 times lower.

Shown below is the corrected Leica 8 bit compression table, versus a NEF compression with 689 steps.

The NEF compression is almost identical to the original uncompressed curve, confirming the statement in the article.

Leica's compression loses 0,5 stop in the middle gray area.

 

Hans

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

Leica Compression

 

NEF compression

Link to post
Share on other sites

Isn't this just another way to say that the compression will bring some quantization noise, and that this additional noise will be more noticeable where other noise levels are low?

 

So, at low ISO with full 14(16) bit A/D and no compression the image may appear a little clearer and smoother? Perhaps something like what people say about the DMR?

 

Regards

Per

 

or additionally, that the worst case for noise added by compression occurs in the brighter regions of the picture, where they are less noticeable? and that in quarter tones, the extra noise is minimal?

 

thanks for the detective work

Link to post
Share on other sites

Advertisement (gone after registration)

Any way to explain this interesting work to semi illiterate people like me? :eek:

 

The practical implications are also of interest to me. I've been thinking about starting a thread about the compression algorithm myself, as I've been wondering about the strange posterization of certain areas that occur in mid-shadow transitions: visible, say, in the light shadow of a person's ear, or in the folds of their clothing.

 

This has only manifested itself in pixel-peeping at the moment, because the few images I've printed (small) have looked fine. But I wonder what the posterization would look like if I needed to print much larger, and it nags me that maybe the compression process has thrown away the details that have become untidy clumps?

 

Is the compression intended to save time (in writing to the card) or space (on the card itself)? If write-time is significantly longer for an uncompressed file, how much of that time is saved by not needing to go through the compression process? If the only motivation is saving space on cards, why do we still have it when 8GB (and larger?) cards are useable on the M8 these days?

 

All in all, I do wish that uncompressed files were at least a choice one could make when using the camera.

Link to post
Share on other sites

Oh my, this is Greek to me.

Any way to explain this interesting work to semi illiterate people like me? :eek:

 

I may well be wrong, but if I understand the posts correctly, they are saying that any extra noise caused by Leica's compression algorithm is minimal, and don't worry about it under normal circumstances. :)

Link to post
Share on other sites

I may well be wrong, but if I understand the posts correctly, they are saying that any extra noise caused by Leica's compression algorithm is minimal, and don't worry about it under normal circumstances. :)

 

Well even if that's a positive spin, there IS detail lost. And I wonder if it's even necessary now - as I said above.

Link to post
Share on other sites

I would be interested to hear if anyone have printed 8bit versus higher bit on inkjet or 'real' offset print and noticed differences?

 

My workflow is 8-bit in photoshop but I sometimes wonder what I might be missing in final print by not maintaining the original higher bit fromt he scanner, camea, etc.

Link to post
Share on other sites

We know the theory, just compare DMR and M8 1:1 (with 90AA, same lens for M and R) - I've never seen any difference and nobody ever showed a comparison on the net which showed any difference either...

Link to post
Share on other sites

My workflow is 8-bit in photoshop but I sometimes wonder what I might be missing in final print by not maintaining the original higher bit fromt he scanner, camea, etc.

Printing 8 bits at least on an inkjet is harmless, but you are doing yourself a disservice with a 8 bit editing workflow. 16 bits gives you more editing headroom and a more accurate result - there should be no reason not to do it.

Link to post
Share on other sites

Not only that, When using 8-bits you lose values, because they "get moved into the next bit" If that already contains information it is lumped together and a value is lost. This leads to posterisation. If there are "empty bits" the data are just shifted, not lost. So always edit in 16 bits. In Lightroom it is a bit ;) different, as all corrections are only applied when the image is processed. Photoshop 8-bits editing is not the same thing as RAW 8 bits compression.

Edited by jaapv
Link to post
Share on other sites

I've never seen any difference and nobody ever showed a comparison on the net which showed any difference either...

 

Do you have a source for 16bit comparisons on 'the net'? By this, do you mean downloadable DNG files? Or do you mean some meaningless comparison of two 8bit compressed jpegs?

 

These discussions are so dispiriting I find: there are always people who take any discussion of any possible shortcoming in the M8 as virtually a personal insult, and throw out statements for which I can't see there can possibly be any justification.

 

Is it heresy to suggest that with larger and faster cards, that users could at least be given the option of choosing 16bit? (actually 14bit, but never mind).

 

There are incidentally quite a few people who think that the greater accuracy of the DMR actually does translate to a perceptibly better final file.

Link to post
Share on other sites

I may well be wrong, but if I understand the posts correctly, they are saying that any extra noise caused by Leica's compression algorithm is minimal, and don't worry about it under normal circumstances. :)

 

Nicole, thank you very much. :)

Link to post
Share on other sites

We know the theory, just compare DMR and M8 1:1 (with 90AA, same lens for M and R) - I've never seen any difference and nobody ever showed a comparison on the net which showed any difference either...

 

Well, I did compare. Simply worked on the same capture(8-bits , an old Canon 10D) in 8-bits and in 16 bits. The 8-bits workflow exhibited heavy posterisation around the sun, the 16- bits hardly any. That was enough for me, especially as there is no conceivable rationale not to work in 16 bits.

Link to post
Share on other sites

The Canon 10D used 8bit logarithmic or linear? The M8 doesn't show posterisation.

 

I've tried the DMR before the M8 arrived and never saw significant differences (only colors). Lets show some 1:1 comparisons, yes 16bit TIFF, just tiny crops (bright areas, where the logarithmic has less information) so they don't become too big for the net.

Link to post
Share on other sites

Georg, the DMR is a native 16-bits camera, so processing is automatically in 16 bits.

 

But what is the problem? Once you have your RAW developer set to 16-bits, you don't even notice the fact that your processing is in 16 bits. Just make a little action before saving to get down to 8. I have two: one for printing - Whatever mode the image is in, it will be converted to 8-bits aRGB, for the web it has an additional conversion to sRGB incorporated. So it does not make a difference to a considerable number of images. It will make a difference to a number, and a dramatic difference to a few. Yes, and? Why go to the trouble of deciding individually per image? Life is too short...

Edited by jaapv
Link to post
Share on other sites

Thanks.

 

So (being on my laptop now and haven't checked my editing compouter yet) this means that Lightroom is deafult 16bit (or could be - I'll check) and then when I export hires JPG's these are automatically 8bit.

 

As I understand it, anything above 16bit can't be seen on a screen but only in hires output for offset printing and a few similar devices (perhaps some inkjet printers).

 

I'll check my workflow in Lightroom, FlexColor, etc.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...