DNG Compression, Any Difference in Quality?

[t024484]

Advertisement (gone after registration)

I think I found an example where compressed DNG makes a difference for the worse. .

The only valid way to see the effect of compression is to construct a second image from an uncompressed DNG by compressing / decompressing all pixel values and compare the two images.

 

Hans

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[01af]

The error in any of the three color channels after compressing and decompressing apparently remains comfortably below 1 % of the original value. Even the worst case (deviations in the color channels with opposite signs) would not give color casts as strong as shown in your image.

This is what theory suggests. However this is not an image taken in bright, healthy daylight at low ISO. Instead it was taken at light that is both low in intensity and heavily distorted in colour and temperature, at the camera's maximum ISO, and in post-processing severely shifted away from regular temperature and tint values and pushed up in the shadows in order to arrive at some visually half-decent colour and tone rendition. This means digging down to the very bottom of the bit depth, and here even small compression losses, which otherwise are entirely insignificant, may (repeat: MAY) make a perceptible difference in the final result.

 

 

I think we see the results of unevenly applied additive mixing of colored lights which in turn is the result of the spatial distribution of colored spot lights.

That is most definitely not the reason for the sharp-edged blotches that appear when cranking up the tint value.

[01af]

The error in any of the three color channels after compressing and decompressing apparently remains comfortably below 1 % of the original value.

In the meantime, I did some calculations myself. If I understand Leica's compression scheme correctly then the error in each channel will be below 0.78 % of the original value in the brightest highlights but will gradually (and non-linearly) grow to a maximum of 33 % in the deepest shadows. The 14-bit value 3, after compression and decompression, will be mapped to 4. While this sounds bad, it mostly isn't, as 14-bit values this low usually will contain nothing but noise anyway ... with the possible exception of underexposed and badly colour-shifted high-ISO shots that require severe post-processing.

[charlesphoto99]

In which case - convert to b&w!

[dwbell]

That is why the "noise" generated with this way of compressing, just adds roughly 0.1 stop noise over the whole range from Black to White and only at SO 160. As mentioned before this is detectable with a computer but not by the human eye.

As from ISO 640, the generated compression noise completey vanishes within other noise sources that are getting larger and larger. So it is not as bad as it seems, that is why Leica offers this way of compression.

Hans

 

Assuming the above is correct, a very low light noisy shot at ISO2500 isn't going to prove or disprove anything is it? Especially not as dramatically as to be visible in a web crop image?

 

I've seen the horrible pink skin tone on my images and I remember it being discussed in the forum at least once before, I forget the outcome. Might try a search though.

[bocaburger]

I've spoken with 3 Akademie pros thus far and all of them said they themselves shoot compressed. That's good enough for me. I use 2GB non-HC cards and have zero issues with them, and a 5 yr old 2.8gHzPentium V computer with 3GB RAM and an 80GB HD and running XP...with likewise no problems. No intention to replace any of it with newer and bigger. I use CS2 also, and no plans to upgrade, nor to pay to upgrade from the free LR3 that came with my M9. Whatever ephemeral advantages theoretically do or don't exist by shooting uncompressed, I'm not interested in it, not for the sake of my photography nor for the sake of the "mine's bigger" advantage on internet forums

[01af]

Advertisement (gone after registration)

I think I found an example where compressed DNG makes a difference for the worse. See the image of dancers at a prom night below [...] The shift towards magenta does not appear gradually in the skin but as sharp-edged blotches while some parts of the skin hardly assume any shift towards magenta at all.

 

I suppose this is an artifact of the reduced bit depth ...

In the meantime, I found another available-light image, taken with M9 at ISO 2500/35° in uncompressed DNG format, which shows the same unpleasant magenta blotches in skin tones ... which means the reduced bit depth of the compressed DNG format was not the culprit. Sorry. So the quest for a proof that there might be a perceptible difference between Leica's compressed and uncompressed DNG formats continues ...

[vanhulsenbeek]

and may never be found

[Lindolfi]

I can't see the difference in images that are made under the same conditions with the compressed and uncompressed setting on the camera. There is however a measurable difference: I took 4 pictures

 

1. Uncompressed

2. Uncompressed

3. Compressed

4. Compressed

 

and loaded them in LightRoom with the same settings. Loaded them in PhotoShop placed them in layers and created difference images

A] the difference between 1. and 2.

B] the difference between 3. and 4.

C] the difference between 2. and 3.

 

After that I moved the right slider of the Levels panel to to the 15 bin location, which brought out the shadows by expanding the 16 lowest bins over the range of 256 bins

 

All was done in 16 bit loading and processing.

 

The results are here:

 

Image A:

Image B:

Image C:

 

And to show the normal image:

 

As you can see all difference images show information (the images are not identical), but the uncompressed images show the least difference ( seen in Image A) and the difference between the compressed and uncompressed images (seen in Image C) is the largest.

 

Part of the differences are due to noise, part by a not perfect register of the images at exposure and part due to the compression algorithm (in Image C).

[t024484]

Bert,

 

Interesting exercise, but it does only show that there are (mathematical) differences between the images.

In no way does it prove that compressed pictures are inferior or that compression cause some visible negative effect.

 

I can only repeat that the only valid way to see the effect of compression is to construct a second DNG from an uncompressed DNG by compressing / decompressing all pixel values and compare the two images.

 

I have made this exercise with pictures taken at the most critical value of ISO 160, because they are the cleanest in terms of noise.

Hoping for more, I had to come to the conclusion that although mathematical differences were quite easy to show, no visible difference were found, not even with the help of the mathematical information of where to look.

 

A leaf of a yellow tulip, gradually changing it color, did possibly show some banding effect after compression, but even this was very hard to see.

High quality studio pictures of faces did not reveal any visible effect of compression.

 

Hans

[algrove]

I've spoken with 3 Akademie pros thus far and all of them said they themselves shoot compressed. That's good enough for me. I use 2GB non-HC cards and have zero issues with them, and a 5 yr old 2.8gHzPentium V computer with 3GB RAM and an 80GB HD and running XP...with likewise no problems. No intention to replace any of it with newer and bigger. I use CS2 also, and no plans to upgrade, nor to pay to upgrade from the free LR3 that came with my M9. Whatever ephemeral advantages theoretically do or don't exist by shooting uncompressed, I'm not interested in it, not for the sake of my photography nor for the sake of the "mine's bigger" advantage on internet forums

 

Did you happen to ask these pros why they shoot compressed versus uncompressed?

[Lindolfi]

Bert,.

In no way does it prove that compressed pictures are inferior or that compression cause some visible negative effect.

 

Hans, as I wrote: I can not see the difference between compressed and uncompressed images, but technically I have shown that the compressed images have more inherent noise than uncompressed images. It visualises what you already wrote:

 

That is why the "noise" generated with this way of compressing, just adds roughly 0.1 stop noise over the whole range from Black to White and only at SO 160.

 

The noise increase I measured is about 7 bins and that is about 0.2 stops.

 

What is detectable for the eye depends a lot on the properties of the local part of the image. Our eyes are very complex sensors indeed. But as I said, I have not seen any difference so far. There is a whole list of other factors that are more important for image quality than the type of compression Leica uses.

[hlockwood]

The only valid way to see the effect of compression is to construct a second image from an uncompressed DNG by compressing / decompressing all pixel values and compare the two images.

 

Hans

 

It seems to me that there is an important element missing in all these arguments, pro and con, about the effects of compression. And that is, how is the image going to be displayed? After all, isn't that the principal purpose of creating images?

 

If the information/data is for archival purposes only, then we go down one path. If the purpose is to make the best possible print, that's another path. Web display? Another path.

 

Let's talk prints. 4x6 in., or 24x30 in.? Color or grayscale? Archival: paper and "inks", be they silver or carbon or dye?

 

I find these technical discussions quite enlightening, and I don't suggest suppressing them. But for me, as a serious amateur, the question comes down to this: how will compression affect my workflow in achieving the best possible, say, 12x18 in., archival print?

 

So, Hans (and others), tell me how you intend to "compare [the quality of] the two images" and why that method, instead of mine, is more valid.

 

Harry

[Anasebi]

I don't care if it's off topic, it has to be said:

 

Adobe is a overrated joke of a company who has left RAW and Photoshop in a pathetic state after years of neglect. CS5, which I have the misfortune of using has somewhat buggy GPU support but even with an 24 gigabytes of ram on a 4GHZ Six Core i7 with Hyperthreading; dealing with DNG import and JPEG export is annoying at best. If Adobe had been doing the job they should be processing large RAW should be snap finger fast on OSX (and windows for that matter)

 

Their damn camera RAW and Jpeg engine, along with most filter and tool functions are single threaded, or pseudo dual hyperthreaded, so irrigardless of so called 'progress' its much more tedious ordeal to convert DNG than it should be by a long shot. I hate Adobe, even their flagship Flash software seems to always be taking on far more bugs and performance decreases than they add with every worthless update.

 

Yet I still always use uncompressed DNG, it just seems so wrong to output small files on an M9. I can see very subtle differences, and thats reason enough.

[t024484]

So, Hans (and others), tell me how you intend to "compare [the quality of] the two images" and why that method, instead of mine, is more valid.

 

Harry

 

Harry,

 

When comparing the same image in compressed and in uncompressed form, the first step will be: are any differences visible between the two, either on the display by pixelpeeping, or in printed form, preferably with 300 DPI giving something like 12X18 inch.

When the answer is a no, the process stops here.

 

When slight differences are visible, a more difficult answer has to be found: which of the two is the best or has the better quality.

But to my opinion, the closer to the original the better, so the uncompressed version wins in that case.

 

Although I could not find visible differences with my experiments, it does not prove that they are never there.

But when you have enough memory space, and don't bother with the file transfer times, simply go for the uncompressed files for the lower ISO values.

For ISO 640 and above, compressing has no negative side effects and can be used without having to worry that image quality will be negatively affected.

 

Hans