Jump to content

M9 - compressed / uncompressed DNG - who's using what?


chris_tribble

Recommended Posts

Advertisement (gone after registration)

This comes up from time to time, and since printing and viewing on-screen both take place in 8 bits (with a colour profile, not the linear data that is in a raw capture), one should normally not see the difference. However, after drastic editing (especially brightening of a too-dark image), one might see posterization, a kind of banding in the gradations of the highlights. I have tried to produce this effect with the M8, so far without success. The lack of a full 16-bit (or 14-bit) option have prevented me from spending too much time on this, but once I get my M9, hopefully in late fall, I will try again.

Link to post
Share on other sites

  • Replies 55
  • Created
  • Last Reply
Hmmm - I give below a quickly done set of shots at a space across from where I'm sitting.

 

All taken at 400 ISO 1/60th second f2.8 using a monopod as stabiliser. NO sharpening applied.

 

First set are DNG compressed / Second set DNG uncompressed.

 

I'm fairly sure that in prints I'd not notice a difference. Interesting to see if others are up for doing some parallel tests...

 

I've copied the original DNG's to:

http://dl.getdropbox.com/u/862415/Leica_M9/L1001098.DNG

http://dl.getdropbox.com/u/862415/Leica_M9/L1001101.DNG

 

NB - you have to register with DropBox to access the files... also - the DNGs are currently uploading (11.43 London time) and won't be available for a little while...

 

Chris,

Thanks for the links. Can you do a similar comparison (without posting photos, but just links to the original DNG files) at ISO 1600? Thanks

Link to post
Share on other sites

The lack of a full 16-bit (or 14-bit) option have prevented me from spending too much time on this, but once I get my M9, hopefully in late fall, I will try again.

 

Pascal - interested to see the results!

 

Lars - in that we're dealing with perceptions at the end of the day, I'm not sure that positivist experimental method will offer a solution to the conundrum... however tempting it might sound.

 

What I've heard so far as that uncompressed DNG may give you more wiggle room if things go awry + there may be less vulnerability to noise in poor light. My current feeling is to use the uncompressed option for extreme conditions (I'm keeping a 1600 uncompressed preset) but to use compressed DNG for most other purposes - until I hear a convincing case the other way.

Link to post
Share on other sites

I am Pascal today, eh? :)

 

"Wiggle room" is exactly right. If you expose every shot perfectly, there should be no need for uncompressed (nor for raw...). However, how often does it happen that you grabbed a really good shot but didn't quite get the exposure right, with no chance to reshoot? Not all that often for me, but often enough to make me think that it is worth shooting uncompressed. When I import into Lightroom, I then let Lightroom compress it (losslessly), which brings the size on my harddrive into reasonable bounds.

Link to post
Share on other sites

I have an M8, and am going to stick with it for a while.

 

We can probably draw a reasonable conclusion from our years of working with 16-bit vs. 8-bit film scans and DSLR photos. My educated guess is that the difference will be very slight, and probably not visible in prints, for photos that do not require large level and curve adjustments.

 

For photos that do require large adjustments, the uncompressed DNG will probably produce better results in terms of less posterization / smoother gradation, particularly at the extremes of light and dark. It's just a matter of having more discrete values to work with when you start applying mathematical crunches to the numbers.

 

FWIW, I've rarely noticed a difference in the quality of 8 vs. 16 bit color workflow where small to moderate curves and tweaks are applied, even if I could see a bit of tooth-combing in the final histogram. With black-and-white, 16-bit workflow is often very necessary, and I can notice a difference. And where I had to do major messing around with highlight or shadow levels in color, it also made a visible difference.

 

Leica chose the compression scheme for the M8 and the M9 with all this in mind. So my suspicion is that using uncompressed DNGs will make a difference in the same situations where a 16-bit data and workflow was visibly necessary with film and DSLRs, but the 8-bit square-root compression will take less of a quality "hit" than using an 8-bit workflow would. The difference will probably be mostly observable in the highlight gradations, as this is where the M8 compression throws away the most values. Many of us have noticed that the M8 has a lot of wiggle room in the shadows compared to many DSLRs, but less in the highlights.

 

Noise, dynamic range and ISO issues make the issue much more complex than just the 8 vs. 16-bit math would indicate.

 

--Peter

Link to post
Share on other sites

Advertisement (gone after registration)

I am Pascal today, eh? :)

When I import into Lightroom, I then let Lightroom compress it (losslessly), which brings the size on my harddrive into reasonable bounds.

 

Carsten / Pascal ... (:)) -- what's the compression procedure on import? I seem to have missed that (is if "import and convert to DNG"? - sort of a double DNG encoding?)

Link to post
Share on other sites

Hmm - I've just tried the "convert to DNG" option in Lightroom IMPORT - and it's true - the 34 MB Leica DNG comes down to 20,386 MB - and seems to render a 100% preview more quickly than the uncompressed version! This could be a way forward (and the compressed files come down to a light-weight 8MB. No idea why I'd never noticed this before - anyone other than Carsten using this. Any downsides noted? Looks a bit suspiciously like a free lunch!!! ;)

Link to post
Share on other sites

I'm eager to hear the answer on that one as well. As far as I recall, when choosing the Import as DNG it didn't import the JPGs though it stored them in the import folder ..?!

 

Any experience/advice on Lightroom speed. I don't care so much for harddrive space, but my big issue these days is preview speed in Lightroom. I've started to import with 100% previews so as to get previews faster. When you have 200 or more pictures you don't have time to wait 1-2 second for each to preview (part of what I look for is sharpness why it require that I can zoom in on them fast).

 

Any help would be appreciated.

Link to post
Share on other sites

Hmm - I've just tried the "convert to DNG" option in Lightroom IMPORT - and it's true - the 34 MB Leica DNG comes down to 20,386 MB

 

I had forgotten that I have convert to DNG turned on by default (I turned it on originally to convert all Canon RAW files to DNG). It seems that this size difference affects the M8 as well. A straight DNG from the camera is around 10MB but an M8 DNG imported as a DNG to Lightroom then goes down to about 7MB.

Link to post
Share on other sites

Hmm - I've just tried the "convert to DNG" option in Lightroom IMPORT - and it's true - the 34 MB Leica DNG comes down to 20,386 MB - and seems to render a 100% preview more quickly than the uncompressed version! This could be a way forward (and the compressed files come down to a light-weight 8MB. No idea why I'd never noticed this before - anyone other than Carsten using this. Any downsides noted? Looks a bit suspiciously like a free lunch!!! ;)

 

Chris, Adobe has stated that the more efficient compression of the DNG files on import is truly lossless. You can prove it yourself comparing two versions of an imported DNG if desired.This is a completely separate function to what is going on in the camera of course.

The complication is that the lossless compression is designed to be applied to new DNG files made from other Raw formats on import.

In this instance (M8/M9) you are already importing DNGs of course. So the lossless compression is dependant on specific settings (related to building previews for one) to be applied.

Link to post
Share on other sites

Chris, one reason the further-compressed DNGs aren't more widely used is that when the M8 first came out, Capture One could work with the camera's DNGs, but wouldn't handle the more compressed 'standard' version that Adobe's DNG > DNG gave.

 

Remember, the M8 was the first camera to use the lookup table in the DNG. It was an alternative allowed in the DNG specification of the time, but no one had done it before.

 

Adobe quickly came out with a new version of ACR able to read the M8's files. But if you then ran them through ACR's DNG conversion, apparently (I'm guessing) they were converted to the standard version of DNG. Whatever the conversion did, although Bridge and ACR could still read them, Capture One couldn't.

 

I assume C1 has modernized and can now read both styles from the M8, but I haven't checked. And I don't think I've seen anything about whether the Phase One product responds to the ACR-compressed version with the M9.

Link to post
Share on other sites

You might want to check if the compressed DNGs written by Lightroom are still compatible with Capture One. I noticed that when I opened some DMR files I had compressed with the ADOBE DNG converter, Capture one no longer had a color profile for the DMR, but used a DNG neutral profile instead. Applying the DMR profile made for odd colors. In other words, you may be loosing some of the important Leica maker info when Adobe compresses the files.

 

As for compressed or uncompressed, I find the DMR (uncompressed DNG) gives better colour than the M8. It also has smoother skin tones. Chris might want to redo his test using a model and then look at how smooth the skin tone is rendered.

 

Robert

Link to post
Share on other sites

You might want to check if the compressed DNGs written by Lightroom are still compatible with Capture One. I noticed that when I opened some DMR files I had compressed with the ADOBE DNG converter, Capture one no longer had a color profile for the DMR, but used a DNG neutral profile instead. Applying the DMR profile made for odd colors. In other words, you may be loosing some of the important Leica maker info when Adobe compresses the files.

 

As for compressed or uncompressed, I find the DMR (uncompressed DNG) gives better colour than the M8. It also has smoother skin tones. Chris might want to redo his test using a model and then look at how smooth the skin tone is rendered.

 

Robert

I can only speak on the M8 DNGs. Yes they do still open. This changed with updated versions of the different converters. The DNG specifications do not force the software companies to use every part of the specification, if you like. You probably know too that a 'profile' is a different thing in C1 vs. LR.

Link to post
Share on other sites

I can only speak on the M8 DNGs. Yes they do still open....

Thanks, Geoff. Good to know my assumption was correct.

 

... a 'profile' is a different thing in C1 vs. LR.

How's that? Aren't they functionally the same? How do they operate differently?

Link to post
Share on other sites

OK this could get messy!:)

This is to the best of my knowledge. I rely on the books by Bruce Fraser/Jeff Schewe and Martin Evening to learn from. I am by no means a qualified expert.

I am most familiar with Adobe Camera Raw (5.5) and am learning Lightroom (2.5), which shares the same Raw processing engine.

I just updated my C1 to the latest version (4.8.3) with the M9 profile.

I tested with M9 (FW 1.02) DNGs.

C1 can embed ICC profiles in the files. Once you embed a profile in a DNG it can not be removed without a hex-editor as I understand it.

The camera calibration options that you can choose in LR (or ACR) are not ICC profiles. DNG profiles deal with the colour a camera has captured instead of the output colours as regular ICC profiles do. (This from Real World Camera Raw)

 

Now my experiment.

C1 can open both ~18MB DNGs as from camera and the same files once losslessly compressed (to ~10MB) by LR.

C1 has no lossless compression options itself.

LR can open DNGs processed in C1 with an ICC profile embedded. Right now you can select either the embedded (ICC) profile or any custom calibration profiles that have been made with the DNG profile editor. I have the excellent initial custom M9 calibration that Sandy has made available to us (currently M9 18 Sep).

If you process a DNG in LR selecting the Adobe M9 18 Sep profile then open it in C1 you cannot choose any profile other than 'embedded'.

 

 

Incidentally I found that neither C1 4.8.3 nor LR2.5 displays the Camera serial number in EXIF currently.

Sorry about the acronym overload. Does anyone's brain hurt yet? ;)

Link to post
Share on other sites

Chris, there is no free lunch, of course, but this one is very cheap. The Adobe compression is lossless, and the smaller file will load faster from disk, which is a bottleneck in intel-architecture computers. The small price is that some of the maker-notes in EXIF are not brought along, since the file is completely rewritten by Lightroom. I don't like losing the notes, but I prefer clean, compressed files by far.

 

In fact, Lightroom will convert images in its library in-place, as long as they are not already in DNG format (which sadly the M8 images are). I converted my entire CR2 Canon 5D library to DNG and halved the disk requirements. There doesn't appear to be much detail in those Canon files, and they compress famously.

 

Thorsten, you can select the images you want previews generated for and then go to the Library > Previews > Render... Previews. I generate the standard preview (the size of the image when not zoomed in, roughly) upon import to keep the times down, and generate the full previews for batches, as needed. You can flush them again when you are done with a project, because they do add up.

Link to post
Share on other sites

Chris, there is no free lunch, of course, but this one is very cheap. The Adobe compression is lossless, and the smaller file will load faster from disk, which is a bottleneck in intel-architecture computers. The small price is that some of the maker-notes in EXIF are not brought along, since the file is completely rewritten by Lightroom. I don't like losing the notes, but I prefer clean, compressed files by far.

 

In fact, Lightroom will convert images in its library in-place, as long as they are not already in DNG format (which sadly the M8 images are). I converted my entire CR2 Canon 5D library to DNG and halved the disk requirements. There doesn't appear to be much detail in those Canon files, and they compress famously.

 

Thorsten, you can select the images you want previews generated for and then go to the Library > Previews > Render... Previews. I generate the standard preview (the size of the image when not zoomed in, roughly) upon import to keep the times down, and generate the full previews for batches, as needed. You can flush them again when you are done with a project, because they do add up.

 

And LR will losslessly compress your M8/M9 DNGs after import too. You need to use the update DNG preview and metadata option as the workaround.

Link to post
Share on other sites

And LR will losslessly compress your M8/M9 DNGs after import too. You need to use the update DNG preview and metadata option as the workaround.

 

I just tried that, but it didn't work. I selected a dozen M8 DNG images, at 10.3MB each, and chose Metadata > Update DNG Preview & Metadata, but nothing happened?

Link to post
Share on other sites

I just tried that, but it didn't work. I selected a dozen M8 DNG images, at 10.3MB each, and chose Metadata > Update DNG Preview & Metadata, but nothing happened?

aah, it involves voodoo:)

You need to make some change to the images, for example change WB then select the update DNG preview and metadata option.

Link to post
Share on other sites

Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...