Who can explain this regarding DNGs

[Sausalito]

Advertisement (gone after registration)

Just when I thought I had a pretty decent understanding of RAW files

 

Because Expression media (although Apple OS X 10.6.5 can) does not look at X1 DNG files and show a preview, I got a tip from a forum member that it might if I processed the X1 DNG with adobe dng converter... yippee, it works fine.

 

My issue is this. The X1 DNG is about 17.5 MB but the processed DNG is only about 11.5 MB. That said when opened with Photoshop RAW converter both files turn into 69 MB files (which is what I expect since it is the same sensor as the Nikon D300s).

 

Can anyone explain this? and what it's implications are?

 

thanks,

 

tom

[ChiILX1]

Just when I thought I had a pretty decent understanding of RAW files

 

Because Expression media (although Apple OS X 10.6.5 can) does not look at X1 DNG files and show a preview, I got a tip from a forum member that it might if I processed the X1 DNG with adobe dng converter... yippee, it works fine.

 

My issue is this. The X1 DNG is about 17.5 MB but the processed DNG is only about 11.5 MB. That said when opened with Photoshop RAW converter both files turn into 69 MB files (which is what I expect since it is the same sensor as the Nikon D300s).

 

Can anyone explain this? and what it's implications are?

 

thanks,

 

tom

 

Whoa

 

Big time cookie for the person who explicates this one.

[hoppyman]

Tom, you seem to have run into a limitation with Expression Media and your OS. You may like to look at the Adobe DNG User Forum.

I suspect that your OS and software do not (yet?) properly implement the current DNG standard (Version 1.3). (AKA the fault is not with Adobe or Leica! )

 

On DNG file sizes, that is normal.

When you process any DNG from any Leica camera in Adobe Camera Raw or Lightroom or the DNG converter, the Adobe Software can apply lossless compression to the file, 'Converting a DNG to a DNG' does that automatically. There is no information loss whatsoever and there are significant file size savings. As far as I know that is the default behaviour for every Adobe app now and may not even be optional to turn off any longer.

[ChiILX1]

Tom, you seem to have run into a limitation with Expression Media and your OS. You may like to look at the Adobe DNG User Forum.

I suspect that your OS and software do not (yet?) properly implement the current DNG standard (Version 1.3). (AKA the fault is not with Adobe or Leica! )

 

On DNG file sizes, that is normal.

When you process any DNG from any Leica camera in Adobe Camera Raw or Lightroom or the DNG converter, the Adobe Software can apply lossless compression to the file, 'Converting a DNG to a DNG' does that automatically. There is no information loss whatsoever and there are significant file size savings. As far as I know that is the default behaviour for every Adobe app now and may not even be optional to turn off any longer.

 

I can say it's not his OS, I'm on the same version of Snow.

[Sausalito]

Tom, you seem to have run into a limitation with Expression Media and your OS. You may like to look at the Adobe DNG User Forum.

I suspect that your OS and software do not (yet?) properly implement the current DNG standard (Version 1.3). (AKA the fault is not with Adobe or Leica! )

 

On DNG file sizes, that is normal.

When you process any DNG from any Leica camera in Adobe Camera Raw or Lightroom or the DNG converter, the Adobe Software can apply lossless compression to the file, 'Converting a DNG to a DNG' does that automatically. There is no information loss whatsoever and there are significant file size savings. As far as I know that is the default behaviour for every Adobe app now and may not even be optional to turn off any longer.

 

Thanks, Hoppyman.

 

I guess the DNG compression makes sense (I don't think NEF files work that way and I wonder if a second conversion would compress even more . I sure can't see any loss of detail but I have always avoided compression and worried.

 

As to any fault being either adobe or leica, I suspected Expression Media, though the guys at Phase one are not really up to speed yet (after buying the app from microsoft). Whenever Nikon has a new camera, Expression media always needs info from nikon about minor header changes in the NEF before they display correct thumbnails in EM and I really think that is probably what is going on here, too.

 

If anyone else is using EM and has a solution, I would love to hear from you.

 

cheers,

 

tom

[hoppyman]

By way of amplification for you, I just checked and, as expected, my current version of Capture One (4.8.3) and later has no difficulty providing thumbnails from and working with my DNG files compressed on import to my computer by Lightroom. That would have been so for some earlier versions as well, I guess at least for 12 months or so. In any event, there is no issue now. Since they have acquired Expression Media, why not lodge a Case with them asking about resolutions now? I found them very responsive and helpful when I first got C1 with my M8.

[Sausalito]

Advertisement (gone after registration)

By way of amplification for you, I just checked and, as expected, my current version of Capture One (4.8.3) and later has no difficulty providing thumbnails from and working with my DNG files compressed on import to my computer by Lightroom. That would have been so for some earlier versions as well, I guess at least for 12 months or so. In any event, there is no issue now. Since they have acquired Expression Media, why not lodge a Case with them asking about resolutions now? I found them very responsive and helpful when I first got C1 with my M8.

 

Thanks for that... I did lodge a case and was told that if I could view a preview in Finder, then it should work. I have not been able to find anyone else with and X1 and Media Expression to confirm that it isn't just me (but unlikely just me since the problem exists on three different macs).

 

I like the attitude of the Phase One people but they really haven't made any changes in the product yet as I am sure they will. With new Nikons (and microsoft) you had to wait several months for support in the product.

 

This is really a persistent annoyance... the workflow of deleteing all jpg's and then converting all DNG files is pesty... but I will persist :-)

 

cheers,

 

tom

[hoppyman]

Well it actually has nothing at all to do with the X1 or any other DNG native camera specifically.

 

Presumably you have some reason why you don't just use the Lightroom that came with your camera though and save yourself the aggravation?

Best of luck getting it all tamed your satisfaction anyway.

[ChiILX1]

If you know a student, Aperture 3 is $75

[Shootist]

Simple fact is that all Leica DNG files are based on the Adobe DNG standard but they are slightly different. Not sure exactly how but they are. That is why your photo editing program can't display them but when converting them to REAL Adobe DNG's it can. If Leica followed the Adobe DNG standard to the letter any RAW converter that can read DNG's would display a preview. When converting Leica DNG's to REAL Adobe DNG's the file get changed, not only compressed. Even with compression turned off the file will get smaller. This also happens if you are converting Nikon NEF files to DNG.

[sandymc]

Simple fact is that all Leica DNG files are based on the Adobe DNG standard but they are slightly different. Not sure exactly how but they are. That is why your photo editing program can't display them but when converting them to REAL Adobe DNG's it can. If Leica followed the Adobe DNG standard to the letter any RAW converter that can read DNG's would display a preview. When converting Leica DNG's to REAL Adobe DNG's the file get changed, not only compressed. Even with compression turned off the file will get smaller. This also happens if you are converting Nikon NEF files to DNG.

 

Not true.

 

Leica DNG's (other than perhaps the S2, depending on how you read the spec), are 100% compliant - you can run them through Adobe's dng_validate tool to check.

 

The problem is that that most viewers aren't 100% compliant. Actually no viewer other than Lightroom and Photoshop are 100% compliant.

 

Sandy

[ho_co]

Ed, you may be thinking of the fact that with the M8, Leica was the first company to use the LUT aspect already in the DNG definition. Even Adobe's products at the time had to be updated to apply this second half of the definition.

 

Second historical aspect, not directly related: When Panasonic and Leica began storing lens correction data in the LX3/D-Lux 4 RAW files, Adobe's products could convert them to DNG while keeping the corrections, but only in demosaicked form. Adobe then redefined the DNG and re-wrote its software to maintain the extra data without demosaicking.

 

IOW, as I understand it, Leica has always been compliant, but sometimes a bit ahead of Adobe in implementation.

[Shootist]

Ed, you may be thinking of the fact that with the M8, Leica was the first company to use the LUT aspect already in the DNG definition. Even Adobe's products at the time had to be updated to apply this second half of the definition.

 

Second historical aspect, not directly related: When Panasonic and Leica began storing lens correction data in the LX3/D-Lux 4 RAW files, Adobe's products could convert them to DNG while keeping the corrections, but only in demosaicked form. Adobe then redefined the DNG and re-wrote its software to maintain the extra data without demosaicking.

 

IOW, as I understand it, Leica has always been compliant, but sometimes a bit ahead of Adobe in implementation.

 

Yes Howard I think you are correct. But why then do even Adobe products have a hard time reading Leica DNG's if the software is not updated for that specific camera but if you take that same file and run it through the Adobe DNG coverter there software then can read it, with all the maker notes (EXIF data) intact.

 

I see what Sandy is saying and I know he knows more about this then I do.

[mjh]

Raw converters always “have a hard time” dealing with raw files from new camera models they don’t yet know about. Converting a raw file from an unknown camera with Adobe’s DNG Converter sometimes helps making the software play along (whether the original file was in DNG format or some other format). The general approach taken by developers of raw conversion software (or in fact any software dealing with raw files) is to rather err on the strict side, i.e. not to open raw files when they aren’t explicitly supported, even when the file format as such is. Expression Media used to have issues with NEF files from the (then new) D3 and D300, nothing new here.

[adan]

"But why then do even Adobe products have a hard time reading Leica DNG's if the software is not updated for that specific camera...."

 

Actually, I was able to open and/or view M9 DNGs from day one (Sept. 11, 2009) with really old versions of Adobe Camera Raw and Bridge CS3 (ACR 3.5 - the same one I had been using for M8 images since 2007). No issues at all (except the usual need for a better color profile/calibration).

 

Used that combo for 3 months until an upgrade to an Intel Mac required CS4 anyway.

 

Everything the old Adobe software needed to know about the image (pixel array and dimensions and so on) was built right into the DNG files

[hoppyman]

Which was really rather the point of DNG in the first place. To avoid yet another propietary Raw format that required its own converter and in Adobe's case yet another update, somewhere north of 100 formats having to be supported, sometimes by reverse engineering.

[ho_co]

... why then do even Adobe products have a hard time reading Leica DNG's if the software is not updated for that specific camera ...

 

I'm way out of my depth here, but Michael gives the general answer:

 

... The general approach taken by developers of raw conversion software (or in fact any software dealing with raw files) is to rather err on the strict side, i.e. not to open raw files when they aren’t explicitly supported...

 

I understand the idea, but I'm curious as well. Maybe the software looks for a particular string (like "NIKOND300") before it rates the file as safe?

 

IIRC, Sandy recently updated CornerFix to work with Pentax files converted by ACR.

 

How would a developer do that? As I see it, he might add a check-string for that model; or he might delete the model-checking routine completely; or maybe some other way?

[ho_co]

Which was really rather the point of DNG in the first place. To avoid yet another propietary Raw format that required its own converter and in Adobe's case yet another update, somewhere north of 100 formats having to be supported, sometimes by reverse engineering.

 

Don't you know it!

 

And since so few manufacturers jumped on the bandwagon, Adobe's still in the same place it was, plus it has to keep the DNG definition up to date.

 

Damned if you do, damned if you don't, in this case.

[mjh]

Maybe the software looks for a particular string (like "NIKOND300") before it rates the file as safe?

Probably something like that. While a raw file contains all the data and meta data necessary, software developers, as a rule, prefer to apply their own camera-specific profiles rather than the conversion matrices embedded in the raw files. Without such a profile no attempt at a conversion is made. On the one hand, this allows raw converters to achieve a better fit for the idiosyncracies of individual camera models; on the other hand it requires an update for each new camera introduced (or more realistically a batch update adding profiles for a whole bunch of recent models).

 

The advent of DNG didn’t change this. What it did change was the need to reverse engineer small changes camera vendors love to introduce in their raw file formats. While DNG isn’t cast in stone and will see further development, at least such developments are documented. But since most camera vendors still prefer to use their proprietary formats, it’s not like the task of third-party software developers was any easier now.

[Advertisement]

Hallo,

Look here: Interesting Leica gear

[Sausalito]

Well it actually has nothing at all to do with the X1 or any other DNG native camera specifically.

 

Presumably you have some reason why you don't just use the Lightroom that came with your camera though and save yourself the aggravation?

Best of luck getting it all tamed your satisfaction anyway.

 

I use photoshop, but for archiving LR just won't cut it... I have about 122,000 images on a 4TB drive and searching is a breeze with EM (when I used windows and mac, it was a lifesaver because it was cross platform). I don't know of any other archive app that will handle what I need and include all the legacy file formats.

 

If I could use LR or even Aperture, I would... the other problem with so many files is the database is huge in LR or Aperture and I couldn't get it on my macbook and still have room to shoot

 

tom