Does LR alter your referenced raw files

[stunsworth]

Advertisement (gone after registration)

Keith, the reset buttons on the bottom right hand side of the screen in the develop module - see below...

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

[cocker]

Thanks Steve - how long have I been using LR and not really registered that?!!

[glenerrolrd]

Ben

 

I think you are absolutely correct on needing a "original camera generated raw archive". This ,of course , protects your raw images but at the expense of a duplicate archive. This is not a small amount of storage with my archive. 50k raw images from mostly the M8 and the D3 is 750GB. M9 raw uncompressed images will be 4X as large...at 25K images per year..this is a big deal. But that is just the raw archive. This is useful for potentially reprocessing of the originals as raw convertors improve. Finding them is another discussion.

 

However as you pointed out ..that repository is completely independent of LR.

 

So back to using LR ....the in use raw files require a totally separate series of backups..maybe not as many levels but at least two copies ..the in use and a backup.

 

So what the LR developers blow off as ..its your choice if you want to maintain the raws ...can lead to a bigger deal in your backup process.

 

Ben ....why did Adobe recommend that you convert the raw files to DNG....this is exactly what screws up the use of LR with C1. I think with the right settings and care in processing you can build a pretty integrated workflow with C1 .

 

 

 

 

 

 

 

 

 

Roger, again, this is the Achilles heel, "One of the primary value propositions of LR is the ability to work non destructively with your raw files (or so I thought). "

 

I was instructed by a friend at Adobe to NEVER import the RAW file into LR. NEVER, NEVER, NEVER. As you pointed out, the Leica RAW file is designed by LEICA, etal. YOU MUST COPY & THEN COVERT the COPIED RAW file into a DNG. Then import ONLY the DNG. PERIOD. STOP. NO OTHER CHOICE.!!!!!!

 

SECOND. MAKE AN ADDITIONAL (READ 2nd) COPY OF ALL RAW FILES ON A DIFFERENT MEDIUM/DEVICE.

 

Never allow the raw files to exist on a platform where it is easy to import the original RAW files any where. NEVER. These files are GOLDEN. In the future, the images that we will be able to create with them are going to be outrageous. DON"T SPOIL THEM!

 

People don't take this seriously. Archive the original RAW files and NEVER touch them. You should always make a copy of any RAW file you want to convert to a DNG & then import that into LR or any other software. NEVER, EVER, trust any software company with your ORIGNAL SOURCEWORK!!!!!!

[ho_co]

This is MISLEADING. The problem is that you want to have the ORIGINAL RAW FILE, not the DNG. The DNG is not equivalent to the RAW file. DNG's are altered. The files start as RAW, then are CONVERTED into DNG's. PERIOD. Let's get this straight, A DNG IS NOT A RAW FILE!!

 

Ben--The M8, the M9, and a number of medium format cameras produce DNGs. So for those cameras, it's meaningless to say, "A DNG IS NOT A RAW FILE." The only RAW files they produce are DNGs.

 

One of the topics of this thread is the fact that there are several types of DNG. When the M8 came out, Adobe for the first time utilized a section of the DNG definition they hadn't previously implemented. And now the DNG definition has been updated to accommodate RAW output that includes optical corrections, without having to de-mosaic the file.

 

 

 

... I was instructed by a friend at Adobe to NEVER import the RAW file into LR. NEVER, NEVER, NEVER. As you pointed out, the Leica RAW file is designed by LEICA, etal. YOU MUST COPY & THEN COVERT the COPIED RAW file into a DNG. Then import ONLY the DNG. PERIOD. STOP. NO OTHER CHOICE.! ...

So I get a DNG from my M8/M9. I archive a copy. Then you say I have to copy the Leica DNG file and then convert it to a DNG. How do I convert my DNG into a DNG without importing it into Lightroom or using the identical converter in Bridge or in the standalone DNG converter?

 

All the instructions I've read for using Lightroom say it's designed specifically to import and manage RAW files, so, like Roger, I'm curious as to why your friend said (loudly, apparently) that I should "NEVER import the RAW file into LR. NEVER, NEVER, NEVER."

 

Seems to me that he must have had a point that is getting lost in transmittal.

 

 

 

 

 

 

Also, just a personal request: Please don't use "caps lock" as a substitute for underline, bold, italic. It's hard to read a paragraph where it looks as if the writer was SCREAMING ALL THE TIME.

 

If you don't currently have access to the forum's additional formatting options, you can change your profile to enable use of the wysiwyg editor.

[roguewave]

Ben

 

I think you are absolutely correct on needing a "original camera generated raw archive". This ,of course , protects your raw images but at the expense of a duplicate archive. This is not a small amount of storage with my archive. 50k raw images from mostly the M8 and the D3 is 750GB. M9 raw uncompressed images will be 4X as large...at 25K images per year..this is a big deal. But that is just the raw archive. This is useful for potentially reprocessing of the originals as raw convertors improve. Finding them is another discussion.

 

However as you pointed out ..that repository is completely independent of LR.

 

So back to using LR ....the in use raw files require a totally separate series of backups..maybe not as many levels but at least two copies ..the in use and a backup.

 

So what the LR developers blow off as ..its your choice if you want to maintain the raws ...can lead to a bigger deal in your backup process.

 

Ben ....why did Adobe recommend that you convert the raw files to DNG....this is exactly what screws up the use of LR with C1. I think with the right settings and care in processing you can build a pretty integrated workflow with C1 .

 

Roger, BE CAREFUL. You are doing it again! If you choose to use LR here's what I was instructed to do by an engineer with Adobe: 1) Keep a separate archive of ALL RAW files. 2) If you want to IMPORT a COPY of the RAW files TO COVERT IT INTO A DNG here's the method: Make a COPY of the RAW files on the disk where you will keep the working DNG"S once the RAW files are converted. ERASE the "RAW" files used to make the conversions to DNG's, just in case something went wrong in the conversion & so you won't accidently copy those "RAW" files over any existing RAW files in the archive.

 

Make sure to remember: a RAW file IS NOT A DNG. A DNG IS NOT A RAW FILE. That is the only TRUTH. Don't mistake one for the other.

 

One imports the RAW file into LR ONLY BY CONVERTING AS A DNG FIRST.

 

These are commandments, not suggestions. The engineers I correspond with are seeing all kinds of legacy problems with the explosive growth of alternative RAW CONVERSION SOFTWARE, both with Adobe' ACR & others that are in competition.

 

You won't get hurt if you archive every RAW file. I'm using a DROBO, like David has. With 8TB (4 x 2TB drives), I have double redundancy & sleep at night knowing everything is OK. I also keep an archive of all RAW files in a cloud. Cheap insurance. BTW, a 1TB drive is about $125 today. I also keep an archive up & running on my local computer in case I want to go back to an original RAW file to re-inspect the data & try a new version. Again, cheap insurance for the real "keepers". Remember, RAW is RAW, NOT a DNG!!. See you soon I hope.

[roguewave]

Ben--It's clear that there are several flavors of DNG.

 

The M8, the M9, and a number of medium format cameras produce DNGs. The DNGs from those cameras are as close as we can get to their RAW output. The only RAW files they produce are DNGs.

 

 

 

Also, please don't use "caps lock" as a substitute for underline, bold, italic. It's hard to read a paragraph where it looks as if the writer was SCREAMING ALL THE TIME.

 

If you don't currently have access to the forum's additional formatting options, you can change your profile to enable use of the wysiwyg editor.

 

Howard, thanks. I reset the editing. I think it's important for people to know that the moniker DNG is still misleading. The raw file I get from my DMR is different than the "dng" one has once the raw file in brought into LR & "converted" into a "dng". Therein lies a very problematic issue. Alll "dng's" are not the same.

 

BTW, when I use FlexColor ( I don't like C1) instead of LR, I make a copy of the original raw file for processing & store those files together as a package so they can't get mixed up.

 

Adobe only cares about Adobe. It's not going to protect the user & probably would like to limit the easy of use & interoperability of so called "DNG" files. Beware & be vigilant.

 

Folks clearly don't understand the long term implication of their workflow & the limits they are putting on the future use & exploration of their files.

 

I spent a few hours over the weekend with some programmers at a Columbia University Computer Science lab, where they remapped a raw file from a Nikon D7000. Up came an "on the fly" control panel that let me move segments of the image (shadows, forms) and move in a 360 degree rotation. Yes the system was a bit slow, but it created imagery that was not even in the file from existing architecture. Good times are ahead.

[ho_co]

Advertisement (gone after registration)

Ben--

Thanks! We've all got a lot to learn on this matter, and you've obviously got a lot to offer.

 

I'm actually not surprised at what you say, because I don't think the "one size fits all" concept has ever worked, and Adobe has already had to rework the DNG implementation more than once just to try to keep it in line with the new RAW technologies.

 

OTOH, Adobe has certainly done a good job of selling the DNG idea.

[glenerrolrd]

Ben I still don t quite understand your point on the DNGs. Lets break it down.

 

1. I understand the concept of moving the raw files into a separate archive. Now these files in my case will be Leica generated DNG s and Nikon .nef native raw files. They will be off line . This will be a very large data base for me with the uncompressed M9 files coming in at 40mb each ....thats 1TB per year(25K images) x 3 copies. These don t get used unless I want to reprocess a file and the files may be easy to find individually but a bitch to pull a collection . I have no DAM software to manage this archive.(because I can t define it to LR without LR using the DNG to store things.)

 

2. I need a working database of images (and appropriate backups) that I define to LR. When I import these to LR ..I am referencing them without moving . LR provides some control over updates..not perfect but workable. This will be the primary working DB of my images . You can make this work and it appears that you can pull an true raw file from the DB using the LR tools. (This is a big deal with a archive headed for 100K + images).

 

3.Questions...

 

a. What is the primary reason for converting your original raw to DNGs before "importing" them? Is it to make sure you never confuse them with the your "pure" raw archive? Just to make this clearer......you want to convert my .NEF files to DNGs before LR sees them? Why?

 

b. OK lets call your working DB ....the new DNG repository. In this DB LR will update the master files as appropriate(meta data,develop settings etc) . Now I will need 3 copies of this archive as well ? Because it would take forever to "recover" from the original raw files.

[stunsworth]

he raw file I get from my DMR is different than the "dng" one has once the raw file in brought into LR & "converted" into a "dng". Therein lies a very problematic issue. Alll "dng's" are not the same

 

But the 'convert to DNG' is an option that can be switched off. Are you saying that if you import a DNG from an M8 or M9 it will be altered in a detrimental way even if the user specified that the files are _not_ to be converted to DNG?

 

See the attached screenshot for the relevant option...

 

Welcome, dear visitor! As registered member you'd see an image here…

Simply register for free here – We are always happy to welcome new members!

[sandymc]

But the 'convert to DNG' is an option that can be switched off. Are you saying that if you import a DNG from an M8 or M9 it will be altered in a detrimental way even if the user specified that the files are _not_ to be converted to DNG?

 

To break the answer to that into two parts:

 

a) Changed : yes.

 

in a detrimental way : difficult to say for certain, wherein lies the risk. You can be pretty sure that C1 (or whatever else) will always be able to process an entirely native "in camera" M9 file. Will the next version process a M9 DNG with different metadata, different data tiling, etc? Probably, but you don't have a guarantee.

 

Where I think Roger is coming from, although I don't thinks he's put it this way, is that a DAM such as LR should preserve raw files exactly, unless explicitly given permission not to. Unfortunately, that's not LR's default behavior now.

 

Eric Chan's recommendation on this, which I think got a bit lost in heat of the moment, is to mark all DNGs as read only in the operating system. Then LR will treat them as it treats Nikon, Canon, etc raws, and store edits, etc in a sidecar file. (Eric is a senior Camera Raw engineer at Adobe, btw). Eric has also said on a thread over at the Adobe forum that he understands the problem, and is looking at it, so I suspect that in some future version of LR, read-only treatment of "in-camera" DNGs will become the default.

 

Sandy

[roguewave]

To break the answer to that into two parts:

 

a) Changed : yes.

 

in a detrimental way : difficult to say for certain, wherein lies the risk. You can be pretty sure that C1 (or whatever else) will always be able to process an entirely native "in camera" M9 file. Will the next version process a M9 DNG with different metadata, different data tiling, etc? Probably, but you don't have a guarantee.

 

Where I think Roger is coming from, although I don't thinks he's put it this way, is that a DAM such as LR should preserve raw files exactly, unless explicitly given permission not to. Unfortunately, that's not LR's default behavior now.

 

Eric Chan's recommendation on this, which I think got a bit lost in heat of the moment, is to mark all DNGs as read only in the operating system. Then LR will treat them as it treats Nikon, Canon, etc raws, and store edits, etc in a sidecar file. (Eric is a senior Camera Raw engineer at Adobe, btw). Eric has also said on a thread over at the Adobe forum that he understands the problem, and is looking at it, so I suspect that in some future version of LR, read-only treatment of "in-camera" DNGs will become the default.

 

Sandy

 

Sandy, nice job explicating this. There's an even bigger problem to keep in mind. The preservation & isolation of the original raw files is paramount because of future tools & assests that will allow much more sophisticated processing.

 

As Roger asked, & Eric noted, I think it is best to create a copy of the raw files and to use those to "import & convert to DNG". I save those dng's in a folder marked "Adobe DNG". That way I know these are not the raw files, but the converted files that are now & forever Adobe converted dng's for use in LR.

For Roger, I believe you can have a digital management scheme for working images & the converted raw files that we now call Adobe dngs. Bridge is one way, but by using catalogs in LR and making sure that all the key words have been parsed outside of LR so that all the key words can be searched in any catalog you wish to make for any search or project, there is internal consistency. Essentially, one needs to set up an Adobe workflow & keep that seperate from the use of any other camera. For Roger, you need a scheme that differentiates M8, M9, D3 & Nikon original raw files & the "converted, working dng's and the jpegs that issue from them.

 

I believe these problems will get more serious as the competing raw file software makers will have to become more propriety to keep their sheep in the flock.

 

The whole idea of using the "convert to dng" is paramount. Adobe is telling you, "we are now changing this file for use in our software". For users of C 1 and other programs, I suggest that you querry them about they are doing under the hood to keep the files in their stable as well. That's the model for the software business.

 

Most important, never assume anything. Most software companies do not want to disclose back doors & the architecture of their propriety code. Don't be a hapless victim.

 

Above all, keep those original files that were written on cards safe, secure & untouched, forever. You can always make exact copies to play with as you wish, but never import them directly.

[glenerrolrd]

To break the answer to that into two parts:

 

a) Changed : yes.

 

in a detrimental way : difficult to say for certain, wherein lies the risk. You can be pretty sure that C1 (or whatever else) will always be able to process an entirely native "in camera" M9 file. Will the next version process a M9 DNG with different metadata, different data tiling, etc? Probably, but you don't have a guarantee.

 

Where I think Roger is coming from, although I don't thinks he's put it this way, is that a DAM such as LR should preserve raw files exactly, unless explicitly given permission not to. Unfortunately, that's not LR's default behavior now.

 

Eric Chan's recommendation on this, which I think got a bit lost in heat of the moment, is to mark all DNGs as read only in the operating system. Then LR will treat them as it treats Nikon, Canon, etc raws, and store edits, etc in a sidecar file. (Eric is a senior Camera Raw engineer at Adobe, btw). Eric has also said on a thread over at the Adobe forum that he understands the problem, and is looking at it, so I suspect that in some future version of LR, read-only treatment of "in-camera" DNGs will become the default.

 

Sandy

 

Sandy

 

Thanks for the explanation . I think unfortunately the impact of the way LR processes native DNGs ..results in a need to keep an entirely separate archive of native raw files. This is more than a storage issue as redundant archives are very difficult to synch and navigate. You would be shot if you designed a business application this way and its completely unnecessary.

 

If you follow the recommendations of most LR tutorials ...you are reading in your Leica native DNG ...and creating a new folder and possibly a new backup . But keep in mind that you can add meta data, possibly process to your standard etc....those are great features for a .NEF file because all that stuff is contained in the .xmp . If you do this with the Leica raw file ...it will stuff the same information into a Adobe DNG. (that has will certainly be kept up to date). I would have to test this but my guess would be that both your master file and the back up would be in the new Adobe DNG format. When you reformat your card..the raw capture is gone. If you are OK with one of the many work arounds for using the raw data stored in the converted DNG and you are not concerned that this can change at anytime ..........then you don t have a problem.

 

Keep in mind that my workflow starts with an independent import into a structure folder repository under the Mac OS. This facilitates a variety of storage and backup alternatives. For example original cards, the mac pro drive, 1 tb of high speed mobile storage, an Epson P etc..and in the future a 128 -256gb flash memory stick. Now you need to think through which folder am I working with ..the native raw or the adobe DNG?

 

For most users this is a problem waiting to happen.

[glenerrolrd]

Ben

 

I see no reason to convert to DNG on import other than possibly easier identification of the imported images . ( assume I get it on how to build a raw repository and back it up). As many have pointed out , you can currently use the primary image data base through Lr and create selects off of it . These appear as unprocessed Leica raw files in C1 . If however you specify to add the metadata or create an compressed DNG you could be screwed because the conversion seems to be baked in and C1 see the file as a converted DNG and doesn t apply the M8/M9 profile. With some effort I am sure this can be managed to a successful integrated process....but not after the fact unless you saved the unprocessed raws.

 

I can t see any reason to convert to DNG and for sure I don t want to put the folders inside LR ?

 

Roger

[roguewave]

Roger, I tread lightly knowing how you made your fortune.

 

Since I am not an engineer, but a lowly academic/psychologist bookworm trying his best not to blow up my collection of a life's image making, here's what's up for me.

 

You, I & many people have tried various solutions for managing the ever growing sea of data we create. I'm certain of one thing: there's no solution that will work for everyone.

 

What I tried to layout was a way in which the user has put a Chinese Wall around a propriety solution (i.e. Adobe) so that whatever workflow you design does not mix up disparate standards. The reason for using "Copy as a DNG" as the method to import the raw file & have a copy converted to Adobe's standard, its to codify it as just that. Not a universal raw file, but to signify that it's now in the Adobe queue & system. It must remain there for your own protection. I find that I can get to almost any of the converted dng's, the psd files I generated from them & the various jpegs that issued forth from the psd with no problem. Same with the scans I make. Once I get the scan I like best, I save the original & then import the tiff as an Adobe dng so that the rest of the work flow is consistent and easily managed. From there I know what's Abobe only & keep it that way.

 

If, per chance, I decide that I want to reprocess raw file #10, I make a copy of the raw from one of the 2 archive virgins and bring them into the new workflow, in this case Hasselblad's Flexcolor. All the workflow that emanates from this raw file remains out of the Adobe content (catalogs....) It is named, saved & collated as Flex, or C1 if one wants to use that.

 

In my testing, I can do all the manipulations of the raw file I might like to make in Flexcolor, keep that whole package with all the files & adjustments intact & archive the folder. No problem. I can also make a psd file from the output of those manipulations & re-import them in my "Flexcolor" catalogs with LR as psd files or more importantly as "Coverted to DNG" files in catalogs named as "Flexcolor-Adobe DNG". Bang. I get to keep my same habitual stupid behavior, but with this loud warning telling me the source of the material. Indeed, once you have the "keeper" psd/tiff file converted and looking as desired, you can make prints, jpegs whatever & remain in flow. Most important, I still have all the original raw files separated from any processing.

 

LR is very cool and only going to get a lot cooler. Once Adobe merges the functionality of LR & PS, then we're talking real synergy.

[ho_co]

... one of the 2 archive virgins ...

Funny, isn't it, how this photography thing grows till we treat it almost like a religion?

 

We used to have Vestal Virgins. Now it's Archive Virgins.

 

Plus ça change...

[roguewave]

Howard, when I was young it was whole other species that was more of interest.

[glenerrolrd]

Ben

 

I am the guy that didn t suspect that Lr would touch a file thats referenced. The whole architecture was based on separating the referencing(indexing ,metadata,collections) and the develop settings from the raw file. This was so that you could at anytime go back and rework an image. This is essentially true for 99.9% of the situations LR faces.

 

You know for example if you output a TIFF or a PSD for post processing that you backed in the profile etc.

 

I can definitely understand why an Adobe engineer would design the process and why Leica would use the .DNG format. Its just unfortunate that ..it just doesn t seem to be important to Adobe do this in a way that insures the integrity of the native raws. But I missed it thinking "non destructive" might mean you didn t change anything.

 

Thanks Ben for the input and all the others that contributed. I think I get it now and when I have some time (moving south this week) I will test this . I think there is a way to create a workflow thats both safe and efficient.

 

Now that Leica includes LR with the products ....I would expect another 8-10K leica&LR users.

[JPSuisse]

Hi folks:

 

This thread might also interest you: Adobe Forums: Lightroom should handle camera...

 

1.) My opinion is that a lot of people may get a nasty surprise in the future to find out that there Lightroom DNGs are no longer useable in other raw converters.

 

2.) DNG does not equal "non-destructive image editing."

 

3.) I also am backing up my data redundantly, 2 backups of edited DNGs and 2 backups of unedited DNGs.

 

4.) I would be happier, if Leica would have created a new RAW format for the M8 and M9. Then, we could just use sidecars...

 

5.) Bibble, by the way, does just this. It creates a sidecar for the M8 DNGs. Now that's "non-destructive" editing!!!

 

Kind regards,

 

JP

[ho_co]

JP--

Thanks for the link and comments, and welcome to the forum.

 

It's beginning to look as if DNG redounds to Adobe's favor and no one else's.

 

I guess it's easier for a company like Leica to assume an already-defined and supposedly universal format like DNG than to create a new one.

 

But wasn't one of the selling points of DNG supposed to be its universality? Harumph!

 

 

 

BTW--does the Adobe link work? May be just short-term, but I'm getting "Site Area Temporarily Unavailable." Does it need a login or something else I don't have?

[Advertisement]

Hallo,

Look here: Interesting Leica gear

[Chris_R]

Does Capture One do the same or is it non-destructive?

 

Cheers

Chris R