RAW Converter standards (technical warning)

[chris_livsey]

Advertisement (gone after registration)

Another forum has directed me to this paper:http://www.cdiny.com/ArticlesWhitePapers/ISO%20Standards%20for%20Museum%20Imaging_cdi_v1.0.pdf

Adopting ISO Standards for Museum Imaging

 

This may seem dry and esoteric but does answer some of the questions posed here regarding the abilities of the various RAW file software available and will be of interest to the technically minded. eg

Quote

To illustrate how the current trend in image editing applications can be dangerous for

long-term viability of digital assets for cultural institutions try this experiment:

 

Step 1: Create a new image in Adobe® PhotoshopTM CS3 in the Adobe RGB (1998)

working space. Using the color picker enter the following LAB values: 50, 0, 0. Select

the entire image area and fill it with this value. Note that the RGB values are 118, 118,

118. Save the file as a TIFF format 8 or 16 bit.

 

Step 2: Import the file into LightroomTM. Using the eyedropper, read the RGB values. You will find that the RGB values read 46.6%, 46.6%, 46.6%. Adjust the exposure slider until the RGB values read 50%, 50%, 50% and export the image to Adobe RGB (1998)

working space.

 

Step 3: Open the exported image in PhotoshopTM CS3. You will find that the LAB values

went from 50, 0, 0 to 54, 0, 0 and the RGB values went from 118, 118, 118 to 127, 127,

127. This means that the same RAW file of a properly exposed target processed in

Adobe® LightroomTM and Adobe® Camera RAW will process differently. The root cause is that PhotoshopTM and LightroomTM are based upon different gamma transformation

functions. Today every raw processor will yield different results due to lack of standards.

End quote

 

There is also confirmation that what we see on the histogram may well not translate into making good decisions to produce the "best" RAW file

[gmaurizio]

Chris, how are you doing?

I did your experiment as follows: Created a Lab mode 600x600 image in PS3, filled with L50-0-0. Saved as Tif. The same file was converted to profile and saved in Tif with Profoto, AdobeRGB and sRGB. Every single file still showed L50-0-0 with PS3, while of course RGB triplets were different in each case (although adobeRGB and sRGB showed the same values, as expected).

Imported in LR and Aperture, and every picture looked the same in color calibrated monitor. Exported to full size JPG in every case and got a compressed file with THE EXACT SAME COLOR INFO (L50-0-0 and corresponding RGB values).

I think the mistake or error you are making is correcting exposure guided by the 49.8% showed under the Loupe or WB tool.

You do not know exactly what is the internal color space of Aperture or LR, so it is a mistake to judge by RGB values shown in a slider.

Other than that, I see no problem whatsoever with both pieces of SW, which by the way are designed to handle RAW files.

Please let me know if I am missing something, and what was the point you are trying to make since I do not completely get it.

Thanks

[luigi bertolotti]

Thanks for the link... a paper which goes well in depth... I'm going to sleep, now, but surely worths a deep reading.

[chris_livsey]

Hi Gus, Fine thanks but getting poorer with all these lenses, just picked up a mint 21mmAsph and am very impressed (using 28mm finder which improves the experience).

 

Now, I was quoting from the article and as I don't run lightroom here I can't replicate the steps. What I think they are trying to show, in a reproducible manner, is what we already know from empirical experience, that if you take the same RAW file different software will produce different results and "matching" from one to another is impossible (probably). Also that two cameras (even of the same model) will not produce RAW files,of the same scene, that match in colour reproduction.

As an aside they think that ProPhotoRGB is the internal working space.

 

The argument is for standardization from the camera to the print, we know we can closely approximate to that now with a careful calibrated workflow to produce our own consistent results, but that is individual. They argue that I should be able to take a colour chart on my M8 and produce a print identical to the same chart taken on your D3 without us both going mad in the process.

It's interesting in that they are looking for digital archival storage of works of art and the ability to reproduce them faithfully in print, try this as an example of where we are now: google - images,"michaelangelo creation" or another well know work and see from the thumbnails how the reproduction varies. (This may not show as much difference in a colour aware browser such as Safari - another story but really part of this one)

Whereas we, excluding those working in say areas of medical or forensic imaging, as Photographic "artists" are looking to produce prints which express our intentions/vision and accuracy is not a primary consideration. An accurate portrait is rarely flattering

Although the passion for accurate M8 blacks might negate that argument.

 

The article is interesting for the historical overview of where we are and how we got here.

[gmaurizio]

Chris, I've yet to read your article. Seems interesting since I've been doing some art reproductions for a painter friend of mine and found a lot of problems with metamerism with her cobalt blues...

I think that LR uses an internal color space that, while similar to Profoto (by Kodak), it is not exactly the same. The point is, when you are done with your fixes corrections, you should save an uncompressed TIFF file with a color profile (stable) associated if you are looking for the long term.

I guess the problem comes from the fact that cameras are not easy to be profiled, and many have a wider gamut than AdobeRGB. Also, demosaicing (is that a valid verb?) algorithms change with different releases, so it is not surprising that they differ from SW to SW.

Even different samples of the same brand/model produce different raw files in the same conditions!!!

Since the human color response does not have a filter equivalent match, these problems are never ending. They existed in Film and will continue to exist in digital, even with 3-CCD cameras or sensors, like Foveon and Better light. We are complex image processing machines, and so far nothing even comes close to the human brain and its contextual interpretation of reality.

Thanks for the pointer, and this is a nice topic!

[gmaurizio]

I've browsed quickly through the article. It is thorough, interesting, but not entirely accurate. The problem not addressed is the following: The reproduction of artwork consists of digital/chemical capture of 'reflected and emitted light' by an artpiece. Therefore, the spectral response and correspondent human perception depends on many factors. You must add to that an output stage, where the digital capture will be again illuminated and perceived by a different observer. It is impossible to accurately reproduce.

On top of that, the L* ICC color models do not consider the perception of the brain of different adjacent colors. For example, take a look at any Kincaid 'painter of light' picture. Move the pixels a bit and the overall effect changes a lot.

We need better color models, but we might never achieve what the writer of the article proposes.

[sandymc]

Advertisement (gone after registration)

Not to take away from the point made in the original article that color management as currently practiced is a mess, the example quoted above is just plain wrong - it based on a misunderstanding of what the Photoshop and Lightroom RGB values are actually reading.

 

Photoshop reads on a 0-255 scale, so 118 is 46.27%, not 50%

 

As regards Lightroom, Lightroom's RGB readout is in the Melissa RGB space, which is a proprietary Adobe space - basically, it uses ProPhoto's color values, and an sRGB gamma curve. If you convert lab 50,0,0 to Melissa RGB you get 0.4649,0.4649,0.4649, exactly what Lightroom is reading. You can try that in Apple's ColorSync utility if you want.

 

If you then adust the exposure setting, your are changing the color away from lab 50,0,0. So when you then export your new color, its no longer lab 50,0,0. That's entirely correct behaviour. RGB 50,50,50 is actually lab 53.39,0,0 - again you can check that with Apple's ColorSync.

 

For those interested, I wrote extensively on color spaces and RGB readout for Lightroom, Aperture and C1 here: ChromaSoft: Lightroom, Aperture and Capture One Mini-Review Part 1

 

Sandy

[chris_livsey]

You both make very valid points which I am sure the author of the paper would appreciate being appraised of.

email: scott@cdiny.com

Thanks also from me for expanding my knowledge base.

[gmaurizio]

For those interested, I wrote extensively on color spaces and RGB readout for Lightroom, Aperture and C1 here: ChromaSoft: Lightroom, Aperture and Capture One Mini-Review Part 1

 

Sandy, good work. What are your conclusions, sparing us of the technical stuff?

Some people think that Color Management beyond sRGB everywhere (the default) is a waste of time for most people. Others say that without color management we are lost in the shadows...

I find that the monitor is the worst component, since it barely shows sRGB --except of course those fine EISOs that are very expensive. Also, technical art reproduction has challenges that no one has solved up to now. I still use large format and film for those specific purposes, since digital becomes much more expensive and time consuming.

Will you share your thoughts?

[sandymc]

Sandy, good work. What are your conclusions, sparing us of the technical stuff?

Some people think that Color Management beyond sRGB everywhere (the default) is a waste of time for most people. Others say that without color management we are lost in the shadows...

I find that the monitor is the worst component, since it barely shows sRGB --except of course those fine EISOs that are very expensive. Also, technical art reproduction has challenges that no one has solved up to now. I still use large format and film for those specific purposes, since digital becomes much more expensive and time consuming.

Will you share your thoughts?

 

Well, its certainly true to say that that there are very few end-to-end solutions that are ever going to give you better results than sRGB, just because its probable that somewhere in the chain there is a sRGB device or system. But I'd still say that that's a problem to be solved, not a problem to be endured. So my recommendation is to fully color manage, if for no reason other than it pushes you and you selection of equipment and techniques towards better quality.

 

For what its worth, in term of practical color management, my view is that Aperture stands head and shoulders above either Lightroom or Capture One. Not because LR or C1 are less technically accurate, but because Aperture allows you to do things like measure values in lab coordinates as well as RGB. That means that in practice, it's easier to calibrate colors in Aperture than either of the other two.

 

Regards,

 

Sandy

[Photoskeptic]

Gus, I've had the same problem with art photos. I photograph a local artist's works and then print them on Velvet Fine Art for her to sell as matted prints. I've had hell trying to match her colors to the extent that I must have the original next to my monitor when I do adjustments.

This is an interesting thread on standards. I'm curious why more manufacturers have not adopted the DNG standard and why so many insist on using their own proprietary system to our (the photographers)detriment.

[gmaurizio]

Gus, I've had the same problem with art photos. I photograph a local artist's works and then print them on Velvet Fine Art for her to sell as matted prints. I've had hell trying to match her colors to the extent that I must have the original next to my monitor when I do adjustments.

This is an interesting thread on standards. I'm curious why more manufacturers have not adopted the DNG standard and why so many insist on using their own proprietary system to our (the photographers)detriment.

 

Dear Photoskeptic, how are you doing? I am sorry to hear that you are experiencing problems. So far, my worst problems have been metilen and cobalt blues. After long research, we found that the human eye has a low sensitivity window in some part of the red spectrum, seldom found in real world. These particular blue pigments, do show strong emission in that particular window, therefore rendering the color purple instead of blue.

This is perfectly ok according to the sensors, but not to the human eye. It is clearly a metameric/perceptive failure.

How did we correct the problem? We isolated those areas in the image, and we did careful color shifts using photoshop while printing on canvas, showing it to the artist and its gallery manager until we got it right. Now we can automate that process to a great success. Other similar problems happened with ocres and oranges.

 

As per raw standards, the problem is the manufacturers encode a lot of proprietary info in their files, allowing for their SW to recover much more detail. That is the case with Nikon, Canon and others. While a standard can be beneficial for the users, it might slow down the introduction of new models and features.

Also, get ready for 3-CCD or 3-CMOS sensors real soon now. That will be the next big thing, already on the works. Kiss bye bye to AA filters, aliasing and other problems of the Bayer approach.

[ericperlberg]

Two quick points about the paper linked to above on colour management. On page 5 of the paper is a diagramme showing a colour managed workflow from camera to output. The camera used in the diagramme is a Leica. ok, ok...

 

The other point (and I don't claim to comprehend its meaning, if any) is that the paper talks about the significance of a recently adopted ISO standard for a colour space called eciRGBv2 whose power is that its based on LAB* which itself is closely correlated with how humans actually see as opposed to more abstract spaces including sRGB, AdobeRGB, ProPhotoRGB etc all of which rely on gamma transformations (as in 1.8 or 2.2 or other) which the author posits is the cause of all the instability and resulting problems in archival work. That's the gist of the paper I think.

 

Anyway, in doing a search just now for the material used for the top plate on an M8 I came across the fact that jpgs from the m8 can be put into the eciRGB colour gamut by selecting it in the colour management section of the menu.

 

I imagine that if one wanted to do archival work of the type discussed in the paper one can do that with an M8.

 

Maybe that's why its pictured in the diagramme? Can other good cameras do this or is this an unrealised strength of the M8?

 

And does anyone know what material the top plate of an M8 is made from (brass I hope?).

[gmaurizio]

Anyway, in doing a search just now for the material used for the top plate on an M8 I came across the fact that jpgs from the m8 can be put into the eciRGB colour gamut by selecting it in the colour management section of the menu.

 

I imagine that if one wanted to do archival work of the type discussed in the paper one can do that with an M8.

 

Maybe that's why its pictured in the diagramme? Can other good cameras do this or is this an unrealised strength of the M8?

 

And does anyone know what material the top plate of an M8 is made from (brass I hope?).

 

Eric, I guess the M8 just does sRGB, under ECI standard. Not eciRGB as wide-gamut colorspace. Also, the lenses and MP count of the M8 render it useless for art reproduction. You need extremely flat field lenses, like the Zeiss Planar and very accurate focusing. a 1DSMKIII would be on the entry level, but a Betterlight scanback or a 8x10 technical view camera are in order.

The top of the M8 is brass.

[ericperlberg]

but a Betterlight scanback or a 8x10 technical view camera are in order.

The top of the M8 is brass.

 

Thanks Gus,

Obviously archival work is not my field. I know Betterlight scanback is what they use at the Courtauld here in London. Anyway, thanks for the top plate info.

 

Eric

[gmaurizio]

You are welcome Eric. Somehow the Gamma discussions remind me of those theories that say El Greco (spanish painter) had a vision problem that made him render his models with long faces. Would that have been the case, his paintings would have been perfect, since when he looked at the canvass he would have seen the same distortion.

L*, 1.8, 2.2 and others are just fine, as long as you understand which device expects what gamma. This is the reason most people have bad results using Epson's Advanced B&W from LR. Epson expects a grayscale 2.2 and LR sends a 1.8 if you do not convert.

[AlanG]

I found this interesting:

 

Microsoft hopes scRGB will improve photo colors | Underexposed - CNET News.com