Fallacy of evaluating color by looking at "raw" mage

[MarcRochkind]

Advertisement (gone after registration)

Frequently, in various places but especially here, I read about someone who expresses an opinion about the color fidelity of a camera by looking at the raw file as initially opened in some conversion program (ACR, Camera One, etc.) or viewer.

 

This makes no sense to me whatsoever. What's being evaluated is the combination of (1) the raw data, (2) the default conversion to whatever image is being displayed, (3) the converter/viewer app itself, and (4) the calibration of the display and/or the printing subsystem.

 

Sometimes I even read that the person offering the opinion did no processing of the image at all, as if this means that the experiment is somehow being controlled.

 

To me, all that matters is (1) whether the final image, after processing, is satisfactory and (2) how hard (effort, skill, and time) it was to get that result. If, compared to some other camera, it's consistently too hard to get satisfactory results, only then maybe can one say that one camera produces worse color than another. But, even then, one would have to try several of the raw-processing programs to be sure that the problem was really in the camera. (I frequently read that a camera-manufacturer's own raw app produces better results that an independent one, usually ACR.)

 

I do understand evaluations of JPEGs that come straight from the camera, because at least in that case the first three elements I listed above are inside the camera, and, I trust, everyone who ventures such an opinion has a properly calibrated viewing subsystem. (I personally don't care about those evaluations, since I don't shoot JPEG if I can shoot raw instead. But I understand that others do care about JPEGs.)

 

But, when the camera is asked only to capture the raw data, leaving the other three elements to be decided later, how is the color of the viewed image possibly attributable to the camera?

 

Does anyone else agree with me that most of the evaluations we're reading about are meaningless?

 

(Note: I'm talking here about color, not about other important image qualities, such as noise.)

 

--Marc

[rwfreund]

Well..

When I shoot something like a color checker, I like to open it in acr to see how much horsing around I need to do to get close to the values for my several favorite squares.

More for grins, since I am very capable with most cameras of getting something roughly approximate, I find that although I might use that "personal profile" as a starting point, I almost never publish using that result for any real image.

That exercise does tell me a thing ot two.

-bob

[eronald]

 

But, when the camera is asked only to capture the raw data, leaving the other three elements to be decided later, how is the color of the viewed image possibly attributable to the camera?

 

 

--Marc

 

Because the behavior of all cameras is not equal, as anyone who has owned several cameras knows. The sensor, electronics and adjustments vary a lot. One central element defining the camera are the rgb bayer filters, and these vary a lot from model to model.

 

 

 

Edmund

[MarcRochkind]

Edmund--

 

I think my question may not have been expressed clearly enough. I'm not questioning that cameras vary, but rather that one can't evaluate the color quality of a camera by looking at an unprocessed image.

 

My point is that the color one sees in a processed or unprocessed raw image is not attributable to the camera alone (or even mostly), since much of what affects color occurs outside of the camera.

 

--Marc

[eronald]

Edmund--

 

I think my question may not have been expressed clearly enough. I'm not questioning that cameras vary, but rather that one can't evaluate the color quality of a camera by looking at an unprocessed image.

 

My point is that the color one sees in a processed or unprocessed raw image is not attributable to the camera alone (or even mostly), since much of what affects color occurs outside of the camera.

 

--Marc

 

Marc,

 

I tried to tell you very politely that even if the same electronics and naked sensor array were used in two cameras, if the two had a different Bayer micro-filter array the two cameras would already have wildly differing abilities to capture color. I have been told that it is the micro-filters chosen for the 1Ds2 which create that cameras bad skin reproduction.

 

What happens after capture is important, but many cameras CANNOT even capture the information necessary for processing. At which point what comes after is immaterial.

 

A good example of this is the M8. Which is incapable of capturing decent color as delivered because the IR contaminates the image.

 

 

If I may put it more bluntly: Inverting a singular matrix doesn't work.

Edmund

[MarcRochkind]

Edmund--

 

I'm not saying that cameras don't differ in their ability to record color, or that some cameras aren't better than others in this and other ways.

 

Rather, I am saying that viewing an unprocessed raw image is not the way to evaluate the situation. Surely you're not suggesting that evaluating a raw image, subjected to an unknown default conversion, leads to a useful evaluation? Yet, that erroneous method seems to be the most common evaluative technique used, and that is what I was trying to discourage in my post.

 

What method would you use to compare the color fidelity of two different cameras: Say the M8 and a Nikon D200?

 

(Regarding the IR issue with the M8: As everyone knows, Leica recommends that a filter be attached for color work, and even provides two of them for free. Therefore, evaluations of color without the filter need to be separated from evaluations with the filter. That the filter is regrettably required for accurate color is a completely separate issue from the one that I raised in my post.)

 

--Marc

[barjohn]

Advertisement (gone after registration)

Here is a thought, suppose one took the same application, (say Lightroom) and used the same ZERO profile for both cameras. Wouldn't one get what the basic camera file produced before any alterations. Now, it is undoubtedly easier to process one versus the other to approach an acceptable image or if a shot of a GRETAGMACBETH color chart is shot to see which can be brought closer to accurate colors and the degree of effort required using the same focal length lenses and light exposure on the subject.

[adan]

...Suppose one took the same application, (say Lightroom) and used the same ZERO profile for both cameras. Wouldn't one get what the basic camera file produced before any alterations.

 

Nope..because even the "zero profile" is a set of parameters set up by the guys who wrote that particular application.

 

ACR has a "default" profile for the M8 - but it is based on Adobe imaging guys fooling around with the M8's files and deciding "WE think this is the best interpretation of the colors." If you create your own profile, or use, say Jamie's - you can substitute YOUR opinion (or Jamie's) for Adobe's - but someone's opinion (in the form of a color profile) will always be in effect.

 

Back with M8 firmware 1.06, there was also the option to use an "Embedded" profile instead of the "ACR V.3.6" profile. But that embedded profile was still created by someone, presumably a software guy at Phase One, or Jenoptik, or Leica itself (or perhaps a committee from all three).

 

Both Marc and Edmund are actually right, in their own ways. Cameras do have inherent capabilities based on the hardware - BUT there is no scientific way to evaluate those differences using consumerized ready-to-play RAW applications. Probably some kind of "RAW Inspector" program exists to read the data totally impartially and objectively reveal what the camera does entirely on its own - I don't know what it is, though.

 

I can see - for a fraction of a second - what a really "RAW" M8 dng file looks like. Whenever I make adjustments to an image in ACR and then save the changes, there is often a brief flash of the underlying RAW image as ACR changes the preview from the "old" settings to the "new" ones. (I'll have to see if I can grab a screen shot during that moment when the image is "in-between").

 

A very dark undersaturated ugly little thing, which would immediately have photographers screaming their heads off at Adobe or Phase One or Bibble if all their pictures looked that way to begin with.

 

Therefore the software guys provide at least a rough (and usually reasonably sophisticated) "profile" to get the brightness and color looking rational from the get-go.

 

Totally (almost) off that topic, but related to Edmund's reference to Canon - I have a sneaking suspicion that Canon uses lighter, less saturated RGB in their Bayer filters. I.E. their filters "leak" a little of the other colors rather than being the purest red/green/blue. This would account for their softer colors, and also for their amazing ISO performance - if the "filter factors" for their pixels are only 2x instead of 3x, they can drop the signal gain (and thus the noise) by 50% for the same exposure. (Think of shooting with a 23A red filter as opposed to a 29 red color separation filter).

[MarcRochkind]

Thanks, Adan, for explaining this point more clearly than I think I did...

 

--Marc

[eronald]

Marc,

 

You still don't get it. When I purchased my M8 there *were no filters*, there was no indicated need for filters. The camera made magenta pictures. At that time it was necessary to evaluate the ability of the camera AS DELIVERED to reproduce the visible color palette on real world objects. It was suggested, by people who believe in the primacy of software, that this was somehow the fault of C1 or the fault of whoever had made the profile. It turned out that although software workarounds could be attempted this was a HARDWARE ISSUE. Not everything is as you suggest the "fault" of software.

 

There is really such a thing as a Raw file, and yes, there are ways of looking at it, although they are not necessarily the obvious ones.

 

You do realize that I'm one of the guys making profiles for the M8 and thus spend my time looking at color charts and using various Raw converters and homebrew software ?

 

Edmund--

 

I'm not saying that cameras don't differ in their ability to record color, or that some cameras aren't better than others in this and other ways.

 

Rather, I am saying that viewing an unprocessed raw image is not the way to evaluate the situation.

 

(Regarding the IR issue with the M8: As everyone knows, Leica recommends that a filter be attached for color work, and even provides two of them for free. Therefore, evaluations of color without the filter need to be separated from evaluations with the filter. That the filter is regrettably required for accurate color is a completely separate issue from the one that I raised in my post.)

 

--Marc

[eronald]

related to Edmund's reference to Canon - I have a sneaking suspicion that Canon uses lighter, less saturated RGB in their Bayer filters. I.E. their filters "leak" a little of the other colors rather than being the purest red/green/blue. This would account for their softer colors, and also for their amazing ISO performance - if the "filter factors" for their pixels are only 2x instead of 3x, they can drop the signal gain (and thus the noise) by 50% for the same exposure. (Think of shooting with a 23A red filter as opposed to a 29 red color separation filter).

 

Yes, their filters seem less orthogonal. At least on some cameras.

 

Edmund

[rwfreund]

Oh heck,

Sure there are a lot of variables, but a digital camera is a system of elements that transform light through several steps into a bunch of numbers that can be rendered as an image through convention as to the meaning of the numbers.

I really like to look at raw images through means such as ACR so that I can evaluate some aspects of that transformation. This gives me a tool (one of several) that helps me understand why the image looks as it does in final form. The view provided by ACR is not enough, but the controls permit one to establish a degree of control on the transformation.

The M8 seems to have image characteristics quite different from a D200 and from a 5D.

Some of those I might chose to retain, some to emphasize, and some to eliminate.

I like to "zero out" my images so that my starting point for further manipulation is fairly close to neutral.

The extreme case in illustrating justification for this is to have some strongly colored filter stuck on your lens, or perhaps WB set to tungsten whist you were shooting in daylight. Without the like of the image evaluation capabilities of tools such as ACR you might be stuck into always trying to correct very blue images in PS downstream in your workflow.

I do wish that there were ways in ACR to adjust the luminance response curves for each color. I like control and wish I could do even more.

The M8's "weak reds" for example need this sort of thing IMO.

-bob

[dfarkas]

Take a look at evaluating color between film cameras. If you shoot color negative film, the results depend very much on how the film is printed or scanned. For transparency we get a better comparitive analysis tool, but not a perfect one. Every type of color transparency film has different rendering of colors in a scene. You could look at digital camera profiles as an analogy for different film emulsions, each giving a different feel to an image, with no "absoultely correct" one. I thnk we may get caught up in trying to over-evaluate systems to death. Take a look at the final print (just like we did for film camera images) and judge accordingly. The differences in image quality are pretty evident at 20x30, which is why all the sample prints I show are of this size.

 

David

[MarcRochkind]

Edmund--

 

I'm afraid this is one of those times when forum interaction doesn't work, since we are failing to communicate.

 

I posted a comment (with clarifying follow-ups) that criticized the method of evaluating color by looking at an unprocessed (i.e., default processed) color image (because the camera is not solely responsible for that image), and you disagreed with me by pointing out that cameras vary in their ability to record color and, in the case of an unfiltered M8, even do it poorly.

 

Perhaps you misunderstood my post to mean that I think that the camera doesn't matter, since the corrections can be made later? That's certainly not true, and that's not what I meant to say. In fact, in my first post on this thread, I clearly acknowledged the case that "one camera produces worse color than another."

 

Clearly we are having two different conversations. Anyway, whether you agree with me or not, I certainly agree with you.

 

--Marc

[Guest guy_mancuso]

If i understand the question correctly here . What default is and what we as raw shooters want is a neutral default that is pretty accurate to what the camera is actually spitting out when we open in a certain raw converter. The worst thing is having 600 images and having to play around to a point of being counter productive. Really what you want is a default that requires very little adjustment to the raws. Different profiles are a way to accomplish this task and reason we have folks creating profiles for different converters to get the default into a neutral spot. If one profile always has a red cast than that needs to fixed in our post processing . So what we area all trying to get to is what looks or works the best as a default than you can tweak into any direction you seem fit. now different programs will have different defaults so we need to narrow down the raw processer options to one or two that you will work with and again that is personal apeal to the way you work. I like C1 and Lightroom right now, so whatever i shoot will be processed in either one and i am still going back and forth on which one I like better. I have worked with C1 for 6 years but i do like Lightroom functions better. I'm going to post some raw converter stuff here in a minute with differnt profiles to see where everyone has there neutral spot. Than you can take it from there. Some profiles may have better color's than other's or better tonal range and such. There is no one program fits all to look at a Raw and say this is what truly is coming out of the camera.

[Guest sirvine]

I'll attempt to clarify:

 

1. Fallacy: The M8 color reproduction sucks because even with an IR filter on and coded lenses, the DNG files show poor skin colors.

 

2. Truism: The M8's color reproduction sucks compared to the Phase P45 because, after each is profiled to the user's taste, the M8 lacks the same color range.

 

3. Trusim: The M8 cannot accurately reproduce colors without an IR filter because you can't color correct for IR sensitivity.

 

Is that about right?

[Jamie Roberts]

I'll attempt to clarify:

 

1. Fallacy: The M8 color reproduction sucks because even with an IR filter on and coded lenses, the DNG files show poor skin colors.

 

2. Truism: The M8's color reproduction sucks compared to the Phase P45 because, after each is profiled to the user's taste, the M8 lacks the same color range.

 

3. Trusim: The M8 cannot accurately reproduce colors without an IR filter because you can't color correct for IR sensitivity.

 

Is that about right?

 

Yikes. I'm getting old and tired; I'm not even sure what Marc and Edmund were arguing about

 

It is true that the RAW file you "look at" in C1 is a combination of hardware (what the camera sees) and software (how that's interpreted).

 

But in C1 you're NOT looking at the RAW file. And I would agree with Marc--there is not much point in assessing anything in, say, C1, if you're not going to tweak anything. Unless, as Guy points out, you're looking for neutral. But even then, I bet you're tweaking the WB if nothing else.

 

Otherwise, why not evaluate a JPEG?

 

In fact, the reason I assess color, DR, etc... in RAW is very simple and practical:

 

1) I can make a better JPEG or TIFF than the camera can

2) I can better assess the response of the file to different curves, colour and tonal balances, etc...

3) I can play with input profiles into various colour spaces that optimize my workflow.

 

Got it? This has nothing to do with "accuracy"--though black being magenta sometimes is annoying in the way that red being green would be annoying.

 

Still, 99 times out of 100 a RAW file worth having makes its way to PS for further processing... Including colour correction, of course.

 

As far as the fallacies and truisms go...

 

I agree with 1--the M8 has produces fine skin tones, and fine skin textures too. A wicked combination.

 

I don't know what number 2 means. The Phase backs have more resolution, different lenses, etc... so that's an apples to oranges comparison

 

Number three is a complete and total fallacy.

 

Of course you can color correct for the effects of IR, and shoot without filters if you're willing to do the work. Sometimes that work will be negligible, by the way. Sometimes it will be a lot

 

But you can correct it! Old film scans look way worse coming out of the scanner (RAW) than the M8 ever does, believe me. You can still make a wonderful print if you know what you're doing.

[eronald]

Edmund--

I posted a comment (with clarifying follow-ups) that criticized the method of evaluating color by looking at an unprocessed (i.e., default processed) color image (because the camera is not solely responsible for that image), and you disagreed with me by pointing out that cameras vary in their ability to record color and, in the case of an unfiltered M8, even do it poorly.

 

Clearly we are having two different conversations. Anyway, whether you agree with me or not, I certainly agree with you.

 

--Marc

 

Yup. We were having two different conversations. Sorry.

I don't understand however what one can do with a Raw image, really, because as you point out if it's Raw it is not -yet, quite- an image ...

 

Edmund

[Guest sirvine]

Jamie,

 

On color correction of IR sensitivity-- I only meant that if you've got true magenta in the exposure and also black-as-magenta, any color profile is going to treat the two color values the same way, despite the fact that one should be purple and the other black. I guess you could fix it by selectively changing the color in photoshop, though.

[Advertisement]

Hallo,

Look here: Leica M

[Jamie Roberts]

Jamie,

 

On color correction of IR sensitivity-- I only meant that if you've got true magenta in the exposure and also black-as-magenta, any color profile is going to treat the two color values the same way, despite the fact that one should be purple and the other black. I guess you could fix it by selectively changing the color in photoshop, though.

 

Hey Sol, yes, you're right of course.

 

It's true that if you're profiling out "magenta" and you have real magenta in the shot, then you have a problem at the RAW conversion level (though there are a dozen ways to skin that cat in Photoshop, actually).

 

But even at the RAW level it can be less of a problem than most people think. There are a bunch of different magentas, and more importantly, a bunch of different "blacks" that aren't RGB/LAB 000.

 

Neutrals don't need to be absolutely computationally neutral to be perceived as neutral for most purposes (though when correcting colour one of the first things you look for is neutrals that are way off).

 

Depending on the colour space (and output ink limits, for example) it may not even be possible to print an "absolute neutral" and still use colour inks.

 

So if you're black is biased a little towards red and blue and a wee bit away from green, well, then you're going to let pure magenta come through pretty easily. That's not to say that this is optimal, but it's a pretty good starting point.

 

In fact, if you're counting on bias towards green (like the original Leica C1 profile) then you're going to be disappointed by this approach.

 

This is one reason I like using filters. But they have their own colour problems at the wide end, adding too much cyan from uncoded lenses. The coding fix from Leica--and the ingenuity of the users here--works to fix that!

 

But filters are not strictly *necessary.* They do, however, save a heck of a lot of time!