Puts's latest M8 color comments - seriously flawed

[MarcRochkind]

Advertisement (gone after registration)

A couple of weeks ago I posted a comment to the effect that evaluating color accuracy by looking at a processed raw file made no sense, because it was impossible to separate what the camera and lens were doing from what the raw processing was doing. A few readers agreed with me, and a few disagreed.

 

Now I see that Erwin Puts is also not making sense, at least to me.

 

In his column "Part 9: M8: UV IR filter revisited" he says that in his previous column on this subject he compared shots with and without the filter this way:

 

"The WB was automatic as was the exposure. The image files were Raw converted with Lightroom and here all parameters were set to zero. Color temperature was 5900 and Tint was +9."

 

So, all I get from his report is that one image is better for the case when "The WB was automatic as was the exposure. The image files were Raw converted with Lightroom and here all parameters were set to zero. Color temperature was 5900 and Tint was +9."

 

Does this matter?

 

Can he really be thinking that "all parameters were set to zero" means that somehow Lightroom is rendering some kind of pure, objective color suitable for evaluation? All it means to me is that Lightroom is set to its defaults, which are whatever whoever wrote the configuration file decided was OK on that particular day. Since no IR filters were available on that day, maybe it even means that the default was set up for the unfiltered case, but I think that might be giving him or her too much credit.

 

I truly don't understand the fixation on treating raw processors as though some sort of default is special in any way. Its only purpose is (1) to start the process from a set of reasonable assumptions, and (2) make the initial JPEG preview somewhat usable. It plays no role whatsoever in the ability of the photographer to achieve the final color he or she wants, except to the extent that the initial guess might bias where the photographer ends up.

 

Is it just me? Why can't experts like Puts see that evaluating a camera's ability to produce color by looking at a blindly-processed, default raw conversion is meaningless?

 

(Note: I'm not saying that the IR filter doesn't mess up the Leica's color in the situations where Puts says it does. I have no opinion. I'm only saying that he hasn't made his case, and his, and others', treatment of raw processors' defaults as though they were special contributes to this general misunderstanding.)

 

--Marc

[robertwright]

Well he is comparing apples to apples, in other words, the only difference is the presence or absence of the filter, and he is comparing the results of those situations. To a degree the raw converter is immaterial, he does not say the color is better or worse for what it is, he says that the filter does not "improve" color and in fact introduces noise and unwanted reflections.

 

What is interesting to me is the noise part, that seems significant if it were true.

 

Otherwise the test itself is not that relevant, the Ir issue is unpredictable, you could do what he says and "first try a shot without the filter" but that is crazy from my point of view. While there may be absolute differences, relatively, the color out of the camera is fine. And as you say, the rest is conversion.

 

But the noise issue might be interersting to investigate, I can't see how the filter would add to image noise.

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[lct]

Would be interesting to know if IR-cut filters produce false colors in practice.

I did some tests with Leica IR-cut filters on a R-D1 and Leica lenses (50/1.4asph below) and i felt that the green of grass and leaves is often too saturated with a filter on.

I seem to recall that a list member made that same observation about the M8 here but i don't remember whom sorry.

Pic one unchanged saturation, pic two green saturation -40.

 

 

[erl]

I'm inclined to think, without knowing, that you RAW processor of choice is going to have more influence than the IR filter. Of course, both together could be cumulative, or otherwise.

[scott kirkpatrick]

Given Erwin Puts' tastes, which emphasise reproducability over anticipating the results in any particular workflow, i would expect him to address mostly color accuracy -- shoot a standard color chart, develop the file using procedures which you can describe economically, analyze the results numerically by transforming the RGB values that result into a standard color space and comparing with the originals. That gives useful information when comparing two cameras, or the color shifts of two lenses, but may be completely irrelevant to your daily needs, which are influenced by what you shoot and how you process the results.

 

On this forum, Jono Slack likes the greens that he gets by forgetting about the IR filters entirely. He lives in the wet SW of England. Guy Mancuso likes the crisp yellow greens that he sees around him in Arizona, and gets them reliably with Leica or B&W 486 IT filters. Jono develops in Aperture, Guy I think in Phase One. Their mileage clearly differs, but they each have good reasons for their choices.

 

scott

[Chuck Hatcher]

Although I am no expert, and I have never used Lightroom, I would have to ask how is it possible for WB to be automatic if the color temp is set to 5900 when processing the RAW files? Doesn't changing the color temperature to 5900 negate the white balance setting chosen by the camera?

[robertwright]

Advertisement (gone after registration)

Although I am no expert, and I have never used Lightroom, I would have to ask how is it possible for WB to be automatic if the color temp is set to 5900 when processing the RAW files? Doesn't changing the color temperature to 5900 negate the white balance setting chosen by the camera?

 

have we all not noticed that the M8 only seems to have two wb settings in auto-either 5900+9 or 3900+12? Therefore Lr is only displaying those values in As Shot. Or am I wrong?

[Chuck Hatcher]

have we all not noticed that the M8 only seems to have two wb settings in auto-either 5900+9 or 3900+12? Therefore Lr is only displaying those values in As Shot. Or am I wrong?

 

I don't know how Lightroom handles WB settings. In C1, when I first open a raw file it is set to the camera-selected WB. I have seen more than two combinations (and never 5900/+9), but you are right there may not be very many! If the original post meant the WB as delivered by the camera in auto was 5900/+9 rather than WB was set to 5900/+9 during processing, I misunderstood.

[Guest guy_mancuso]

Robert that seems to be the case is LR brings in images at 5900 and a +9 as the tint which is always wrong. Normally about the best setting in LR is about 5400 and a different tint value but that is based on WB a card from a shoot in LR the numbers will drop down to 5400 in most cases with daylight and with that so will the color change. Also LR tends to have a magenta cast overall in there default and i have seen this consistently. Basically bottom line is it's default is not very accurate but of course we can change all that and i do

[Guest guy_mancuso]

Not sure i agree once again with Erwin here , IR does affect other colors in the visable spectrum and frankly i have no idea how you can get more noise from a IR filter. Another debate that will rage on for 10 pages.

[sean_reid]

I may regret stepping into an Erwin Puts thread but here goes. If the subject used was a color chart, it likely didn't reflect much IR at all. If that was the case, the tests don't tell us anything very useful. Its important to test the effectiveness of the filters on subjects that reflect IR (textiles are only one example). To test the effectiveness of the filters on color accuracy by using a test chart would be to entirely misunderstand the nature of the IR-sensitivity problem. When I tested the effectiveness of these filters, several months back, I tested by photographing a scene made up of many kinds and colors of fabric - that was a relevant subject.

 

When testing is disconnected from the ways in which cameras and lenses are actually used it can tend to yield results that are irrelevant. Photographers on this list who exclusively photograph color test charts (in their actual use of the M8) may find tests done of the same subject to be quite useful. Otherwise...

 

Needless to say, Leica's own testing (after the camera was released) prompted them to spend large amounts of money on sending out filters, modifying firmware, etc. There's a reason for that. I suspect that an over-reliance on testing with color charts, etc. is one reason that they didn't catch the IR problem earlier on. I didn't catch it, at first, with pictures of fruit. One needs to look at color response with subjects made from a wide variety of materials to get a sense of both the problem and the effectiveness of the solution.

 

Erwin's report presents various graphs and such. I don't see any actual pictures and so I don't know what exactly he photographed for these tests. But if the "Color Checker" phrase in his tests indicate that he photographed a GMB color chart then we know why the filters didn't seem to have much effect.

 

I can't stress this enough...without looking at the "real world" relevance of test results, there's little that they can provide to photographers whose purpose is to actually make pictures. The common sense questions always need to be answered.

 

If anyone reading this is not clear on why testing with color charts is not a useful way of measuring the effectiveness of IR filters, please let me know and I'll try to explain further.

 

Cheers,

 

Sean

[Guest guy_mancuso]

To follow on Seans comments than i will bail from this thread just because I am not going to debate Erwins tests because exactly as Sean has mentioned and I said it also is IR filters do effect other color' of the spectrum, just take a shot of a green plant or grass with and without a filter and the answer is obvious. Now my rant than i will get out of this thread. I just can't swallow charts for data as accurate in some cases. I am a photographer not a engineer and i shoot real things not paper charts hanging on walls. If i shoot a portrait and my skins look they way i want than that is the color and tone I like. No chart in the world will tell me any different and there is NO standard to really follow. There all guides to get to a basic working place just like LR brings in daylight images at 5900 kelvin does not mean it is correct but a guide or approximation of what the program thinks daylight should be. End of rant

[haika]

while i fully agree with sean about the usefulness of test chart shots, there is one interesting thing comparing the "noise r g b luminance grey" for filter/no filter on the bottom of erwin's page: if the noise is indeed stronger with the filter, it seems that it is at the same time more uniform, while the curve for "no filter" shows the red channel higher, and at a different curve. this could indicate that the additional noise may be more evenly distributed and in practical photography less noticeable in ir-filtered shots.

 

günter (who's still waiting for the 55 and 60 filters to arrive)

[MarcRochkind]

Sean (and others)--

 

My issue isn't what the filter is or isn't doing. My issue is that people seem to be offering opinions about color accuracy that are based on evaluations of processed raw images or, even worse, JPEGs that were processed in the camera.

 

Since a processed raw file has, by definition, a color interpretation, my questions would be these:

 

1. How can the evaluation be made accurately and objectively by removing from the experiment the color interpretation from the raw processing? Puts seems to think that setting Lightroom's sliders to zero does it, but I'm sure he's wrong.

 

2. Is there a way to evaluate color accuracy in some way that doesn't involve looking at a processed raw image?

 

Wine tasters cleanse their pallets. Shouldn't photographers who are evaluating a camera's ability to record color "cleanse their pallets" as well?

 

And if so, how?

 

--Marc

[Jamie Roberts]

Whew! What Sean said, what Guy said

 

This is perhaps the very silliest thing I've ever seen on colour, colour accuracy, and repeatability. If you take away all the variables (including, as Sean pointed out, IR!) then of *course* there will be no difference.

 

Puts says "...the firmware update and the IR filter in combination with the 6-bit coding do not improve on the general image quality that can be delivered by the basic combination of the lens without coding and the attachment of the IR filter. "

 

That is, for many situations, quite frankly, ridiculous in the extreme. And you *know* I don't mind shooting without filters. But to say there's no difference in image quality, well, let's just say at least there's a (huge) difference in speed of post-processing

 

And what everyone said about RAW converters, too--they're far more likely to make more difference than anything else (well, except the light itself!).

[hankg]

Sean (and others)--

 

My issue isn't what the filter is or isn't doing. My issue is that people seem to be offering opinions about color accuracy that are based on evaluations of processed raw images or, even worse, JPEGs that were processed in the camera.

 

Since a processed raw file has, by definition, a color interpretation, my questions would be these:

 

1. How can the evaluation be made accurately and objectively by removing from the experiment the color interpretation from the raw processing? Puts seems to think that setting Lightroom's sliders to zero does it, but I'm sure he's wrong.

 

2. Is there a way to evaluate color accuracy in some way that doesn't involve looking at a processed raw image?

 

Wine tasters cleanse their pallets. Shouldn't photographers who are evaluating a camera's ability to record color "cleanse their pallets" as well?

 

And if so, how?

 

--Marc

 

I don't think it's posssible. There is in camera processing of the 'raw' raw data and conversion from analog to digital. But remember the camera is really a computer so you need to evealuate the whole system as one device. lens + filter + sensor/incamera software-processor. I suppose the closest you could get would be to set up a set of strobes with very repeatable color temp and exposure. So you can guarantee to repeat the light with every shot then process linear with C1. But any setting in a raw converter is still interpreting the data in a way different from every other raw converter. However the raw processor is like your film developer as long as you process all the same way and don't choose a development process that would favor one sort of capture you can make some comparison. Then there is the question of subject matter and how a change in the light source or the subject might completely change the results.

[barjohn]

What Sean said makes good sense and if I get a chance I will conduct an experiment using a Gretagmacbeth and shooting it on a sunny day with and without filter and profiling it in ACR to see what, if any difference it produces. In theory, at least, there should be no difference since the chart should not reflect much IR but given it is sunlight it may be that the heavy IR in sunlight does get reflected back rather than absorbed. I did get a difference between the B&W and the Leica filter when I ran my test. However, when I look at real photos I'm not sure I like what the profiler produced. When looking at the GretagMacbeth shots they looked much better and closer to the real thing with the profiles.

[wparsonsgisnet]

So, we have a basic problem.

 

1. A color chart, that is a calibrated and precise thing, is no good for evaluation IR-cut filters. And it sure as hell makes a boring picture. I am imagining a gallery of prints of color charts in different sizes and lighting, mounted, framed, and for sale.

2. We all see the same color differently (uh, oh).

3. Leica may still be fooling with the color balance, and (not to be a PITA forever) we DON'T have what I would call a raw file. The thing is bent (or else I am...).

 

It isn't going to do any good to draw lines in the sand today. This camera is shifting sideways.

[hahn73]

Sean (and others)--

 

My issue isn't what the filter is or isn't doing. My issue is that people seem to be offering opinions about color accuracy that are based on evaluations of processed raw images or, even worse, JPEGs that were processed in the camera.

 

Since a processed raw file has, by definition, a color interpretation, my questions would be these:

 

1. How can the evaluation be made accurately and objectively by removing from the experiment the color interpretation from the raw processing? Puts seems to think that setting Lightroom's sliders to zero does it, but I'm sure he's wrong.

 

2. Is there a way to evaluate color accuracy in some way that doesn't involve looking at a processed raw image?

 

Wine tasters cleanse their pallets. Shouldn't photographers who are evaluating a camera's ability to record color "cleanse their pallets" as well?

 

And if so, how?

 

--Marc

Marc - I have had the same issue as you. Which is also why I don't understand all the WB threads that come up in the forum. So what if the camera picked the wrong WB? Just change it during RAW (or rather DNG) processing. Most photographers using an M8 should not being shooting JPEG, unless they want to take the greater risk of uncorrectable exposure or color issues. For what? To save time on processing? The only reason I can think of to use JPEG at all (in conjunction with DNG) is in travel when you might only have a portable photo storage device with an LCD, that only displays JPEGS.

 

With the issue of IR filters, I DO see how it can affect the color though in that it may affect the color palette of a particular shot. Thus, the need for a distinct color profile. But unless I'm mistaken, that is a completely separate issue from WB.

[Advertisement]

Hallo,

Look here: Leica M

[sean_reid]

Sean (and others)--

 

My issue isn't what the filter is or isn't doing. My issue is that people seem to be offering opinions about color accuracy that are based on evaluations of processed raw images or, even worse, JPEGs that were processed in the camera.

 

Since a processed raw file has, by definition, a color interpretation, my questions would be these:

 

1. How can the evaluation be made accurately and objectively by removing from the experiment the color interpretation from the raw processing? Puts seems to think that setting Lightroom's sliders to zero does it, but I'm sure he's wrong.

 

2. Is there a way to evaluate color accuracy in some way that doesn't involve looking at a processed raw image?

 

Wine tasters cleanse their pallets. Shouldn't photographers who are evaluating a camera's ability to record color "cleanse their pallets" as well?

 

And if so, how?

 

--Marc

 

Hi Marc,

 

Its an interesting question. I have a friend named Michael Tapes who's somewhat of an expert in matters of RAW processing, etc. and we've discussed this issue at length. In short, there's no way around the fact that each processing program can create different color interpretations, changing settings in conversion can change color, etc.

 

That being the case, the best one can do (when doing comparison tests) is to choose a specific workflow and try to use it consistently for each comparison of different filters, lenses, etc. That doesn't have to be the default settings in a program (save for setting WB by sampling a WhiBal card) but it might as well be. I use C1 in my tests because I've been impressed with the results from that program for several years. That's not to say that another program couldn't give different (better being somewhat subjective) results but it makes sense to try to keep variables as constant as possible.

 

With the M8, C1 has worked closely with Leica and I find their color support for the M8 to be quite good in most respects. But when one says, "the M8's color" one can fairly ask, as you do, "which M8 color, using what workflow?".

 

So... if one is going to compare differences in color, with and without an IR-filter, he or she will want to keep the RAW conversion variables (whatever they might be) constant (to the degree possible). In C1, it makes sense to use the "M8 Generic" setting for the unfiltered files and the "M8 Generic + UV/IR" for the filtered files. Does that change in profiles introduce a potential confounding variable...of course. But it would be misleading to process either kind of file with profiles it wasn't meant to go with.

 

So, we can talk about relative color differences, to some extent, but we certainly can't talk about color rendering from any camera in terms of absolutes. Is there a way to measure color accuracy in an absolute way? Nope, although many may suggest otherwise. For starters, not every object in the world is made of the same material as GMB chart.

 

That's why I tend to like describing color using pictures of fruits and vegetables. That doesn't give any kind of absolute result but it does give one some reference point to color as it might be related to things in the real world. Skin tones are another interesting subject for color tests (because as humans we're, I believe, biologically predisposed to notice changes in skin tone as they can tell us about emotion, illness, etc. - I imagine this sensitivity to skin color has been with humans for thousands of years) To check color with IR-sensitive cameras, I now use a range of fabrics, etc.

 

Testing and describing color rendering is far more complicated than some of the pseudo-engineering approaches might have us believe.

 

Cheers,

 

Sean