Jump to content
Sign in to follow this  
t024484

dpreview Test

Recommended Posts

Advertisement (gone after registration)

John and SJP--

Many thanks for the pointers.

 

I began correctly with the square root of two as multiplier, but then simply multiplied the original f-stop by 4/3 and 5/3 for the third-stops, and by 1.5 for the half-stops. I knew my results were sloppy but couldn't figure out why. Thanks for the clarifications. (You don't learn if you don't expose your misunderstandings.)

 

And since (clearly) math and I don't connect well, I'm sure I'll be back with more questions...

 

 

John--yes, it's Howard. I hadn't noticed that the forum no longer shows the registered name.

Share this post


Link to post
Share on other sites

John, SJP, anyone with basic math skills:

 

Your suggestion of using (2 ^ 1/4) to increment aperture values by half-stop beginning at f/1 seems to work fine.

 

And your suggestion of using a factor of (2 ^ 1/6) to move from each value to the next in 1/3 stop increments works far better than my earlier attempt.

 

However, apparently due to the irrationality of both Excel and the sixth root of two, problems become apparent beginning with f/5.6, which appears as "f/5.7."

 

The error propagates to later values like "f/11.3," "f/22.6," etc.

 

Despite using the same factor for all the values after f/1, the rounding error isn't visible in the even values f/2, f/4, f/8, etc.

 

If I modify one of the problematic values (e.g., entering f/5.6 directly), a new and worse set of errors crops up, 'corrupting' even the even values.

 

Thanks for any suggestion.

 

 

Hmm. Maybe I should try simply implementing two sequences of doubling for the initial stops? That is, one sequence would be 1, 2, 4, 8, etc; and the other would be based on an initial and exact value (not a calculated one) of 1.4, and run 2.8, 5.6, 11.2, etc. That might take longer to create ugly problems. But since the apertures represent a simple geometric sequence, it shouldn't take that kind of intervention to represent them, should it?

Share this post


Link to post
Share on other sites
Hmm. Maybe I should try simply implementing two sequences of doubling for the initial stops? That is, one sequence would be 1, 2, 4, 8, etc; and the other would be based on an initial and exact value (not a calculated one) of 1.4, and run 2.8, 5.6, 11.2, etc. That might take longer to create ugly problems. But since the apertures represent a simple geometric sequence, it shouldn't take that kind of intervention to represent them, should it?

 

When you are done working on it, I hope you'll realize that some of the values are simply rounded off for convenience. 11 not 11.2, 22 not 22.4. 45 not 44.8

 

This should help you:

 

F-number - Wikipedia, the free encyclopedia

 

"Modern electronically-controlled interchangeable lenses, such as those from Canon and Sigma for SLR cameras, have f-stops specified internally in 1/8-stop increments, so the cameras' 1/3-stop settings are approximated by the nearest 1/8-stop setting in the lens."

 

So you should also be wondering how DPReview sets other cameras to f9.

 

When I studied Materials and Process at RIT we did statistical analysis of the accuracy of all of variables in the photographic process - light meters, lens apertures, shutter speeds, film speeds, processing variations, flash bulb delays. Even user reading and setting errors. 30 people reading three thermometers will give different results. Or 30 people reading the same scene with 30 different brand new Luna Pro meters. We concluded it was pretty surprising that good results came out as often as they did.

 

Did you know that with some lenses when you set them to f16 it will be different if you started from the largest aperture and stopped down to f16 than if you started from f32 and opened up to 16? There is slop in the system.

Share this post


Link to post
Share on other sites

Thanks again, Alan. I'll check the Wikipedia article. I always try to first to build these things from the brain and then turn to outside resources.

 

Yes, for extreme accuracy (e.g. Hollywood), I believe the rule is always to stop down to the required f/stop.

 

BTW--I tried my guess and it has fixed my spreadsheet-elegance problem pretty well: I use two initial values, 1.0 and 1.4, and then double each in its own sequence for full stops, while deriving the half- and third-stop values as John and SJP suggested.

 

As John said above (http://www.l-camera-forum.com/leica-forum/leica-m8-forum/55270-dpreview-test-3.html#post575723), in the all-mechanical days, Nikon had about a 6mm throw of the diaphragm lever from wide open to stopped down, where Leica R lenses had nearly double that. I understand that Nikon improved diaphragm damping and coupling several times mechanically, and maybe with electronics they now have the tolerances down to 1/8 stop, which is about where Leica was 40 years ago. Whatever Nikon has, it works well enough.

 

Yes, I'm still bothered by dpreview claiming that IR affects only some man-made substances and refusing to check the matter by shooting a comparison test. And I'm still bothered by the claim that they shoot the 50 Summicron at f/9.

 

So since I realized that their results are interesting but don't reflect the real world, and that they don't seem interested in checking their results, I decided to clean up my act and figure out just what the 'proper' aperture sequence is. At least I'll learn something.

 

You know, when Erwin or Sean says something that others question, they both are glad to recheck themselves and correct their results or show why they don't need correction. I don't care for the "I'm right and you're wrong" attitude I see at dpreview. Just personal opinion.

 

Say, didn't you and I both swear off this thread a couple days back?

 

 

Hmm. The Wikipedia article is nice and comprehensive, but the author makes the same error I did before John and SJP straightened me out:

Notice that sometimes a number is ambiguous; for example, f/1.2 may be used in either a half-stop or a one-third-stop system; sometimes f/1.3 and f/3.2 and other differences are used for the one-third stop scale.

That's the kind of redundancy I wanted to do away with. (Sigh.)

Share this post


Link to post
Share on other sites
Say, didn't you and I both swear off this thread a couple days back?

 

 

Not the thread, the IR speculation and DPReviews test results. I just didn't see the point of watching someone try to re-invent the wheel with f stop calculations. This was like week one M&P.

 

Yes they made a mistake when they said that IR only affects man-made materials. But my knowledge and deductive reasoning make me believe that a lack of an IR filter in that test couldn't cause that result. So what more can I say? It is either the basic image processing or the jpeg conversion. If it is not caused by the in-camera jpeg process then it may very slightly degrade raw images in certain situations. What this means in "typical" shooting is pretty hard to know.

Share this post


Link to post
Share on other sites

Howard

 

To reduce rounding error, etc., maybe you could short-cut for the higher f values by multiplying by 2^(n/6) instead of multiplying by 2^(1/6) n times over (the latter building up round-off and/or truncation errors with every multiplication). For example, sarting with f/1, three steps down the line would give 2^(3/6) = 2^(0.5) = squareroort of 2, and so on.

Share this post


Link to post
Share on other sites

Advertisement (gone after registration)

PS: I can't promise this will improve things - lord only knows what Excel does under the bonnet!

Share this post


Link to post
Share on other sites
... Yes they made a mistake when they said that IR only affects man-made materials. But my knowledge and deductive reasoning make me believe that a lack of an IR filter in that test couldn't cause that result. ...

Very logical, and a point on which many of us agree. But it's also a point that could easily be checked by reshooting with filter. Without that, we're still guessing.

... It is either the basic image processing or the jpeg conversion. ...

Those are possibilities, but we know the M8's basic image processing doesn't look like that, and whatever we think of the M8's JPGs, I think Michael Hußmann has raised good points against assigning blame for these results to the JPG algorithm.

What this means in "typical" shooting is pretty hard to know.

These images aren't representative of what we get out of the M8 every day; what this test means for general use is nothing.

I don't see how there could be a focus error as all of the color charts are shot at once. So how could one be blurry and the rest sharp? ...

I suppose the charts are shot sequentially, right, not simultaneously? Did the operator touch up focus between shots? If so, she may have mis-focused.

 

It would be nice to know why the dpreview test doesn't represent the real world. It could be any of a number of things, the most reasonable in my mind being focus.

 

But until dpreview reshoots the test controlling a number of variables, we are only speculating. dpreview isn't likely to reshoot because to them we're just a loud cloud of obnoxious flies complaining that our camera was maligned, when they weren't even testing our camera.

 

I have personally got enlightenment here on JPG algorithm and Foveon vs Bayer designs, and received instruction (FWIW) on properly calculating apertures, and I see those as positives.

 

dpreview's sloppiness in this case is apparent. In regard to the M8, their DP1 test is at best an anomaly; more likely, it's just more evidence of their inadequacy to perform at the level they pretend.

 

I think this earlier post was clear, succinct and accurate:

My guess would be that these tests are inconclusive crap.

Share this post


Link to post
Share on other sites
To reduce rounding error, etc., maybe you could short-cut for the higher f values by multiplying by 2^(n/6) instead of multiplying by 2^(1/6) n times over (the latter building up round-off and/or truncation errors with every multiplication).

 

Good point, John. That will reduce the concatenation of errors, as you say--but sadly, 2^(15/6) still rounds to 5.7. I'll check it further.

 

Seeing the following table of aperture designation styles used in 1899 (from F-number - Wikipedia, the free encyclopedia) certainly makes me thankful that we're dealing with the current system!

 

(Interesting that already in those early days, Zeiss was changing their minds about the best way to go.)

Share this post


Link to post
Share on other sites
PS: I can't promise this will improve things - lord only knows what Excel does under the bonnet!

 

I think you'll find Excel maintains full internal precision, rounding and truncation for presentation only.

Share this post


Link to post
Share on other sites
I think you'll find Excel maintains full internal precision, rounding and truncation for presentation only.

 

Point taken Mark, so apologies to Mr Gates. In fact 4sqrt(2) is 5.656854... which would be (correctly) rounded to 5.7. So I guess the photographic convention in marking-up aperture stops must be a simple truncation.

Share this post


Link to post
Share on other sites

The color test charts look to have been taken as single images showing all six charts. (On the M8 shot there are sensor dust spots that do not repeat on each card.) Plus, just looking at the downloaded file, they look to be two panels of three charts - placed on top of each other. So no re-focusing could have occured between the red/blue and black/white panels. A IR focus shift (probably requiring a lot of the illumination to be IR) would have affected the black and white panel also. Any color would have absorbed some of the IR and reflected less IR than the white would have. So the IR/UV filter would have made no difference except to very slightly degrade the overall image as all filters do. And how could the colors be so accurate if IR illumination were present and reflecting off of the subject?

 

Note that the Nikon D60 and the D40 also do poorer on the red/blue than they do on the other color combinations or on black/white. That is because on a Bayer pattern red/blue uses the least number of pixels. I think the key thing about this test is showing off the strength of the Foveon sensor which seems quite impressive in these examples.

 

I'm not sure why anyone needs to calculate the f stop with a great degree of precision. But when you are done doing that you can start measuring the exact focal length of your lenses.

Share this post


Link to post
Share on other sites

Thanks for the clarification of your contention, Alan. This kind of irrelevant test seldom interests me, and this specific one is so unrepresentative of reality that I haven't bothered to download the file(s). If I understand you correctly, I think you've just raised more credibility issues against the dpreview procedure. I've already responded a couple times to your first-paragraph IR questions.

 

I'm not sure why anyone needs to calculate the f stop with a great degree of precision. But when you are done doing that you can start measuring the exact focal length of your lenses.

 

1) I wonder why you would think I might be interested in whether you understand why I'm curious about the aperture calculation sequence. You're clearly confused, since if you've been reading along, it should be clear that I'm indeed not interested in "calculating the f stop with a great degree of precision." If you recall, dpreview claimed they were setting the 50 Summicron to an aperture that it would be impossible to set accurately or repeatedly. (I'm actually wondering why I have to keep explaining this to you.

) When I pointed that out, some nice folks on the forum helped me to refine my calculation.

 

2) If in future I should develop an interest in the exact focal lengths of my lenses, Leitz has already helpfully engraved that information on some of them. Maybe they're getting ahead of you.

Share this post


Link to post
Share on other sites

Well... you wrote, "I suppose the charts are shot sequentially, right, not simultaneously? Did the operator touch up focus between shots? If so, she may have mis-focused."

 

So I thought I'd give my view.

 

I just re-read the review and here is what they said,"For direct comparisons we always use sharp prime lenses stopped down, typically to F9 for 35 mm lenses." So maybe the M8 is atypical and they used f8.

 

I guess what I am trying to understand is why do you think it matters if DPReview could or could not set all of their test lenses accurately and repeatedly to f9. Or if it is good enough to just come close?

Share this post


Link to post
Share on other sites

Alan, if your doctor tells you he's going to remove your tonsils but removes your uvula instead, he will lose some credibility with you.

 

When (now you say "if," where previously you argued that dpreview might have found a way to set a Summicron to f/9) a test agency claims to use a particular aperture, even though it's virtually impossible to set that aperture on the lens in question, the review loses some credibility because the statement is likely untrue.

 

Also untrue is their contention that the M8's 'IR problem' exists only with certain man-made materials.

 

They say their lights are daylight-balanced >98% CRI. But since daylight normally contains a fair amount of IR, I want more specificity. (That's even more the case when they deny that IR could have affected the results and at the same time show they don't understand the M8's 'IR problem.' Before they can deny the effect of IR, they need to run a test (easier) or demonstrate the spectral range of their lamps.) The one specification I read said that daylight compatibility required a certain amount of UV--Absorban to the rescue here!--but said nothing about presence or absence of IR.

 

These are to me all examples of the sloppiness of the dpreview test procedure. The more discrepancies they display, the less trustworthy the test is.

 

You keep posting quotes but never a link to verify. When you've "seen the light," unless I can see the sources of your information, I have a hard time following your logic. That's why I thanked you for clarifying your contention: I finally had enough information to get on the same page with you. The data have been there all along, but I hadn't paid them the attention you had.

 

Like all of us, I'm curious why their test doesn't represent the M8's real-world performance. Since in this case the M8 was only an adjunct to the test of the DP1, it wouldn't be worth their time to re-test the M8 to satisfy complaining M8 users. (I'm sure they also get complaints from people who own other brands when their cameras don't show up well.)

 

They were testing the DP1, and found that its optical performance was surprisingly good, but that its overall performance was a bit lacking. Okay, fine, that's about what other reviewers are saying.

 

Their review is fine for general conclusions such as that.

 

But it's too sloppy overall for us to try to deduce why the M8 doesn't show up better.

Share this post


Link to post
Share on other sites

Oh. One more error, just discovered, from Sigma DP1 Review: 20. Compared to...:

Lighting: daylight simulation, >98% CRI.
But CRI is a scale, not a percentage. (Google "define:Color Rendering Index")

 

We can assess any individual element greater or less weight, but the more mistakes in a test, the less validity I grant the specifics of the test.

 

IMHO it's a sloppy test and not worth the bandwidth we've used here.

Share this post


Link to post
Share on other sites

Oh. One more error, just discovered, at Sigma DP1 Review: 20. Compared to...:

Lighting: daylight simulation, >98% CRI.
But CRI is a scale, not a percentage. (Google "define:Color Rendering Index")

 

We can assess any individual element greater or less weight, but the more mistakes in a test, the less validity I grant the specifics of the test.

 

IMHO it's a sloppy test and not worth the bandwidth we've used here.

Share this post


Link to post
Share on other sites

The M8 doesn't show good? Just because someone from DP says so?

And you've drawn this conclusion from an inconclusive test which shows "no real tests"?

Do you shoot jpegs with your M8?

Share this post


Link to post
Share on other sites

Well Howard,

 

I have nothing against the test being repeated with an IR filter if you think it will make a difference. Is that what you are saying?

 

CRI does not mean that a given bulb duplicates the spectral response of daylight or tungsten light sources. And the CRI only addresses light, not invisible regions of the spectrum. Here is GE's explanation of CRI. " To help indicate how colors will appear under different light sources, a system was devised some years ago that mathematically compares how a light source shifts the location of eight specified pastel colors on a version of the C.I.E. color space as compared to the same colors lighted by a reference source of the same Color Temperature."

 

Learn About Light: Color Rendering: GE Commercial Lighting Products

 

You asked for links. So here are three spectral charts - daylight, tungsten and "daylight" fluorescent. Obviously the fluorescent is much different than daylight yet it can still be used for color photogrphy. Note how little energy is put out at the red end of the spectrum on the fluorescent bulb. So IR will be very low from that.

Share this post


Link to post
Share on other sites
The color test charts look to have been taken as single images showing all six charts.

 

I think this is right.

 

looking at the downloaded file, they look to be two panels of three charts - placed on top of each other. So no re-focusing could have occured between the red/blue and black/white panels.

 

Also right, I think

 

A IR focus shift (probably requiring a lot of the illumination to be IR) would have affected the black and white panel also. Any color would have absorbed some of the IR and reflected less IR than the white would have.

 

Here's where I don't agree. This depends on the paint or dye used to create the color panels. Some black paints reflect IR; some don't. This is one of the reasons the M8's IR problem was noticed in the first place. The same is true for whites - and other colors. And for dyes. I'd love to see the spectral curves for the various color swatches outside of the visible region.

 

So the IR/UV filter would have made no difference except to very slightly degrade the overall image as all filters do.

 

I'm prepared to believe that IR wasn't the cause of this problem, but nothing DPreview has said and nothing you've said leads me to believe that this statement has been firmly established. I'd like to see a test with and without IR filter, and a test with RAW and JPEG.

 

I've also not seen a diagram of the spectrum of the lights DPreview actually used.

 

Note that the Nikon D60 and the D40 also do poorer on the red/blue than they do on the other color combinations or on black/white. That is because on a Bayer pattern red/blue uses the least number of pixels. I think the key thing about this test is showing off the strength of the Foveon sensor which seems quite impressive in these examples.

 

I think this may be the case (and I think your judgment is better informed than mine on the point).

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
Sign in to follow this  

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...