DXO sensor mark--how is it done?

[nugat]

Advertisement (gone after registration)

I cannot understand from the DXO site explanations how they arrive at the DXO sensor mark that makes 5D better with mark 70, than M8 with 58. I see it consists of three situational test: landscape, portrait and sports relating to DR, color depth, ISO.

How they mix the three to get the compound mark? How they measure the three "submarks"? (they call them "metrics"??? what's a "metric"?). I have a slight feeling now that it's like comparing cars based on three factors: speed,cargo capacity, comfort--all merged in one.

[sandymc]

I have a slight feeling now that it's like comparing cars based on three factors: speed,cargo capacity, comfort--all merged in one.

 

Yes.

 

Sandy

[SJP]

It is based on some sort of weighted average using the results they managed to obtain by some sort of method.

[plasticman]

It is based on some sort of weighted average using the results they managed to obtain by some sort of method.

 

And the method is obviously wrong because my camera did not win.

 

(but it will be totally correct if I change camera to one with a higher score).

[ho_co]

I think you've hit the nail on the head. It's another carefully constructed web site with pretty graphs.

 

Maybe DxO wants to increase site traffic in hopes of selling more software?

 

dpreview is today celebrating a year of IMO meaningless but snazzy-looking lens tests.

 

I think it's just what you do if you have a web site. At one time I had a roommate who would rearrange the furniture once a week. Probably the same personality type at work here.

 

And for the rest of us, as Sean says, "it's another data point."

[sean_reid]

And for the rest of us, as Sean says, "it's another data point."

 

Are you confused about why I suggest that DXO is another data point?

 

Cheers,

 

Sean

[AlanG]

Advertisement (gone after registration)

I think you've hit the nail on the head. It's another carefully constructed web site with pretty graphs.

 

Maybe DxO wants to increase site traffic in hopes of selling more software?

 

dpreview is today celebrating a year of IMO meaningless but snazzy-looking lens tests.

 

I think it's just what you do if you have a web site. At one time I had a roommate who would rearrange the furniture once a week. Probably the same personality type at work here.

 

And for the rest of us, as Sean says, "it's another data point."

 

I thought the whole point of the dpreview site was to test and compare gear. What else do you expect them to do? DXO Mark is simply the equivalent of testing that was done using film... by manufacturers and scientists. Some cold evening, you might want to cozy up to "Photographic Sensitometry" by Hollis Todd and Richard Zakia.

[nugat]

I thought the whole point of the dpreview site was to test and compare gear. What else do you expect them to do? DXO Mark is simply the equivalent of testing that was done using film... by manufacturers and scientists. Some cold evening, you might want to cozy up to "Photographic Sensitometry" by Hollis Todd and Richard Zakia.

 

The thing is when the film was the medium it was analyzed within one size standard, usually 24x36.

DXO has a "sensormark" that puts one measure to sensors of vastly different sizes.

If 5D FF is 100% area, then Leica M8 is 56% and G1 is 25%. But somehow they stop from throwing the medium format in.

The DXO sensor mark consists of three submarks ("metrics") and I'm not quite clear how they are weighed for the final score. However it is obvious that ISO "metric" constitutes a large part of the DXOmark. Eg with 5D, M8 and G1 the results are below.

I recalculated each score into % mark accordning to their own scale:

 

 

Canon EOS 5D

DxOMark Sensor

70.9/100 or 71%

Color Depth

22.9/26 88%

Dynamic Range

11.1/15 74%

Low-Light ISO

1368/2526 54%

 

Leica M8

DxOMark Sensor

59.4/100 59%

Color Depth

21.1/26 81%

Dynamic Range

11.3/15 75%

Low-Light ISO

842/2526 33%

 

Panasonic Lumix DMC G1

DxOMark Sensor

53/100 53%

Color Depth

21.1/26 81%

Dynamic Range

10.3/15 69%

Low-Light ISO

463/2526 18%

 

Where the sensor size plays lesser role the results are really close: color depth 88%, 81%, 81%; DR 74%, 75%, 69%. For ISO they obviously differ dramatically: 54%, 33%, 18%. But remember the sensor size: Full, Half, Quarter. So the discovery is that four times bigger area vessel catches more rain .

Also it seems that the class valedictorian has a crooked GPA. Actually it holds the arythmetical avarage of 72%, but the countercandidates to laurels (m8, G1) averages should be higher at 69% and 56% respectively (they get 59% and 53% from DXO). Not mentioning the fact that one of the subjects, phys ed, favored obviously the guy with 4 times bigger cardiovascular capacity.

Indeed, there are"lies, big lies and statistics".

[tashley]

So the discovery is that four times bigger area vessel catches more rain .

Also it seems that the class valedictorian has a crooked GPA. Actually it holds the arythmetical avarage of 72%, but the countercandidates to laurels (m8, G1) averages should be higher at 69% and 56% respectively (they get 59% and 53% from DXO). Not mentioning the fact that one of the subjects, phys ed, favored obviously the guy with 4 times bigger cardiovascular capacity.

Indeed, there are"lies, big lies and statistics".

 

Thank you. Brain training and humour in one post. A perfect ten.

(or maybe 9.67?)

[nugat]

One correction. I mistyped M8 average score as 69.

It is 63.

[delander †]

My M8 already has an inferiority complex, cant handle all the new toys on the block, this stuff has sent it into a deep depression.

 

Jeff

[ho_co]

You can set up a procedure to test anything you want and then report it. Procedural validity may be of less interest, so long as people cite your site.

 

Thanks for the thread and analysis, nugat.

[sean_reid]

I

And for the rest of us, as Sean says, "it's another data point."

 

Hi Howard,

 

Perhaps you missed my post above. What is confusing to you about my suggestion that DXO is another data point?

 

Cheers,

 

Sean

[ho_co]

Didn't miss the post, Sean. Just couldn't figure out where you got the idea you were confusing.

 

Clearly it's another data point. Some rearrange furniture, others enter the data. Different strokes, different folks. Sorry if you took the smiley as relating to your comment rather than to the thread. "For the rest of us, it's another data point." Then, after putting myself in your camp, I gave you credit for the comment.

[sean_reid]

Oh, I thought you were confused by that because you added a "confused" emoticon at the end of the sentence.

 

Web conversations are so much more confusing than real life conversations.

 

Cheers,

 

Sean

[AlanG]

The thing is when the film was the medium it was analyzed within one size standard, usually 24x36.

.

 

I wasn't aware of that. We used sensitometers, densitometers, and microdensitometers that would have no way of knowing the film format.

[nugat]

I wasn't aware of that. We used sensitometers, densitometers, and microdensitometers that would have no way of knowing the film format.

 

My poor example indeed. A method for film comparisons can be unaware of the format.

But throwing into one bag CCDs, LiveMOS, CMOS of varying size and construction and then making one ranking out of them result in a dangerous and eventually meaningless oversimplification, IMHO. It remains a marketing curiosity or a beauty contest at most.

Miss Universe is more fun, at least for me.

[AlanG]

My poor example indeed. A method for film comparisons can be unaware of the format.

But throwing into one bag CCDs, LiveMOS, CMOS of varying size and construction and then making one ranking out of them result in a dangerous and eventually meaningless oversimplification, IMHO.

 

They assign an overall rating for the cameras based on how DXO may value various aspects. That is why they give you a detailed breakdown. So you can decide what aspects of the sensor are important to you. For instance, if you are only going to be shooting at a low ISO, then cameras that test well for high ISO may not rate as highly to you. Remember this is a test of raw sensor performance only. It is not a camera test.

[nugat]

I appreciate the effort put into the detailed sections of the test. They can give some guidance for certain aspects. I dispute the all-inclusive DxOsensormark. I could not find how the elements are weighed in there, it doesn't look like through arithmetical averages. The ISO part only states the obvious--big CMOS with onboard processing=lower high ISO noise. If they had categories of sensor size and type all results within categories would equalize.

I understand marketing needs drama. Such as comparing D3x to G1, though it makes as much sense as doing this with the Maybach and Toyota Corolla.

[Advertisement]

Hallo,

Look here: Leica M

[Rick]

Oh, I thought you were confused by that because you added a "confused" emoticon at the end of the sentence.

 

Web conversations are so much more confusing than real life conversations.

 

Cheers,

 

Sean

 

Sean..., Geez, it sounds like you're trying to keep Howard on a pretty short leash.

 

Nugat... I like the way your mind works, very good. That was a clear and logical post. I now have a better understanding for how their system is actually set up to "not" factor sensor size into their conclusions. Funny how we think that statistics have a meaning of themselves.

 

Here are some numbers from my own research:

 

137%

-34.7%

 

These numbers represent the amount of fun and satisfaction I got from carrying around and actually taking "real" pictures of things that caught my mind's fancy. I took no pictures of small watch faces or toys or fruit. I used 2 different classes of cameras (DSLR and RF). The test were all carried out while on my vacations. I averaged 3 factors on a sample size (n=3 vacations) and applied a "weighted scale" of my own design to the three factors I measured.*

 

*(I won't disclose the weighting formula but, I will state that I took points away for the size of the camera systems and wether or not the camera system made me feel like a I was walking around with a bazooka hanging from my neck).

 

My conclusion is that the M8(RF) out scored the DSLR used in this test by a factor of 167.7 So, the M8 must be better for everyone.

 

In all seriousness, I am happy with my M8 system (camera & lenses) and I don't have "Canon envy" but, I wouldn't mind a bigger bucket in my M8 to catch more rain and less noise.