Results of M8 Lockup/Failure Survey

[xrogers]

Advertisement (gone after registration)

My impression based on forum noise was that the failure rate was much higher than ten percent. This survey is useful to me because it tells me that ten percent failure is almost certainly an upper bound (the real rate is almost certainly lower than a forum survey), and that forum scuttlebutt (at least for me) provides a quite inaccurate impression of the camera's reliability.

 

Thanks, Sean!

 

Clyde Rogers

[Advertisement]

The Leica Q3 is available for pre-order at B&H Photo Video and Adorama

[barjohn]

Sandymc,

 

If the boot code is damaged and you can't perform a firmware update as you speculate in the firt part of your comments, then your last paragraph can't be correct. Usually, the boot code is protected by being in a separate sector or even a separate memory. Granted a static discharge could damage the boot sector memory rendering the camera dead without a trip to Solms.

 

Sean, did you ever hear from Leica what actually failed in your dead camera? Was it a hardware failure or a firmware crash that was unrecoverable? Since you stated in another post that you could not reflash the firmware it would appear to have also clobbered the boot firmware.

[xrogers]

I think Sandy is right on the money with his statements. I had one "white screen" on the LCD when hopping around reviewing and deleting images while the camera was jpg processing (camera set to jpg+dng). I immediately stopped messing around, let the camera finish writing, and things straightened out. All was well, but ever since I've assumed the firmware has race conditions. Nowadays I avoid making the camera multitask, and let it finish writing photos before doing anything with the review or settings.

 

Until later,

 

Clyde Rogers

[sandymc]

John,

 

What I'm talking about is not really the boot code getting damaged - that's quite unusual, because the boot code isn't changed under normal circumstances. But operating data is changed. The problem is more likely to be data getting corrupted by only one part of a structure getting written - any structure that's larger than the physical width of the memory has to be written in sequence. If that sequence is disturbed, or stopped half way through the sequence, you can get corruption.

 

Almost certainly, there's a factory-only "deep reflash" function in the M8, that only accessible if you have the right equipment/software. This would not only reflash the firmware code, but would also reflash the data sections of the memory, which the kind of firmware reload that you or I could do would not. But then you lose all data in camera, e.g., serial numbers, etc

 

Sandy

[sean_reid]

Sorry Sean but it's meaninless. It's a self selecting survey. To find the true figure for breakdowns you need to select people at random and question them.

 

Steve,

 

I don't think you read my text.

 

Sean

[sean_reid]

Let's be fair to Sean: He is aware of the limtations of his statistics. And I am convinced Leica is reading this with great interest! As I said, my worry is the tendency to read selectively that a number of users, mainly on other forums than this one, have exhibited in the whole M8 hullabuloo.

 

Thank you. Not only am I aware of the limitations but it sounds like many people here looked at the tables without reading the text. I was very specific about the implications of the results and the degree to which they could or could not be generalized. I would encourage everyone to carefully read the full text before responding on this thread.

 

If anyone has access to a larger, randomly-selected sample of M8 owners (who can be verified as M8 owners) then, by all means, please publish your results. Otherwise, one works with the best information available and notes its limitations.

 

Cheers,

 

Sean

[sean_reid]

Advertisement (gone after registration)

Sean,

 

Thanks for this.

 

I have a first generation M8 with 1.09 firmware. I've had to turn it off to undo a freeze, but never had to take out the battery. I didn't respond to your poll. ( guilty as charged)

 

The ten percent figure is probably way high because people like me with no problem are less likely to respond than people with a problem. And people with a problem are more likely to join the forum.

 

But, you have added to our and Leica's knowledge.

 

Thanks again,

 

Mitchell

 

Thanks Mitchell. I repeatedly asked people to respond to that survey even if their M8s had not experienced the problem. I can only work with what I got.

 

Cheers,

 

Sean

[sean_reid]

Part of this may also be do to the reset function that Germany recommended recently of resetting the camera because of the battery bar status firmware issue. Folks may have thought they had a full battery but in reality the battery may have been below operating capaciity. So there may have been a curve ball in there on this alone. Interesting results though and hopefully soon some of these failures will be minimized.

 

Great work Sean on collecting this data and a good reference point for Leica to see.

 

Thank you.

 

Cheers,

 

Sean

[sean_reid]

Sean,

 

Thanks for all of your efforts and posting the results for the benefit of forum members. As you stated in your text, it's difficult to draw any hard conclusions based on the sample size. (Some respondents to your post should read what you wrote a little more carefully.)

 

Larry

 

Hi Larry,

 

Exactly! I find it troubling that so many people seem to have looked at those numbers without carefully reading the text. I tried to gather data that would be useful to Leica and to us and then I noted its limitations in my analysis. That's the best one can do.

 

To know how statistically significant the sample size is one would need to know how many M8s had been sold so far. I'll try to find out.

 

Cheers,

 

Sean

[sean_reid]

As a comment, the descriptions of the lockups that I've read on this forum sound more like problems with the firmware than quality problems with the manufacture of the M8. What tends to happen with products like the M8 is that internal data structures get corrupted, and the camera locks up trying to make sense of nonsense. There are usually multiple layers of data in the camera, ranging from stuff that is kept only when the power is on (e.g., where in menu structure you are) through settings (like lens detection on/off) that would be irritating but not fatal to lose, down to very low level stuff like number of shutter accentuations, that Leica would not want reset under any normal circumstance. The problem is that typically the lowest level of data isn't reset by a "cold" reset, so if that gets corrupted, the only way to get the camera back into working order is a full reflash of all memory. And loading new firmware, btw, would also not reset this data.

 

Memory corruption would, for example, be a danger if the M8 finds itself trying to process events that it wouldn't normally see. Sean's static hypothesis is quite plausible in this regard. While it quite easy to protect a camera against physical damage from static, a charge of static can easily generate spurious sensor readings. So, for example, if the M8 has both a shutter cocked and a shutter uncocked sensor, static could cause both to flip on simultaneously. The M8 then tries to process both a cocked and non-cocked event at the same time, corrupts its own data, and that's it.

 

These kind of problems are avoidable in firmware in an number of ways - check codes on data, "atomic commits", etc, but if the firmware development was rushed, as has been suggested elsewhere, corruption due to unusual events is a typical area to see problems.

 

If this is the case, the bad news is that anyone hoping that their M8 is a "good" one is probably wrong - it's just a matter of time. The good news is that this is a problem that can be corrected by revised firmware only, meaning that your M8 won't have to go back to Leica to be as reliable as any other digital camera, unless it actually does lock up.

 

 

Sandy

 

Hi Sandy,

 

That's very interesting.Thanks for the post. With your permission, I'll forward it directly to some key people at Leica. In the high-static conditions where I was doing the shoot, both M8s locked up. One came back and one didn't. I continue to be struck that they both showed the same symptoms within minutes of each other. I'm hopeful that the problem can be solved in firmware.

 

My gut sense (which certainly is not statistically significant <G>) is that the number of cameras that have experienced this failure has nothing to do with differences among the various units out there and much more to do with differences in usage, conditions, luck, etc. In other words, I suspect that this could happen to any M8 running 1.06 or 1.09 but that's mostly just intuition and speculation.

 

None of this would keep me from buying an M8 all over again. Leica just has another computer challenge to figure out.

 

Cheers,

 

Sean

[Guest guy_mancuso]

Very good point and how many of the total number post on this forum ,not all so it is really hard to come to any conclusive number but I agree it is at least a base point that can be looked at and really the final number does not count but the issues do and that is what i know Sean was after and something to be looked at and digested by Leica. This is one reason leica i know comes and reads these threads and also for us as end users to provide good data to work off of. i think and hope many memebers understand why many of us have been working hard and testing to bring the data back to be used to makes the issues fixable. Without the end users giving good data this would just slow that process down and stuff like this makes the answers more clear and gives direction for engineers to work on them.

[Jamie Roberts]

This what writes Sandy sound like most logic what is anyone tell. So only one hope for M8 depend Leica will to find new one partner who can to fix up bat firmware from before partner Jenoptik. If we look whats happen it to DMR when Leica loose partner with Imacon we mabe need scare. Smart costumer I thing wait how this finish before put himself money for M8.

 

Blasko--we know what you think before you post already.

 

While what Sandy says about low-level computer processes is true, it's *also* true that simple edge conditions or occasional bad manufacture also give rise to the same conditions of DOA electronics.

 

What Sandy says may be logical, but it's also pure speculation, as are interpreting the results of the survey. While it's good work, ethnographically, and as it relates to *people on this forum,* without a defined manufacture number or random selection of participants it's statistically useless.

 

That doesn't mean it's useless--and Sean pointed this out, bless him. Just that there's no CAUSAL or quantifiable relationship here, ok? Steve got this completely right, statistically.

[sandymc]

Sean,

 

By all means forward, although I would think that my speculation (Jamie's right, it is pure speculation without having access to Leica's failure data) would not be any surprise to Leica

 

Sandy

[sean_reid]

My impression based on forum noise was that the failure rate was much higher than ten percent. This survey is useful to me because it tells me that ten percent failure is almost certainly an upper bound (the real rate is almost certainly lower than a forum survey), and that forum scuttlebutt (at least for me) provides a quite inaccurate impression of the camera's reliability.

 

Thanks, Sean!

 

Clyde Rogers

 

You're welcome and thanks for the comments. We know that, according to this self-report data, 9% of 140 M8s failed. As people who read my text carefully will know, we cannot know how that figure relates to the total number of M8s sold. Given the tendency for bad news to travel faster than good (as I also discussed), the actual failure rate may be well under 10%. That said, its possible that any production M8 (firmware 1.06 or 1.09) could potentially experience non-recoverable lockup under exactly the wrong conditions. I suspect static may be part of those conditions. It also seems very possible that a firmware change could reduce or eliminate this risk.

 

Cheers,

 

Sean

[sean_reid]

I haven't published yet but, for those who are interested, here's a revision of the (latter) text that also emphasizes some of the same points I made to Mark Norton in a PM a couple of weeks ago (in response to some accusations he made):

 

Although I did break down the survey responses according to what version of the M8 the owner had worked with, we can see that the responses are fairly consistent between the two most prevalent types of M8. Fifty-six percent of "First Generation" M8 owners reported "no failure" as compared to sixty-two percent of "Second Generation" owners. That difference is minor enough, especially given this small sample size, that it could be attributed to random variation. Those same two groups also show similar response percentage relationships for the other two categories. The sample sizes for the other two types of M8 were so small that we can't reliably conclude anything about how the type of M8 might be related to these problems.

 

In short, then, the results presented here don't suggest any significant relationship between type of M8 and the prevalence of the problems being discussed. As such, its the overall percentages that are most relevant to consider. The majority of M8 cameras reported on, 61 percent, had not shown evidence of lock-up or failure. Just under a third, 30 percent, of the cameras had experienced problems that required a reboot (battery removal) and just under a tenth, 9 percent, failed and needed to be sent to Leica.

 

If this was a much larger sample, say 1000 cameras or more, we might be a better position to estimate that about ten percent of M8s have failed and required repair. In this sample of 140 cameras, however, 9 percent actually represents only 13 cameras. That result can't tell us how many M8s in the larger population are likely to have failed in this way but it certainly does confirm that the problem has occurred in more than just a couple of cameras.

 

I think its very important to realize that, with the DMR and M8, Leica has become, in part, a computer company.That moves them far beyond their core areas of experience and expertise. The other Leica digital cameras, although designed by Leica and Panasonic together, have been built by Panasonic - one the largest manufacturers of electronics in the world. I have no doubt that Leica will come to be a very good computer company but there's still a lot of digital experience the company needs to develop. It could not be otherwise. They can beta test as much as they want but nothing teaches a company more than producing products and having them used by thousands of people. That's not only true for Leica, it is true for any company. Mathematically we can understand that thousands of people using a product will end up exposing that product to conditions, and combinations of conditions, that no amount of pre-production testing could create. This is a challenge even for experienced digital camera makers such as Canon and Nikon.

 

There have been some who have expressed dismay that the M8 has had some electronic teething problems since its introduction and who have wondered, aloud and in print, as to how such a prestigious company could release a product with these problems. But that perspective is somewhat disingenuous. In the areas where Leica has earned its prestige: lenses, finder quality, rangefinder mechanisms, build and materials quality, etc. - the M8 and M system remain of very high quality. But Oscar Barnack was not a computer maker and Leica is going to need some time and practice to become as proficient with computers as they are with cameras and optics. In some respects, they are one of the most experienced camera makers world. But, with respect to computers, they are one of the least experienced. That's a reality that some may have trouble fathoming or accepting but I believe that it is very much the case.

 

I'm told that Leica is investigating this lockup concern and I await further information from them about what solutions are possible. Thirteen confirmed camera failures shouldn't be enough to alarm current or potential M8 owners but it is important information for Leica to consider. While I was disappointed to see this failure in my own camera, I still am very happy overall with the M8 and would repeat my purchase without hesitation. I think its important that Leica continue to work on sorting out problems with the M8 so that it can earn a reputation for reliability. Leica certainly has been responsive to concerns and is actively working on solutions and improvements for this camera. So far, I think they're making good progress. My M8 continues to be my primary and favorite camera.

[stunsworth]

Steve,

 

I don't think you read my text.

 

Sean I skimmed over it because as I say the basis on which it was formed is flawed. The only conclusion you can draw from the results is that some M8s were delivered faulty - something that is to be expected in any manufacturing process You need to ask people at random to get any kind of meaningful response regarding failure rate. The figure in the chart at the end would appear to indicate that nearly half the respondents had a problem. That just isn't so in reality - if it were Leica would be out of business. It's as scientific a survey as the Mac v PC one recently which would appear to indicate that 90% to 95% of the people here use Macs.

[sean_reid]

Sandymc,

 

If the boot code is damaged and you can't perform a firmware update as you speculate in the firt part of your comments, then your last paragraph can't be correct. Usually, the boot code is protected by being in a separate sector or even a separate memory. Granted a static discharge could damage the boot sector memory rendering the camera dead without a trip to Solms.

 

Sean, did you ever hear from Leica what actually failed in your dead camera? Was it a hardware failure or a firmware crash that was unrecoverable? Since you stated in another post that you could not reflash the firmware it would appear to have also clobbered the boot firmware.

 

Hi John,

 

I have not yet heard anything on that but will be discussing it with them again today.

 

Cheers,

 

Sean

[jrc]

Sorry Sean but it's meaninless. It's a self selecting survey. To find the true figure for breakdowns you need to select people at random and question them.

 

I disagree. You need random selection if you want to use statistical tests to determine some characteristic of a population when the population is too big to account for all cases. You don't need statistical tests when you can survey 100 percent of some population.

 

You could consider the active M8-owning membership of this forum to be a population. Most didn't come here to report problems; most came here to discuss the M8, and I'd argue that most seem to have a tendency to play down problems.

 

What we know about this population is that a relatively large percentage of it [M8-owning active forum members] has had this problem. I wouldn't want to try to quantify it as Sean has, because the numbers implies a precision that doesn't exist. However, the numbers aren't meaningless: they are indeterminate, but large.

 

If, for example, you want to take an even smaller population, professional photographers who have frequented this forum since before the camera was released (i.e. before you could complain about problems), then, as close as I can tell, 100% of them have experienced the problem. That doesn't mean that 100 percent of all professional photographers have had the problem, but it does mean that of those you know, a more than expected number have had the problem. (With that small a sample, with Leica-level QC, you might expect the possibility of a defect, but you wouldn't expect that all of them would see one.)

 

JC

[sean_reid]

Sean,

 

By all means forward, although I would think that my speculation (Jamie's right, it is pure speculation without having access to Leica's failure data) would not be any surprise to Leica

 

Sandy

 

I doubt they would be surprised but I always keep my open to the possibility that someone might suggest a direction for investigation that the engineers had not yet considered, even though they are being very thorough.

 

Cheers,

 

Sean

[Advertisement]

Hallo,

Look here: Interesting Leica gear

[sean_reid]

Sean I skimmed over it because as I say the basis on which it was formed is flawed. The only conclusion you can draw from the results is that some M8s were delivered faulty - something that is to be expected in any manufacturing process You need to ask people at random to get any kind of meaningful response regarding failure rate. The figure in the chart at the end would appear to indicate that nearly half the respondents had a problem. That just isn't so in reality - if it were Leica would be out of business. It's as scientific a survey as the Mac v PC one recently which would appear to indicate that 90% to 95% of the people here use Macs.

 

Steve,

 

Before you write anything else, please read it. In fact, I'll be more emphatic here. If people choose to look at the tables without reading the text carefully, then I can't respond seriously to the resulting comments.

 

This data tells us something about a specific set of 140 M8s. That's information we didn't have yesterday and it has some value.

 

We have a couple of options here folks. We can try to learn something from the limited information I was able to gather here, or we can ignore it and someone can try to do a random survey of thousands of M8 owners. If someone is able to do the latter, I'll be eager to read the results.

 

Sean