Jakob Nielsen does suggest [1] that only 5 users will get you about 80% of the problems in your software, and 15 will get you damn-near-everything. So 11 is pretty good.
Also, if you do have the resources to do, say, 100 user tests, you actually actively don't want to do them all at the same time or it will be a waste because they will all say the same thing.
If anything, this was overtested. There's a lot of redundancy in those results, just as Nielsen would predict. This is not a scientific experiment to determine the effect of a drug, and you do not always need or even necessarily want a 95% confidence interval, as the costs would exceed the benefits here.
I am a dev (though not on frontend/UI stuff). But that's exactly the point. Many users are for the most part pretty stupid, and that is reality. Unusable software is unusable.
Comments
Jakob Nielsen does suggest [1] that only 5 users will get you about 80% of the problems in your software, and 15 will get you damn-near-everything. So 11 is pretty good.
[1] http://www.useit.com/alertbox/20000319.html
Also, if you do have the resources to do, say, 100 user tests, you actually actively don't want to do them all at the same time or it will be a waste because they will all say the same thing.
If anything, this was overtested. There's a lot of redundancy in those results, just as Nielsen would predict. This is not a scientific experiment to determine the effect of a drug, and you do not always need or even necessarily want a 95% confidence interval, as the costs would exceed the benefits here.
There's a lot of redundancy in those results
Helps so devs can't reply, "Well, that guy was just stupid".
I am a dev (though not on frontend/UI stuff). But that's exactly the point. Many users are for the most part pretty stupid, and that is reality. Unusable software is unusable.
Even if your users are not stupid, designing as if they are is usually beneficial.