It's just that the culture of security in the industry seems to be more willing to lock people out of their account when they lose the factor they have than when they lose the factor they know.

It could be that losing what you have is easier for people to remedy than a forgotten password, but somehow I suspect that it probably comes down to which one generates the fewest angry customers/calls/support tickets. Maybe it's easier for us to blame ourselves if we forget to carry a token or our cell phone?