It's been too long since I've taken statistics to be sure, but I wonder if you could account for this by randomly placing the options in the top position then comparing the results of when A is on top versus when B is on top?
Yes, that's a very good idea (and actually the one I had in mind when I talked about correcting the bias). The real problem is when avoidance of surveys correlates with one of the survey answers, like "Should we start using micro surveys?".
Comments
It's been too long since I've taken statistics to be sure, but I wonder if you could account for this by randomly placing the options in the top position then comparing the results of when A is on top versus when B is on top?
Yes, that's a very good idea (and actually the one I had in mind when I talked about correcting the bias). The real problem is when avoidance of surveys correlates with one of the survey answers, like "Should we start using micro surveys?".