Yes you can do pairwise comparison and ranking of subjective things (how long will this task take) to get a less arbitrary measure of it.
In the past I've done this for consumer product testing - if you ask someone to rate hair straightness out of 10... it's just really difficult and inconsistent. But you can show them two photos they can usually pick the best, even if it feels pretty arbitrary.
Bradley-Terry is the standard method, it's extremely easy to use.
I have never seen anyone try it for bug estimation though. I think it would be too much effort and tbh the industry hasn't even figured out that you need to be able to write down confidence for estimates yet. There's a long way to go before we start doing things properly.
Comments
Yes you can do pairwise comparison and ranking of subjective things (how long will this task take) to get a less arbitrary measure of it.
In the past I've done this for consumer product testing - if you ask someone to rate hair straightness out of 10... it's just really difficult and inconsistent. But you can show them two photos they can usually pick the best, even if it feels pretty arbitrary.
Bradley-Terry is the standard method, it's extremely easy to use.
I have never seen anyone try it for bug estimation though. I think it would be too much effort and tbh the industry hasn't even figured out that you need to be able to write down confidence for estimates yet. There's a long way to go before we start doing things properly.