It was clear that Yahoo.com was the definitive result for the query "yahoo" so it would score a 10. Other Yahoo pages would be ok (perhaps a 5 or 6). Irrelevant pages stuffed with Yahoo-related keywords would be spam.
As someone who worked on search quality at Google for some time, this bit jumped out at me as a terrible mistake. The correct way to judge results for the query [yahoo] is:
(a) Where is yahoo.com? At the top?
(b) There is no (b).
It seems like a slight difference, but it leads to the wrong priorities. For the query [yahoo], it does not matter if spam or non-spam is in spot #5. The only thing that matters is where you put yahoo.com.
Most people don't look farther than the first 3 results, so what comes after that isn't very high priority. Might be the reason some more specific/obscure queries on google return terrible results :)
Comments
It was clear that Yahoo.com was the definitive result for the query "yahoo" so it would score a 10. Other Yahoo pages would be ok (perhaps a 5 or 6). Irrelevant pages stuffed with Yahoo-related keywords would be spam.
As someone who worked on search quality at Google for some time, this bit jumped out at me as a terrible mistake. The correct way to judge results for the query [yahoo] is:
(a) Where is yahoo.com? At the top?
(b) There is no (b).
It seems like a slight difference, but it leads to the wrong priorities. For the query [yahoo], it does not matter if spam or non-spam is in spot #5. The only thing that matters is where you put yahoo.com.
Can you elaborate a bit more on this? I think I understand your point but a few more details would be great. Thanks!
Most people don't look farther than the first 3 results, so what comes after that isn't very high priority. Might be the reason some more specific/obscure queries on google return terrible results :)
More specifically, if people find what they're looking for in the first result, they ignore the rest.