When a feedback instrument surveys eight colleagues about your business acumen, your score of 3.79 is far greater a distortion than if it simply surveyed one person about you—the 3.79 number is all noise, no signal.
Which implies to me that they believe there is signal there, but that it goes away when aggregated?
I think by "surveyed" they don't mean "asked one person for a score" but rather got some overall information from one person including their qualitative feelings and perceptions. There is signal in those as they discuss elsewhere in the article, but the quantitative rating allegedly has no value even when averaging. That's the charitable reading, anyway.
Yeah, I'd like to see what statistical theory they are using here. I don't think it's sound. It's unfortunate since I think the article is otherwise quite good.
Comments
They say later that
Which implies to me that they believe there is signal there, but that it goes away when aggregated?
I think by "surveyed" they don't mean "asked one person for a score" but rather got some overall information from one person including their qualitative feelings and perceptions. There is signal in those as they discuss elsewhere in the article, but the quantitative rating allegedly has no value even when averaging. That's the charitable reading, anyway.
Yeah, I'd like to see what statistical theory they are using here. I don't think it's sound. It's unfortunate since I think the article is otherwise quite good.