“Annotator agreement” is a measure of confidence in the correctness of labels. And you should always keep an eye out for how these are handled, when reading papers that present a dataset.
Saying we should start doing model agreement is a really good idea imho.
Comments
You do that with human annotators.
“Annotator agreement” is a measure of confidence in the correctness of labels. And you should always keep an eye out for how these are handled, when reading papers that present a dataset.
Saying we should start doing model agreement is a really good idea imho.