The article mentions the importance of having a diverse group of people running the study, but more specifically you need multiple people coding or tagging the data independently, and then you need to run analysis on inter-coder reliability to measure the degree to which those independent coders' judgments agree with each other.
You cannot reliably quantify qualitative data without assessing inter-coder reliability.
It's a great point. It's not the next post I'm working on, but the one after that is explicitly about ensuring reliability, both with regards to the initial coder and other researchers, and the same researchers over time, and how much inter-coder reliability matters.
Comments
The article mentions the importance of having a diverse group of people running the study, but more specifically you need multiple people coding or tagging the data independently, and then you need to run analysis on inter-coder reliability to measure the degree to which those independent coders' judgments agree with each other.
You cannot reliably quantify qualitative data without assessing inter-coder reliability.
It's a great point. It's not the next post I'm working on, but the one after that is explicitly about ensuring reliability, both with regards to the initial coder and other researchers, and the same researchers over time, and how much inter-coder reliability matters.