I'm surprised he didn't break it down to probabilities according to answer distributions instead of right vs wrong. For instance, given 2 questions with their answers and the percent of the class that answered for each choice:
1. A is the answer
A - 60%
B - 20%
C - 5%
D - 15%
2. C is the answer
A - 10%
B - 5%
C - 70%
D - 15%
The probability of randomly selecting 2 students that both chose:
- A and C (both correct) is 0.1764
- D and B (both wrong) is 0.000056
That doesn't necessarily work because wrong answers can have natural correlations. Let's say you have two questions about finding a slope on an exam. Anyone who thinks that slope is change in x over change in y is going to get both of those questions wrong in the same way, even if those wrong answers are both individually rare. It would be really hard to distinguish between when these correlations are expected and when they're unexpected.
Comments
I'm surprised he didn't break it down to probabilities according to answer distributions instead of right vs wrong. For instance, given 2 questions with their answers and the percent of the class that answered for each choice:
1. A is the answer A - 60% B - 20% C - 5% D - 15%
2. C is the answer A - 10% B - 5% C - 70% D - 15%
The probability of randomly selecting 2 students that both chose:
- A and C (both correct) is 0.1764 - D and B (both wrong) is 0.000056
* Appreciate corrections
That doesn't necessarily work because wrong answers can have natural correlations. Let's say you have two questions about finding a slope on an exam. Anyone who thinks that slope is change in x over change in y is going to get both of those questions wrong in the same way, even if those wrong answers are both individually rare. It would be really hard to distinguish between when these correlations are expected and when they're unexpected.