I am all for edumetrics but there doesn't seem to a way to get a good signal on a general "teaching skills" metric. Does such a metric even make sense? I would assume that a proper metric for a teacher would be dict like {"cares":"5*", "energy":4, "perceptibility":3, "subject_knowledge":5, general_knowledge:3}.
Furthermore I am not sure how school boards and schools will use the metrics. Should you fire a teacher because some data fit decided that you are a bad teacher? No way! Anyone who is willing to put in the energy and spend time with kids teaching them stuff should continue to do it. Metrics for self assessment YES, but metrics for firing teachers NO.
Also, the whole idea of "Value added" score has been called bullshit upon
here https://news.ycombinator.com/item?id=5059737 --> http://garyrubinstein.teachforus.org/2013/01/09/the-50-milli... .
[ quote: ... the correlation is so low that I, and many others who have created similar graphs, concluded that this kind of measurement is far from ready to be used for high-stakes purposes like determining salaries and for laying off senior teachers who are below average on this metric. ]
The author basically says that there is no correlation of the "value added" metric that a teacher brings from year to year.
This lack of correlation is masked in the report "Measures of Effective Teaching" because "they averaged the predicted and actual scores in five percentile groups. In doing this, they mask a lot of the variability that happens" to make it look as if "value added" is a good stable metric.
The author uses a bad graph to convince the reader that there is no correlation, even when (by his own admission, see the comments) one is present.
The year to year correlation is 0.3. The correlation across percentile groups is much higher because that increases the sample size and thereby reduces statistical noise.
The conclusion we can draw here is that measuring the performance of a single teacher based on a single class will yield a very large confidence interval. That's not the same as "bullshit".
If you want to understand this issue please take the time to look at the actual MET project report as well as other well done research. Gary Rubinstein's analysis seems very misleading (see comments to prior HN story). Apparently is is very controversial to even try to measure teacher effectiveness.
Comments
I am all for edumetrics but there doesn't seem to a way to get a good signal on a general "teaching skills" metric. Does such a metric even make sense? I would assume that a proper metric for a teacher would be dict like {"cares":"5*", "energy":4, "perceptibility":3, "subject_knowledge":5, general_knowledge:3}.
Furthermore I am not sure how school boards and schools will use the metrics. Should you fire a teacher because some data fit decided that you are a bad teacher? No way! Anyone who is willing to put in the energy and spend time with kids teaching them stuff should continue to do it. Metrics for self assessment YES, but metrics for firing teachers NO.
Also, the whole idea of "Value added" score has been called bullshit upon here https://news.ycombinator.com/item?id=5059737 --> http://garyrubinstein.teachforus.org/2013/01/09/the-50-milli... . [ quote: ... the correlation is so low that I, and many others who have created similar graphs, concluded that this kind of measurement is far from ready to be used for high-stakes purposes like determining salaries and for laying off senior teachers who are below average on this metric. ]
The author basically says that there is no correlation of the "value added" metric that a teacher brings from year to year.
This lack of correlation is masked in the report "Measures of Effective Teaching" because "they averaged the predicted and actual scores in five percentile groups. In doing this, they mask a lot of the variability that happens" to make it look as if "value added" is a good stable metric.
The author uses a bad graph to convince the reader that there is no correlation, even when (by his own admission, see the comments) one is present.
The year to year correlation is 0.3. The correlation across percentile groups is much higher because that increases the sample size and thereby reduces statistical noise.
The conclusion we can draw here is that measuring the performance of a single teacher based on a single class will yield a very large confidence interval. That's not the same as "bullshit".
If you want to understand this issue please take the time to look at the actual MET project report as well as other well done research. Gary Rubinstein's analysis seems very misleading (see comments to prior HN story). Apparently is is very controversial to even try to measure teacher effectiveness.
The author explicitly states there is a 24% correlation, and then says there is no correlation. Can he or she not make up their mind?