We know D.C. Get our free newsletter to stay in the know.
Update: Thanks to DCPS spokeswoman Jennifer Calloway for providing this response:
We recognize that there will be many criticisms of our teacher evaluation system, IMPACT. We will not shy away from addressing any criticism candidly and directly, because we have tremendous confidence in the work. IMPACT was developed in partnership with Mathematica Policy Research, one of the longest-standing and most well-respected research firms in the nation with specific expertise in value-added modeling. Simply put, the value-added model used in IMPACT is valid.
Calloway also gave LL some names of experts who she said could refute Pallas’ claims. LL will try to call them as soon as possible.
Over at the Post blog The Answer Sheet, Columbia University Professor Aaron Pallas slams the teacher rating system DCPS used to justify some of the teacher firings announced last week.
There’s no polite way to say this: The procedures described in the DCPS IMPACT Guidebook for producing a value-added score are idiotic.
…
According to the DCPS IMPACT Guidebook, the actual growth is a student’s scaled score at the end of a given year minus his or her scaled score at the end of the prior year. If a fifth-grader received a scaled score of 535 in math and a score of 448 on the fourth-grade test the previous year, his actual gain would be calculated as 87 points.
Subtracting one score from another only makes sense if the two scores are on the same scale. We wouldn’t, for example, subtract 448 apples from 535 oranges and expect an interpretable result. But that’s exactly what the DC value-added approach is doing: Subtracting values from scales that aren’t comparable.
…
A fifth-grade student who got every question wrong on the reading test at the end of fourth grade and every question wrong at the end of fifth grade would show an actual gain of 500–400=100 points.
A fifth-grader repeating fifth grade who had a scaled score of 510 the first time through, and a scaled score of 530 during his or her second year in fifth grade, would show an actual gain of just 20 points. DC’s value-added methods may, of course, simply exclude students who are retained in grade from the calculations, but that sends an unpleasant message about whose scores count when teachers are evaluated.
Did DCPS completely botch the calculation of value-added scores for teachers, and then use these erroneous scores to justify firing 26 teachers and lay the groundwork for firing hundreds more next year?
According to the only published account of how these scores were calculated, the answer, shockingly, is yes.
Those are some pretty heavy charges being made against school Chancellor Michelle Rhee’s school reform efforts. LL is certainly no expert in the field, and it wouldn’t be fair to make any conclusions until DCPS gets a chance to respond. But, wow, if DCPS did indeed “botch” its calculations for determining which teachers got canned, mayoral hopeful Vincent Gray might actually be able to say publicly what he’d do with Rhee if wins the election.