Why 'Fixing Teachers' Is As 'Unrealistic' As 'Fixing Poverty'

I've spent the last two days discussing the problems with value-added teacher evaluation, and I thought I would turn it over to the readers, since there has been some really good discussion. At the end, I'll revisit some statistical and methodological issues, but I want to address a good question raised by becca--and, in other posts about education, this question has been raised:

But do you really think it's easier to fix poverty than it is to get good teachers?

Two responses to that. First, it's not clear to me that teacher quality plays anything more than a minor role--and a transitory one at that (as I noted yesterday). If teacher quality had an effect on par with that of poverty, we would have detected it rather easily by now. But the other issue is one raised by Mokele:

It's not an issue of "easy", it's an issue of whether it'll work at all. IMHO, better teachers won't accomplish much, maybe raise things a few percentage points, tops. As is so often the case, the *real* solution which would lead to major improvement is also the most difficult, expensive, and time-consuming. We need to stop looking for quick fixes and band-aids, and start addressing the real issues.

At this point, people will say it's hard to 'solve' poverty. And it is. But there are programs that can ameliorate its effects. Summer reading programs that counteract the increased academic decline among poor children would help, along with other summer enrichment activities*. For that matter, we could make sure that kids have adequate nutrition year round. There are other interventions that already have had society-wide effects: the removal of lead from the environment being the most noticeable. In Tennessee, one school district is starting the school day a half-hour later, and anecdotally (it's too early to do any rigorous analysis), the students are performing better.

This isn't to say that a child with parents for whom English isn't a first language will start at the same place as a child with two very-well educated parents. But we can do much more to provide safe, healthy environments for children.

The flip side of this is that improving teachers isn't trivial. Consider this estimate of what it would take to remove 'bad teachers' from a school system:

When they ran the numbers, the answer their computer spat out had them reviewing their work looking for programming errors. The optimal rate of firing produced by the simulation simply seemed too high: Maximizing teacher performance required that 80 percent of new teachers be fired after two years' probation....

Their point isn't that we can or should fire 80 percent of new hires, but that their work should be seen as a "thought experiment" on the extreme measures that would be required to really improve American education, provided we can't figure out how to find better teachers at the get-go or develop reliable methods of improving teachers once they're in the system.

While I've called for ongoing teacher training (and it's something Massachusetts emphasizes relative to many other states), none of the educational 'reformers' have talked about how we're supposed to find all of these available teachers--or, for that matter, how we're supposed to make poorly performing ones better. Maybe there is a surplus pool of teachers waiting in the wings, but I doubt that, certainly not without some good salaries and benefits.

Finally, some methodological issues. First, John Hawks, along with Tim Bartik, argue that there's a regression to the mean problem. Bartik:

For example, he finds that the FIFTH-grade teacher you are assigned is correlated with your FOURTH-grade "value added" gains. Different teachers get assigned different types of students. Furthermore, there is a strong indication in his data of reversion to the mean in test score levels. That is, students who have high value-added gains in 4th grade tend to have lower value-added gains in 5th grade. Therefore, 5th-grade teachers who for whatever reason happen to get assigned a great many students who had high value-added gains in 4th grade will tend to have lower value-added gains. These lower value-added gains are obviously not solely due to the 5th-grade teacher.

And John Hawks:

The cases that feature in the article all look consistent with regression to the mean -- an award-winning teacher with high scoring students has a slight decline, one with very low scoring students has a marked increase. If those were selected for the story because they represent the extremes, there's no story there.

I've never seen that these statistics are actually showing anything other than the effect of sample size. They keep saying that "having the right teacher makes more difference than the right school", but a classroom is a much smaller sample than a school, so naturally there's more statistical variance among them.

If students are shuffled year-to-year, the classrooms ought to start with something nearer the mean, and the variance among classrooms ought to increase by year end, if the teachers are actually doing anything at all.

Sounds testable to me. Finally, there's a potential curriculum effect:

I'd be interested to know if the v-a methodology takes into account that California standard curriculum introduces new material to students in the odd grades (1st, 3rd, 5th, etc) and (more or less) repeats the material in the even grades, making it easier to appear to be effective.

This definitely wasn't mentioned in the article.

I've said it before, but I'll say it again: some states do very well compared to other countries, and much better than expected given their poverty rates. Copy what those states are doing, instead of trying out untested ideas.