Programs that raise test scores also do objective good

Economists like to use improvements in standardized tests as a measure of how well education programs are working. This commonly leads to dissent on the grounds that standardized tests don’t really measure anything of value. My go-to response to that critique has long been that these exam scores are predictive of objectively important outcomes. Change the test scores (in a meaningful way) and you will shift those outcomes as well.

A new paper by Will Dobbie and Roland Fryer, “The Medium-Term Impacts of High-Achieving Charter Schools on Non-Test Score Outcomes“, makes exactly that point. Participation in the Promise Academy (which famously raised students’ test scores in the short term, by half a standard deviation in math and a fifth of an SD in English) led to big gains in important outcomes in the medium term. Their abstract:

High-performing charter schools can significantly increase the test scores of poor urban students. It is unclear whether these test score gains translate into improved outcomes later in life. We estimate the effects of high-performing charter schools on human capital, risky behaviors, and health outcomes using survey data from the Promise Academy in the Harlem Children’s Zone. Six years after the random admissions lottery, youth offered admission to the Promise Academy middle school score 0.283 standard deviations higher on a nationally-normed math achievement test and are 14.1 percentage points more likely to enroll in college. Admitted females are 12.1 percentage points less likely to be pregnant in their teens, and males are 4.3 percentage points less likely to be incarcerated. We find little impact of the Promise Academy on self-reported health. We conclude with speculative evidence that high-performing schools may be sufficient to significantly improve human capital and reduce certain risky behaviors among the poor.

I was lucky enough to meet with Fryer when he came to Michigan to give a talk. One thing he said is that his research agenda is motivated by the robust link between racial gaps in test scores and racial gaps in outcomes. Fix the black-white gap in test scores, he said, and you could fix almost all of the gap in crime, teen pregnancy, and so forth. This is the first time, however, that I’ve seen compelling evidence that the observed relationship between test scores and overall quality of life holds up when you actually get low-scoring kids to do better in school.

Granted, a lot of other stuff could be going on. The Promise Academy surely had other effects beyond academics. But this study provides important evidence in favor of using test score improvements as a meaningful measure of improvements in education.