Over the past couple of months, I’ve been blogging less than usual – depriving my readers of my valuable opinions, such as why certain statistical methods are terrible. In large part this is because I’ve been working at major revisions to multiple papers, which has eaten up a large fraction of my writing energy.
Rebecca Thornton and I just finished one of those revisions, to a paper that began its life as the third chapter of my dissertation. The revised version is called “Making the Grade: The Trade-off between Efficiency and Effectiveness in Improving Student Learning“. Here is the abstract:
Relatively small changes to the inputs used in education programs can drastically change their effectiveness if there are large trade-offs between effectiveness and efficiency in the production of education. We study these trade-offs using an experimental evaluation of a literacy program in Uganda that provides teachers with professional development, classroom materials, and support. When implemented as designed, and at full cost, the program improves reading by 0.64 SDs and writing by 0.45 SDs. An adapted program with reduced costs instead yields statistically-insignificant effects on reading – and large negative effects on writing. Detailed classroom observations provide some evidence on the mechanisms driving the results, but mediation analyses show that teacher and student behavior can account for only 6 percent of the differences in effectiveness. Machine-learning results suggest that the education production function involves important nonlinearities and complementarities – which could make education programs highly sensitive to small input changes. Given the sensitivity of treatment effects to small changes in inputs, the literature on education interventions – which focuses overwhelmingly on stripped-down programs and individual inputs – could systematically underestimate the total gains from investing in schools.