Foreign exchange and false advertising

I’m currently in Johannesburg, en route to Malawi to work on one project that is close to the end of data collection, and another project still in the field (note to self: neither of these are on my “Work in Progress” page! Time to update that.) Malawi recently introduced a $75 visa fee (which you can’t pay in the local currency, amusingly) for Americans that I always forget about until I am already outside the US. So I had to change some Rand into dollars.

Changing currencies is much more complex and expensive than it looks. I went up to four different ForEx counters at JNB, and each one had huge hidden fees for my transaction – 25%, on top of their aggressive exchange rate. This has always been my experience with these counters: these fees are not on their signs, and are often hidden in the transaction amount.

I suspect that there is a behavioral-type story here: it’s quite hard to detect how bad you are getting gouged on fees and commissions given all the arithmetic you need to do just to translate currencies. Firms can exploit the difficulty of understanding these fees to extract rents from their less-sophisticated customers. Laibson and Gabaix’s 2006 QJE paper is the classic reference on this idea.

In Laibson and Gabaix, though, sophisticates can dodge the hidden fees. There is no “out” at the ForEx counter – everyone has their own secret fee schedule. But there is a side effect: everyone who walks up is getting gouged. So my solution was to wait for another person who was offered a shockingly terrible rate and beat that price. Worked like a charm, and now I can get into Malawi.

The lesson of this experience, by the way, is the opposite of what you might think: never bring US dollars with you abroad! Or, rather, bring only what you need for the fees and a small emergency fund. Changing cash abroad will get you killed on fees.

Making the Grade: The Trade-off between Efficiency and Effectiveness in Improving Student Learning

Over the past couple of months, I’ve been blogging less than usual – depriving my readers of my valuable opinions, such as why certain statistical methods are terrible. In large part this is because I’ve been working at major revisions to multiple papers, which has eaten up a large fraction of my writing energy.

Rebecca Thornton and I just finished one of those revisions, to a paper that began its life as the third chapter of my dissertation. The revised version is called “Making the Grade: The Trade-off between Efficiency and Effectiveness in Improving Student Learning“. Here is the abstract:

Relatively small changes to the inputs used in education programs can drastically change their effectiveness if there are large trade-offs between effectiveness and efficiency in the production of education. We study these trade-offs using an experimental evaluation of a literacy program in Uganda that provides teachers with professional development, classroom materials, and support. When implemented as designed, and at full cost, the program improves reading by 0.64 SDs and writing by 0.45 SDs. An adapted program with reduced costs instead yields statistically-insignificant effects on reading – and large negative effects on writing. Detailed classroom observations provide some evidence on the mechanisms driving the results, but mediation analyses show that teacher and student behavior can account for only 6 percent of the differences in effectiveness. Machine-learning results suggest that the education production function involves important nonlinearities and complementarities – which could make education programs highly sensitive to small input changes. Given the sensitivity of treatment effects to small changes in inputs, the literature on education interventions – which focuses overwhelmingly on stripped-down programs and individual inputs – could systematically underestimate the total gains from investing in schools.

The latest version of the paper is available on my website here. We have also posted the paper to SSRN (link).

Real and fake good news from the 2015-16 Malawi DHS

Malawi’s National Statistical Office and ICF International recently released a report containing the main findings from the 2015-16 Demographic and Health Survey for Malawi. The DHS is an amazing resource for researchers studying developing countries – it provides nationally representative repeated cross-section data for dozens of countries. Tons of amazing papers have come out of the DHS data. I am a particular fan of Oster (2012), which shows that the limited behavior change in response to HIV risks is explained, at least in part, by other health risks. (A shout-out to IPUMS-DHS here at the University of Minnesota, which lets researchers quickly access standardized data across countries and time).

Another great use of the DHS is to look at trend lines – what health and social indicators are getting better? Are any getting worse? There is some really good news in here for Malawi: the total fertility rate (the number of children a woman can expect to have over her lifetime) has declined significantly in the past 6 years, from 5.7 to 4.4. This is a big drop, and it comes from declines in fertility at all ages:

DHS fertility trends

You can argue about whether lower fertility is necessarily better, but it seems to go hand-in-hand with improving economic and social indicators across the board, and reducing fertility probably causes improvements in women’s empowerment and access to the labor market. Moreover, the average desired family size in Malawi is 3.7 kids, so at least in the aggregate Malawians would also see this decline as an improvement.

There’s another eye-catching headline number in the report, which would be amazingly good news – the prevalence of HIV has fallen from 10.6% to 8.8%. That’s a huge drop! As long as it didn’t happen due to high mortality among the HIV-positive population, that’s really good news. Only it didn’t actually happen. What did happen was that they changed their testing algorithm to reflect the current best practice, which uses an additional test to reduce the false-positive rate. They have the data to compute the prevalence in the old way, too. Here’s what that looks like:

DHS trends in HIV

Almost totally flat. If anything there’s a small improvement for women and a slight increase in prevalence for men, but we can’t rule out zero change. This shouldn’t be a surprise, though – people can increasingly access life-saving ARVs in Malawi, which tends to push up the prevalence of HIV because HIV-positive people stay alive.

To the credit of the people who put together the report, they never present trendlines that compare the old method to the new one. People who assemble cross-country summary stats from the DHS are likely to be misled, however. It would probably be better to mention both numbers anywhere the prevalence is mentioned, since they are both valid summaries of the data done slightly different ways – and 10.4% is the number than is more comparable to previous DHS waves.

There is a nugget of good news about HIV buried in here, as well – these findings imply that the prevalence of HIV was never quite as high as we thought it was, in Malawi or anywhere else. Taking this discrepancy as a guide, we have been overstating the number of HIV-positive people worldwide by about 15%. It is definitely a good thing that those people aren’t infected, and that’s a 15% reduction in the cost of reaching the WHO’s goal of providing ARVs to every HIV-positive person in the world.

You can’t be your own friend, and that’s a big problem for measuring peer effects

Peer effects – the impact of your friends, colleagues, and neighbors on your own behavior – are important in many areas of social science. To analyze peer effects we usually estimate equations of the form

Y_i=\alpha+\beta PeerAbility+e_i

We want to know the value of β – how much does an increase in your peers’ performance raise your own performance?

Peer effects are notoriously difficult to measure: if you see that smokers tend to be friends with other smokers, is that because smoking rubs off on your friend? Or because smokers tend to come from similar demographic groups? Even if you can show that the these selections problems don’t affect your data, you face an issue that Charles Manski called the “reflection problem”: if having a high-achieving friend raises my test scores, then my higher test scores should in turn raise his scores, and so on, so the magnitude of the peer effects is hard to pin down.

A standard way of addressing these problems is to randomly assign people to peers, and to use a measure of performance that is measured ex ante or otherwise unaffected by reflection. That fixes the problems, so we get consistent estimates of beta, right?

Wrong. We still have a subtle problem whose importance wasn’t formally raised until 2009 in a paper by Guryan, Kroft, and Notowidigdo: you can’t be your own friend, or your own peer, or your own neighbor. Suppose our setting is assigning students a study partner , and the outcome we are interested in is test scores. We want to know the impact of having a higher-ability peer (as measured by the most recent previous test score) on future test scores. The fact that you can’t be your own peer creates a mechanical negative correlation between each student’s ability and that of their assigned peer. To see why, imagine assigning the peer for the highest-ability student in the class. Any partner she is assigned to – even if we choose entirely at random from the other students – will have a lower score on the most-recent test than her. And for any student who is above-average, their assigned peer will, on average, be lower-ability than them. The reverse applies to students who are below the class average.

This is a big problem for estimating beta in the equation above. The error term ei can be broken up into a part that is driven by student ability, OwnAbilityi, and a remaining component, vi. Since OwnAbilityi is negatively correlated with PeerAbilityi, so is the overall error term. Hence, even in our random experiment, we have a classic case of omitted-variable bias. The estimated effect of your peers’ ability on your own performance is biased downward – it is an underestimate, and often a very large one.

What this means is that randomized experiments are not enough. If you randomly assign people to peers and estimate β using the equation above, you will get the wrong answer. Fortunately, there are solutions. In a new paper, Bet Caeyers and Marcel Fafchamps describe this “can’t be your own friend” problem in detail, calling it “exclusion bias”. They show that several common econometric approaches actually make the problem worse. For example, controlling for cluster fixed effects often exacerbates the bias because the clusters are often correlated with the groups used to draw the peers. They also show that 2SLS estimates of peer effects do not suffer from exclusion bias – which helps explain why 2SLS estimates of peer effects are often larger than OLS estimates.

They also show how to get unbiased estimates of peer effects for different kinds of network structure. Unfortunately there is no simple answer – the approach that works depends closely on the kind of data that you have. But the paper is a fantastic resource for anyone who wants to get consistent estimates of the effect of people’s peers on their own performance.

The quality of the data depends on people’s incentives

Two recent news stories show how sensitive social science is to issues of data quality. According to John Kennedy and Shi Yaojiang, a large share of the missing women in China actually aren’t missing at all. Instead, their parents and local officials either never registered their births or registered them late. Vincent Galoso reports that Cuba’s remarkable infant mortality rate is partly attributable to doctors re-coding deaths in the first 28 days of life as deaths in the last few weeks of gestation.

Both of these data problems affect important scientific debates. The cost-effectiveness of Cuban health care is the envy of the world and has prompted research into how they do it and discussions of how we should trade off freedom and health. China’s missing women are an even bigger issue. Amartya Sen’s original book on the topic has over 1000 citations, and there are probably dozens of lines of research studying the causes and consequences of missing women – many of whom may in fact not be missing at all.

I am not sure that either of these reports is totally correct. What I am sure about is that each of these patterns must be going on to some extent. If officials in China can hit a heavily-promoted population target by hiding births, of course some of them will do so. Likewise, if parents can avoid a fine by lying about their kids, they are going to do that. And in a patriarchal culture, registering boys and giving them the associated rights makes more sense than registering girls. The same set of incentives holds in Cuba: doctors can hit their infant mortality targets either by improving health outcomes, by preventing less-healthy fetuses from coming to term, or by making some minor changes to paperwork. It stands to reason that people will do the latter at least some of the time.

Morton Jerven points out a similar issue in his phenomenal work Poor Numbers. Macroeconomic data for Africa is based on very spotty primary sources, and the resulting public datasets have errors that are driven by various people’s incentives – even the simple incentive to avoid missing data. These errors have real consequences: there is an extensive literature that uses these datasets to estimate cross-country growth regressions, which have played an important role in policy debates.

At my first job after college, my boss, Grecia Marrufo, told me that variables are only recorded correctly if someone is getting paid to get them right. She was referring to the fact that in health insurance data, lots of stuff isn’t important for payments and so it has mistakes. There is a stronger version of this claim, though: if someone is being coerced to get data wrong, the data will be wrong. And anytime people’s incentives aren’t aligned with getting the right answers, you will get systematic mistakes. I’ve seen this myself while running surveys; due to various intrinsic and extrinsic motivations, enumerators try to finish surveys quickly and end up faking data.

I’m not sure there is anything we can do to prevent fake data from corrupting social scientific research, but I have a couple of ideas that I think would help. First, always cross-check data against other sources when you can. Second, use primary data – and understand how it was collected, by whom, and for what reason – whenever possible. Neither of these can perfectly protect us from chasing fake results down rabbit holes, but they will help a lot. In empirical microeconomics, I have seen a lot of progress on both fronts: important results are debated vigorously and challenged using other data, and more people are collecting their own data. But we still have to be vigilant, and aware of the potential data reporting biases that could be driving results we regard as well-established.

Income Timing, Savings Constraints, and Temptation Spending

Why don’t poor people save more money? This topic – the barriers to saving money that poor people in developing countries face – is one of my major interests within development economics. I’ve written on this blog about the fact that people in poor countries who don’t have bank accounts don’t seem to want them, and that an inability to scare up small amounts of cash can literally be deadly. I’ve also written about clever approaches people have come up with to save more money.

Studying savings constraints has also been one of my major lines of academic research for the past few years. I’ve focused on a novel tool for getting around those constraints: in developing countries, employees often ask for their pay to be withheld from their regular paychecks and paid all at once in a lump sum.

Lasse Brune and I have just finished revising the first paper to come out of this research agenda. We find that deferred lump-sum wages increase people’s savings. This result is most likely due to savings constraints: people face an effective negative interest rate on money they save. (Potential reasons for negative interest rates include the possibility money can be lost or stolen, kin taxes, and the temptation to splurge on impulse purchases – though we find no evidence for the latter.)  The paper, “Income Timing, Savings Constraints, and Temptation Spending: Evidence From a Randomized Field Experiment”, is now available on SSRN. Here is the abstract:

We study a savings technology that is popular but underutilized in developing countries: short-term deferred compensation, in which workers receive a single, later lump sum instead of more frequent installments. Workers who are randomly assigned to lump-sum payments reduce the share of income they spend immediately by 25%, and increase short-term cash holdings by a third. They are 5 percentage points more likely to purchase an artificial “bond” offered through the study. These effects are most likely due to savings constraints: 72% of workers prefer deferred payments, and rationalizing workers’ choices without savings constraints requires implausibly low discount factors. Although workers report that temptation spending is an important driver of savings constraints, we find little evidence for that mechanism. Employers could enhance workers’ welfare at limited cost by offering deferred wage payments.

The paper can also be found on my website here and the online appendix is here. We posted it to SSRN and also on this blog because we are in the final stages of revising it to submit to academic journals – so we would be appreciate any feedback or suggestions you might have.

In addition, Lasse and I, along with Eric Chyn, are working on a new project that uses this idea to develop an actual savings product that we are offering to workers at the Lujeri Tea Estate. Workers can opt in to deferring a portion of their wages from each paycheck into a later lump sum payment. The project is currently entering baseline data collection, and early indications are that demand for deferred wages is high. We look forward to seeing how the product performs.

Story on NPR about my paper, “Scared Straight or Scared to Death?”

I was recently interviewed by NPR’s Shankar Vedantam (the host of Hidden Brain) for a story about my paper “Scared Straight or Scared to Death? The Effect of Risk Beliefs on Risky Behaviors”. The story ran today on Morning Edition, and you can find it online here:

How Risk Affects The Way People Think About Their Health

The story does a nice job of overviewing the key finding in my paper, which is that overstating the risks of an activity too much can backfire, causing people to give up and stop trying to protect themselves. This is the opposite of the usual pattern we observe.

It glosses over an important detail about the context and my empirical findings, which is that the fatalism result holds for some people, not everyone. Anthropologists have observed rationally fatalistic reasoning among some men in Malawi, not all of them. In my sample, I find fatalistic responses for 14% of people – the ones with the highest risk beliefs. I also find that those people are less likely to think they already have (or will inevitably contract) HIV, and that they are at higher risk for contracting and spreading HIV than the rest of the population.

I assume that NPR simply shortened the story for time, and I do think that their takeaway is the right one – we should be cautious when trying to scare people into better behavior by playing up how risky certain activities are.

Kerwin_Mpyupyu
Mpyupyu Hill in Zomba District, Southern Malawi – I collected the data for the paper nearby

You can find the latest version of the paper on my website or on SSRN. Here’s the abstract:

This paper tests a model in which risk compensation can be “fatalistic”: higher risks lead to more risk-taking, rather than less. If exposure cannot be perfectly controlled, fatalism arises when risks become sufficiently high. To test the model I randomize the provision of information about HIV transmission rates in Malawi, and use a novel method to decompose the risk elasticity of sexual risk-taking by people’s initial risk beliefs. Matching the model’s predictions, this elasticity varies from -2.3 for the lowest initial beliefs to 2.9 for the highest beliefs. Fatalistic people, who have a positive elasticity, comprise 14% of the population.

For more details about the paper, see my previous posts about it on this blog (first post second post) or on the World Bank’s Development Impact blog (link).

How are wages set for teachers, in theory?

A recent post by Don Boudreaux on the relative wages of actors and teachers has been doing the rounds in the economics blogosphere, garnering favorable mentions by Alex Tabarrok and Ranil Dissanayake. Boudreaux asserts that:

The lower pay of fire fighters and school teachers simply reflects the happy reality that we’re blessed with a much larger supply of superb first-responders and educators than we are of superb jocks and thespians.

There are lots of reasons why superstar actors and actresses earn tons of money. There really is a limited supply of high-end talent there, and that really does, in all likelihood, drive the high wages we observe. And Boudreaux is also right that it is good that the average firefighter or teacher isn’t paid millions of dollars, or we’d never be able to afford enough to fight all our fires and teach all our kids.

What Boudreaux gets wrong, however, is his assumption that straightforward supply and demand can possibly explain the pay earned by teachers. He’s also wrong when he asserts that the lack of high pay for awesome teachers is a good thing.

In the standard microeconomic theory of the labor market, workers earn hourly* wages equal to their marginal revenue product of labor. This is the revenue generated for the firm by the worker’s last hour of work. A worker’s wage is pinned down at their marginal revenue product (MRP) by a no-arbitrage condition. Strictly speaking, workers are assumed to be paid the value of their outside option to their current job.** In a competitive labor market, a worker that paid less than her MRP will be stolen away by another firm who is willing to pay her slightly more, and this repeats until wages reach MRP. A worker that is paid more than her MRP is losing money for her firm and will be fired (or the contract will never be offered).

You can see the problem for setting the wages of teachers. Public-school teachers don’t directly generate any revenue for doing their job, and public schools are not businesses and aren’t trying to maximize profits. Even private-school teachers don’t actually generate more income for their schools by putting in additional hours or being more effective in the classroom.

So how are these wages set? I don’t know the reality, but what the theory actually says is that they get whatever the best competing job is willing to pay them.*** This means the effective floor is not set by the marginal revenue product of labor, or any measure of productivity or effectiveness, but by some alternative job to teaching.

You could imagine competitors paying teachers wages that are based on their classroom effectiveness. Replacing a terrible teacher with an average one is worth $250,000 per classroom just in terms of additional lifetime earnings (Chetty, Friedman, and Rockoff 2014). And great teachers add value in many other ways that don’t directly appear in earnings. So this could lead to parents strongly preferring excellent teachers and paying a premium to private schools to get them. This would bid up wages for great teachers and we’d see very high pay for them.

Rating teachers on classroom effectiveness is hard, though. The Chetty, Friedman, and Rockoff results I linked to are very controversial. There are definitely aspects of teacher ability that cannot be measured through test scores alone. Kane and Staiger (2012) show that expert observations of teachers in the classroom have incremental predictive power for future student performance, on top of Chetty-style value-added measures. Some teachers oppose any use of test scores in evaluating teacher ability.

So back to the question. Is the low pay (and low variance of pay) for teachers a sign that the world is full of awesome teachers? No; they are quite rare. Rather, we have tons of people who could be adequate teachers, some of whom are amazing teachers — and we have little ability to distinguish between them for the purposes of pay, at least in a way that people can agree upon.

And this has consequences. Our limited ability to tie teacher pay to teacher quality means that there are probably lots of potentially-great teachers in other professions. Our limited ability to even measure quality teaching in an agreed-upon way means it’s tough to incentivize improvements in teaching quality among existing teachers – or even for teachers who are motivated by non-pay considerations to know how they should improve. The low and flat pay for teachers is not a blessing. It is a problem that needs to be fixed.

*I picked hourly wages arbitrarily – daily or weekly or annual wages would work just as well, because we use calculus for these models and assume time and effort at work can be varied continuously.
** We assume this is a job, but it could be the value they put on their free time instead. Workers whose labor generates little revenue are said to “choose” “leisure” over work, rather than driven into unemployment. Again, this is a model – not the truth, and not what I believe to be correct.
***This sets a floor on wages. The ceiling imposed by the typical model of wage-setting doesn’t function here because there is no marginal revenue to compare to the marginal cost of employing the teacher. However, pressure to control public budgets probably keeps wages reasonably near their floor.

The unbanked don’t *want* to be banked

Bank accounts as currently offered appear unappealing to the majority of individuals in our three samples of unbanked, rural households – even when these accounts are completely subsidized.

That’s the punchline of a new paper by Dupas, Karlan, Robinson, and Ubfal, “Banking the Unbanked? Evidence from three countries”. In both developing countries and the rich world there is a lot of justified concern about the unbanked population – households with no formal financial accounts, and a popular theory that if we could just improve the accessibility of bank accounts this would be a game-changer for people. Unfortunately, like many other silver bullets before it, this one has failed to kill the stalking werewolf of poverty.

Indeed, it almost doesn’t leave the barrel of the gun. 60% of the treatment group in Malawi and Uganda (and 94% in Chile) never touch the bank accounts. The following depressing graph is the CDF of deposits for people who opened an account:

Deposits

In Malawi, even among account openers, 80% of people had less money in total deposits than the account would have charged in fees (the fees were covered as part of the study). Just a tiny fraction of people had enough in deposits for the interest to cover the fees – which is the minimum for the account to compete with cash on the nominal interest rate.*

The goal of the study was to look at how accounts impact downstream outcomes like incomes and investments, and the paper dutifully reports a host of null effects on those variables. But the authors instead focus on why uptake was so low. The most popular theory among the treatment group was poverty – that people just don’t have enough money to save – and this lines up with some regression analysis results as well. But confusion also appears to be a factor: 15% of treatment-group households in Malawi say they didn’t use the account because they couldn’t meet the minimum balance, even though there was no minimum balance requirement.

Another issue with the poverty model of low savings is that it contrasts with my mental model of income dynamics in Malawi. The overwhelmingly dominant staple in Malawi is maize, and it is harvested more or less all at once in May, generating a huge burst of (in-kind) income that must be smoothed over the rest of the year. Maize is hard to store and storage has significant scale economies, so people often sell off a lot of their harvest and buy maize later with the cash – effectively “renting” storage. This requires lots of cash to be saved.

I can’t quickly put any numbers on this mental model (although maybe I’ll play around with the LSMS to see what it shows). I am quite confident about the spike in income at the harvest, though – the question is how much people need to save in cash. I’d like to see more discussion of this in future work about financial access in Africa. Dupas et al. may not have the data to do it, but future research projects should definitely collect it.

*This assumes away the problem of cash being lost, stolen, or wasted, which would make the nominal interest rate negative.

New website (and a new home for my blog)

I recently changed my personal website over to WordPress. (I had previously been writing the HTML by hand, which was a pain when I made major updates.) An added benefit of that changeover is that it enabled me to start hosting my blog, Ceteris Non Paribus, on my own server, instead of housing it at WordPress.com.

This transition should look more or less seamless from the perspective of existing blog readers. My old blog, nonparibus.wordpress.com, now redirects to its new home on my website, jasonkerwin.com/nonparibus, and all the old posts are now stored here along with their comments.  All my new posts – including this one –  should automatically cross-post to the old blog, and therefore should show up for existing subscriptions through RSS, email, etc. To avoid missing posts in the future, however, you might want to update your subscription using the email or RSS subscription widgets on the right, or by using this direct link to the new RSS feed.