Ceteris Non Paribus

Ceteris Non Paribus is my personal blog, formerly hosted at nonparibus.wordpress.com and now found here. This blog is a place for me to put the ideas I have, and the stuff I come across, that I’ve managed to convince myself other people would be interested in seeing. See the About page for more on the reasons why I maintain a blog and the origin of the blog’s name.

My most recent posts can be found below, and a list of my most popular posts (based on recent views) is on the right.

Ceteris Non Paribus

Negative externalities in marriage markets and polygamy

Democracy in America says that it’s time to think about legalizing polygamous marriages:

If the state lacks a legitimate rationale for imposing on Americans a heterosexual definition of marriage, it seems pretty likely that it likewise lacks a legitimate rationale for imposing on Americans a monogamous definition of marriage. Conservatives have worried that same-sex marriage would somehow entail the ruination of the family as the foundation of society, but we have seen only the flowering of family values among same-sex households, the domestication of the gays. Whatever our fears about polyamorous marriage, I suspect we’ll find them similarly ill-founded.

I’m an economist (Well, a grad student. But still.) I don’t claim to know the state’s rationale for anything, let alone marriage policy. My take is that policy tends to emanate from what the median voter wants, rather than from any kind of cost-benefit analysis. But if the question is about the costs and benefits of gay marriage and polygamy, then indeed that calculus is quite different: polygamous marriage differs sharply from homosexual marriage in that causes a damaging imbalance in marriage markets.

To fix concepts, let’s be clear that when people say “polygamy” they mean “polygyny”, or the pairing of one husband with multiple wives. Polygyny is far more common, both across human history and today. It is sustainable only in fast-growing populations, for reasons that are obvious if you think about them for a second: the sex ratio among US adults is 1.00, meaning there is almost exactly one man for every woman. That means everyone can get married if we all happen to be straight and monogamous (and have sufficiently malleable standards for mates). Suppose the population were a fixed size, and the number of men exceeded the number of women by just one lonely guy. He’d be lonely indeed: the marriage market would not clear – you can only marry one person – and he’d be desperate. He would bid the effective price of a male partner down, way down, in order to get married at any cost. This would leave women with all the bargaining power, and still leave one man unmarried.*

If the population is growing, and men marry down in age, then polygyny can work out just fine. There’s always a new crop of young women to marry and everyone can find a spouse, even if some men take four wives. But population growth is very low in most of the developed world, which probably has much to do with its low rates of polygyny. In the modern US, any appreciable number of polygynous marriages would leave huge numbers of men out in the cold.

This would have negative effects on men in marriages as well, as those excluded from marriage fight for partners: “Don’t want to marry me? What if I raise the kids and also hold down three jobs?” Sometimes these desperate attempts actually create polyandry: one of my favorite courses in college was taught by a professor who research showed that polygyny in pre-modern China led to multiple-husband marriages. The fact that fundamentalist Mormons “discard their surplus boys” is not some random horrible thing they do, unrelated to their marriage practices. It is an essential component of polygynous marital patterns. The men in control of fundamentalist LDS communities do it in order to keep the marriage market favorable for themselves.

This is fundamentally different from gay marriage. Homosexuals and bisexuals are a small share of the US population, and homosexual identity is roughly equally common among men and women. That means that legalizing gay marriage, inasmuch as it encourages people to leave the heterosexual marriage market**, will not lead to imbalance. Legalizing polygamous marriages will.

The state has a compelling policy interest in discouraging plural marriages, inasmuch as such marriages are overwhelmingly polygynous and not polyandrous. A classical liberal would argue that you have the right to do whatever you like as long as you do not hurt others – this is why economists think that markets should operate free of government interference, most of the time. But sometimes what you do in a market harms others. If you build a bar next to my house, the noise affects me: it imposes a negative externality. In principle, even if all the abuses of women and children that so often accompany American polygamy were erased, the very institution of polygamy damages society as a whole. By leaving the remainder of the marriage market imbalanced, it harms not just the men left out of marriages but every male.

*This is not hypothetical – it’s a real issue, that actually happens. Kerwin Charles documented it in some black communities in the US, where high rates of incarceration among men have left women at a severe disadvantage. As any aspiring medical resident can tell you, matching markets are brutal. Any imbalance in the two sides makes life very unpleasant.
**It also seems unlikely that homosexuals would marry the opposite sex in the absence of a ban on their marrying, whereas people forbidden to marry multiple people will likely still marry one.

Not whether but how much

Last week I was lucky enough to attend the Hewlett Foundation’s Quality Education in Developing Countries (QEDC) conference in Uganda, which brought together both Hewlett-funded organizations running education interventions and outside researchers tasked with evaluating the projects. (My advisor and I are working with Mango Tree Uganda to evaluate their Primary Literacy Project.) Evaluation was one of the central themes of the conference, with a particular focus on learning from randomized controlled trials (RCTs). While RCTs are clearly the gold standard for evaluations nowadays, we nevertheless had a healthy discussion of their limitations. One area that got a lot of discussion was that while randomized trials are great for measuring the impact of a program, they typically tell you less about why a program did or did not work well.

We didn’t get into a more fundamental reason that RCTs are seeing pushback, however: the fact that they are framed as answering yes/no questions. Consider the perspective of someone working at an NGO considering an RCT framed that way. In that case a randomized trial is a complicated endeavor that costs a lot of effort and money and has only two possible outcomes: either you (1) learn that your intervention works, which is no surprise and life goes on as usual, or you (2) get told that your program is ineffective. In the latter case, you’re probably inclined to distrust the results: what the hell do researchers know about your program? Are they even measuring it correctly? Moreover, the results aren’t even particularly useful: as noted above, learning that your program isn’t working doesn’t tell you how to fix it.

This yes/no way of thinking about randomized trials is deeply flawed – they usually aren’t even that valuable for yes/no questions. If your question is “does this program we’re running do anything?” and the RCT tells you “no”, what it’s really saying is that no effect can be detected given the size of the sample used for the analysis. That’s not the same as telling you that your program doesn’t work; it’s the best possible estimate of the effect size given the data your collected, and telling you that the best guess is small enough that we can’t rule out no effect at all.

It is true that running a randomized trial will get you an unbiased answer to the “yes” side of the yes/no does-this-work question: if you find a statisticall significant effect, you can be fairly confident that it’s real. But it also tells you a whole lot more. First off, if properly done it will give you a quantitative answer to the question of what a given treatment does. Suppose you’re looking at raising vaccination rates, and the treatment group in your RCT has a rate that is 20 percentage points higher than the control group, significant at the 0.01 level. That’s not just “yes, it works”, it’s “it does about this much”. This is the best possible estimate of what the program is doing, even if it isn’t statistically significant. Better yet, RCTs also give you a lower and an upper bound on what that how much figure is. If your 99% confidence interval is 5 percentage points on either side, then you know with very high confidence that your program’s effect is no less than 15 percentage points (but no more than 25).*

I think a lot of implementers’ unease about RCTs would be mitigated if we focused more on the magnitudes of measured impacts instead of on significance stars. “We can’t rule out a zero effect” is uninformative, useless, and frankly a bit hostile – what we should be talking about is our best estiamte of a program’s effect, given the way it was implemented during the RCT. That alone won’t tell us why a program had less of an impact than we hoped, but it’s a whole lot better than just a thumbs down.

*Many of my stats professors would want to strangle me for putting it this way. 99% refers to the share of identically constructed confidence intervals that would contain the true effect of the program, if you ran your experiment repeatedly. This is different from there being a 99% chance of the effect being in a certain range: the effect is a fixed value, so it’s either in the interval or not. It’s the confidence intervals that vary randomly, not the true value being estimated. The uncertainty is in whether the confidence interval contains the true value of the effect, rather than in whether the true value of the effect lies in the range. If that sounds like pure semantics to you, well, you’re not alone.

Are GMOs per se unethical? I doubt it

I had an interesting conversation at a barbeque last weekend at which a lot of the attendees were in Michigan’s SNRE program (“snerds”, in the campus lingo), and we got into talking about GMO foods. Snerds mostly dislike GMOs, whereas I tend to think they’re a good thing. One thing I tried to do was get at the range of different factors that cause people to oppose them, because I think they are too often confounded. I was especially interested in separating the question of ethics from all the other things people worry about. Here’s what we came up with:

  1. Monsanto. They produce a lot of GMO foods and seeds. I don’t know a ton about their business practices but they sound like jerks and monopolists.
  2. Pesticide and herbicide use. Apparently you can use genetic engineering* to make crops that are more robust to these, and then they get used more, leading to overuse. I have heard the opposite claim as well – that GMO crops let us use less of these noxious chemicals.
  3. Unintended consequences. Who knows what could happen if these things get out into the wild?**
  4. Substitution away from other beneficial farming practices. If people use GMOs, they won’t move toward multicropping, which has ancillary benefits.
  5. Ethics. It is unethical to create organisms in the lab by combining genes from multiple species.

Ethics was the one point where the anti-GMO folks and I fundamentally disagreed. Not the ethics of what Monsanto does, which sound awful, but the basic ethics of modifying life. As I put it, I don’t see using fish genes to modify the genetic code of tomatoes as unethical in any basic sense, whereas one person I was chatting with absolutely did. In particular, she claimed that it was unethical to combine genes from different species.

Now, I’ve always been confused by a lot of what gets called ethics. For example, I once took a test about a (very unrealistic) scenario where you can either choose to kill one person or choose to let five die. The test varied the description, but I chose the same answer every time and apparently my answer – which seems like the only defensible one to me – is one that only 10% of people will ever pick. But my take is that there should be some general principle that underlies judgments of what is ethical and what is not. And I can’t see one driving the belief that adding genes from one species to another is unethical, for three reasons:

First, we already do tons of genetic modification of organisms, which very few people call unethical. If you want to create a pug, for example, the strategy is to: a) breed lots of small dogs; b) wait for mutants to show up with weird smushed faces that make it hard for them to breathe; and c) cross-breed those mutants to isolate that gene. We didn’t go get it from another species, but we waited for it to show up via mutation, which seems fundamentally identical. That might sound unethical (and maybe it is – most pugs I’ve met seem miserable) but if you replace weird faces with a herding instinct, you’ve got border collies.***

Second, we’re just talking about moving around chemicals in sequences of DNA. Most biochemistry isn’t inherently ethical or unethical – but specific acts, like reproducing smallpox, might be unethical, while producing a drug to suppress HIV infection might be very ethical.

Third, “species” is not a well-defined concept. Some taxonomists have put endless effort into defining where one species stops and another ends, but as Darwin pointed out in The Origin of Species****, there are no bright lines demarcating species. All living things exist on a gradient of relatedness, and there’s really no reasonable way to say when one species ends and another begins. My interlocutor said she was comfortable with the traditional definition of species: two groups of animals are of different species if, when they reproduce together, they produce infertile young. By this definition, however, grizzly bears and polar bears are the same species – which might leave neither eligible for endangered species protection.

One thing I want to separate here is the ethics of doing genetic modification of organisms from unethical acts during or resulting from the process. For example, cross-breeding dogs to develop new breeds is not unethical, but creating a breed with horrible congenital problems would be.

Now, it’s not impossible to defend the ethical claim that it’s wrong to modify one animal’s genome by using genes from another’s. But you’d have to come up with a definition of species you’re willing to stick to, and then you’d have to take the idea that this is unethical as a first principle: no mixing of kinds allowed. And that seems like an arbitrary rule, out of place in an ethical framework dedicated to preventing harm.

*What happened to calling this stuff “genetic engineering”, by the way? It sounds a lot more futuristic than “GMO”.
**There is one case in which we do know the answer, and that is concern over “terminator” genes that make later generations of an organism infertile. For simple reasons of natural selection, there is no risk of such genes becoming dominant in the gene pool. We can worry about a lot of stuff with GMOs, but all our crops ceasing to reproduce is not an issue.
***Some people claim that the herding instinct is actually a modified version of the hunting instinct, in which case the mutation is actually the part where they don’t kill certain prey.

****Technically On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life. It’s a remarkably readable and fully-developed book – and it covers lots of the subtle details of evolution, that I would have guessed were sorted out fairly recently.

How to stop eating away at your health capital when you're busy (or abroad)

Economists like to frame everything in terms of economics. One fairly hot topic these days is “health capital”, a stock into which we invest (by avoiding disease, eating well, exercising, and so forth) and which later pays a return (more success in school, higher wages, and what have you). Doug Almond’s fascinating paper about Ramadan is one demonstration of how big these returns can be: a small change in in utero nutrition can have massive effects on eventual physical and mental health, and impact eventual wealth as well. Health capital will also depreciate over time if you don’t maintain it: if you want to stay healthy you have to keep eating well and exercising.

When I go to the developing world to collect data, the cost of maintaining my health capital goes way up. It’s often difficult to find space or time to exercise, and depreciation (from pollution, water-borne illnesses, stress, excessive UV radiation, etc.) is so high that it often feels like I’m actively spending out of the principal of my endowment. Eight months collecting data in Malawi? That’ll be 5% of your overall health, please.

I’m sure I’m not unique in this problem: most people lose access to their preferred diets or exercise regimens for various reasons at some point, and being short of the time, space, or equipment needed for a workout is a common complaint. I’ve never had much in the way of a solution to this problem before, but this NYT piece points to a new workout that takes only seven minutes – and is summarized in a single nifty diagram.

A seven minute miracle workout is no surprise: I have various friends who swear by at least half a dozen different magic workouts between them. The unexpected thing here is that this one appears to be based on (some) actual science. The linked article is a review, but references previous research that not only suggests their approach may work, but actually appears to test it directly.*

I plan to try this out during my next trip to Africa – tentatively scheduled for mid-June.

*They overstate their case a bit, making claims that are supported by references to similar exercises. This just seems to be crying out for an experiment.

Should we keep providing foreign aid through governments?

The default process for providing foreign aid is to direct the money through country governments. I’ve long had my doubts about doing things this way: most of the problems that people like Bill Easterly and Dambisa Moyo attribute to aid really boil down to the fact that, when aid money is directed to governments, it becomes a fungible, capturable resource that is quite similar to natural resource proceeds.

Now it seems there are increasing challenges to that default from across the development world. In a great interview about cost effectiveness, Bill Gates highlights Somalia, which has no functioning government but fairly high vaccination coverage, and Nigeria, which has much less success with vaccines:

Well, in Somalia they’ve given up using the government. The money goes through the NGOs. Whereas in Nigeria they’ve designed a system where the federal government buys the vaccines, the state government provides the electricity, and the one level down below that provides the salaries. It’s just a bad design. You know, the north of India has very poor vaccination rates, so we picked a state up there with 80 million people and we drove it from 30 percent to 80 percent. But they had a really good chief health minister and the federal government was providing lots of money and lots of good technocrats, so the skills were there, as long as you employed them in the right kind of system.

Gates takes a very nuanced view: it’s not that funding vaccinations through governments can’t work, but the conditions need to be right. Ethiopia has done well, but Nigeria has not. He’s also not saying that the only reason that vaccination programs have failed in Northern Nigeria is because of the difficulties of running them through the Nigerian government – widespread urban legends about vaccines have played a major role – but the failings of the government are still an important factor.

In the latest edition of the Development Drums podcast (transcript here) Daron Acemoğlu and James Robinson discuss their book Why Nations Fail. I’m not broadly in agreement with their take on development, mainly because I don’t see their findings as actionable. But I agree strongly with the policy implication that Acemoğlu highlights, which is that providing aid through governments can be bad because it can support extractive institutions:

Daron Acemoğlu
But it’s a better formula than saying whoever is in power, we’re going to hand the power, the money to them.

Owen Barder
Is it implicit in what you are saying that some of the current aid modalities, particularly government to government aid, tend to reinforce the elites?

James Robinson
Absolutely, yes. I mean, I’d say our view was that you know, at the end of the day, that’s probably – if you asked where do all these development problems come from in Africa, are they created by the perverse incentives generated by the aid industry? Our answer to that would be no: they are much more deeply rooted in the history of these societies and you know, so sure you can find examples where aid kept in power, you know, Mobutu for another five years and he wouldn’t otherwise have been there but what did you get instead, you know?

We might not be able to do too much to promote beneficial institutions, but it’s pretty clear we can (and unfortunately do) support crappy ones, not by giving foreign aid, but by doing so through horrible governments. Don’t like what Mobutu is doing to the Congo? You don’t have to cut the people of the Congo off, just their government.

The obvious question is why we were giving aid through governments in the first place. There must be a reason, and any change to the process needs to consider the benefits of the current default as well as the costs. The basic argument I’ve heard, as Owen Barder put it during the Development Drums podcast, is “the thinking of providing aid through the governments is to try to build a stronger social contract between citizens and the state.” This claim is crying out for quantification: how much social contract strengthening actually occurs when aid goes through governments? Is that a valuable end in itself, and if so how much do people value it? What about eventual benefits of other kinds – do they happen? How much?

Given all the light being shed on the downsides of automatically sending aid through governments, it’s no longer enough to have qualitative evidence that sending aid through governments promotes their legitimacy. We need to know how much, and whether it’s worth the cost.

Hat tip: Amanda Stype for the Gates interview.

Choosing the facts you want to hear

Michael Shermer wrote a piece for Scientific American about the broad statistics on gun violence, and how they shaped his decision to give up his personal firearm. The top comment by Jim Pennington is indicative of an awful pattern in modern American political life: people opt out of news sources that report information that they disagree with.

In any case, since Scientific American (I’ve been a subscriber for over 10 years) has decided to get political, then I choose to do the same. Therefore I will NOT be renewing my subscription when it comes due nor will I waste my time reading the issues I still have coming. I also will mention to my acquaintenances that your magazine has become left wing publication just as U.S News and World Report did. You know what happened to them.

The way I see it, this is a pernicious attitude that is independent of whether you agree with Shermer’s opinion or even whether you think the statistics he cited tell the whole story. This isn’t a legitimate complaint about scientific accuracy. Pennington says “Why does this idiot Shermer think that anyone who hasn’t had formal training in firearms is any less competent to use one than someone who has?”, rhetorically suggesting that it is obvious that firearms training is worthless. I strongly doubt he actually believes that. Rather, this is an emotional response against Shermer’s opinion that has led Pennington to reject not only the data from the piece but anything that comes from it in the future.

I don’t think the desire to pick one’s information sources or facts is new, but the ability to pull it off is a fairly novel development. That means we’re seeing more polarization not just of attitudes and political tenets but actual beliefs about objective reality. This process undermines not only sound policy but science itself, which is bad news no matter what you think about gun control.

A simple observation about inflation and exchange rates in Malawi

Shortly after taking office last year, President Joyce Banda gave up on the Malawian Kwacha’s fixed peg against the dollar. Since then the exchange rate has evidently been some kind of managed float, meaning the rates are set in international markets. The unchallenged conventional wisdom is that this has drastically increased inflation in the country.* This piece in the Nyasa Times is a great example – it mentions, in passing, that “Soaring food and fuel prices have been stoking inflation since President Joyce Banda eased the kwacha’s peg against the dollar and devalued the currency by 49 percent.” Even people who support the reforms tend to concede that they have come at a large cost.

The issue is that there’s basically no way this claim is true. To oversimplify a good deal, Malawi is an agrarian society with a CPI basket (used to compute inflation rates) dominated by food. Imports are a pretty trivial share of food consumption, but food price inflation has been running at nearly the same rate as the overall rate. How can costlier imports (that is, a devalued Kwacha) be leading to price rises, when most of the rise in prices is non-imported food? The answer is they almost certainly cannot.

In case that didn’t convince you, here are some basic calculations. Malawi’s GDP was $14.265 billion in 2012, and about 30% of that was agriculture. That means the value of domestic agricultural production was $4.280 billion. Summing up all the food categories on indexMundi’s import breakdown for the country for 2011, I get $0.311 billion, meaning imports were 7% of all food; non-imported food was $3.968 billion. Food prices rose by ~30% year-on-year in February 2013 according to Malawi’s NSO. The exchange rate change is just a one-off rise in the price of imports. That was a drop of 49%, so we have 4.280*1.3=0.311*1.49 + 3.968*i, where i is the non-import (domestic) inflation rate. Solving for i gives us a blistering domestic inflation rate of 29% – imports are barely a drop in the ocean. What if we use the entire 142% decline in the exchange rate against the dollar since Banda took office? We still get a domestic inflation rate of 21%

Now, there are models in which the exchange rate might pass through to the broader economy and raise domestic inflation, but even a sophisticated approach would have to confront the basic fact that most Malawians are farmers who eat very little that is imported. And this is leaving aside the fact that when the currency peg was in place, a substantial share of imports were bought using black-market forex, and my skepticism that the basic inflation numbers are even correct. If we take the data at face value, it is pretty hard to justify the claim that the devaluation of the Kwacha is responsible for Malawi’s high inflation.

*I’ve written about the exchange rate peg and general misunderstandings of how foreign exchange markets work before; complaints about the Kwacha’s devaluation, far from being a Malawi-specific phenomenon, are representative of broadly-common misconceptions. I also have my doubts about the official inflation numbers (around 30% per year), which look way too high given my experiences buying staple goods in Malawi.

How to pronounce "Stata"

From the Statalist FAQ (emphasis mine):

4.1 What is the correct way to pronounce ‘Stata’?

Stata is an invented word. Some pronounce it with a long a as in day (Stay-ta); some pronounce it with a short a as in flat (Sta-ta); and some pronounce it with a long a as in ah (Stah-ta). The correct English pronunciation must remain a mystery, except that personnel of StataCorp use the first of these. Some other languages have stricter rules on pronunciation that will determine this issue for speakers of those languages. (Mata rhymes with Stata, naturally.)

This of course means that there is a right answer, but that StataCorp doesn’t want to take sides because they might piss people off. People are amazingly passionate about how they say the rarely-spoken names of software and other technical terms. If you want to start a fistfight, ask a group of nerds how to pronounce the name of the document markup language “LaTeX” or the image format “GIF”.

EDIT: Nick Cox, who wrote the Statalist FAQ, provided the following corrective in the comments:

Not so; or not really so. I wrote that originally, tongue in cheek, without any prompting whatsoever from the company and I don’t think it appears on any documents that don’t have my name on them, and I am not a StataCorp employee. Naturally, it remains true that the company would not knowingly host on any of their websites statements that they thought inappropriate. But the main interpretation is not that the company don’t want to take sides — there is only one pronunciation used at StataCorp — but that they have a sense of humour about something that isn’t really very important, namely how users and others pronounce the name. Now the correct spelling of Stata: that really is a big deal.

So the correct answer really is that there is no right answer. I am leaving my original post up for posterity, and because of my irrational passion in favor of the “Stay-ta” pronunciation.

An important negative result: teaching people about financial aid doesn't raise college attendance

As an economist and also somebody who loves facts, I never stop beating the drum of sticker prices vs. purchase prices in higher education. Long story short, the dizzying rise in sticker prices (the headline numbers decried by the news media) is mitigated and maybe even reversed when we look at the net price (after accounting for grants and scholarships).   Over the past 5 years or so, the former has risen sharply while the latter is basically flat across all schools and declining for private universities. People, even very smart people, almost uniformly ignore net prices when discussing what the rising cost of education means, and especially when talking about its effect on the poor. This is backwards: lots of programs target low-income students in particular

Since smart people with opinions on education policy don’t pay attention to net prices, it’s not surprising that most Americans aren’t aware of their financial aid options. This suggests an obvious policy change: we should inform people of how much financial aid they are eligible to receive. If we do so, the reasoning goes, more of them will go to college, especially on the low end of the scale. Yesterday I saw a talk by Eric Bettinger on the latest results from an experiment designed to test such a policy. Bettinger and coauthors Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu convinced with H&R Block to offer a group of their tax services clients either a) information about their financial aid eligibility or b) the same information, along with assistance in completing the FAFSA, which is required for almost all financial aid. At the baseline, the typical person overestimated the net cost of college attendance by a factor of 3.

Option b worked like gangbusters: recipients of the FAFSA assistance were 8 percentage points more likely to attend college, and the effect remains detectable well into their college years. Option a – just information about financial aid eligibility, did precisely nothing. And I do mean precise: Bettinger walked through some of the most impressive zeroes I’ve ever seen in a seminar. In general, Bettinger et al. can rule out effects much bigger than 2 percentage points (with -2 percentage points being about equally likely). During the seminar, Bettinger and Michigan’s own Sue Dynarski mentioned the fact that studies testing other ways of communicating this information find similar null effects.

There’s a lot to like about this paper. First, it’s testing a policy that seems obvious to anybody who’s looked at financial aid. If people are unaware of tons of money sitting on the table, some of them have to grab it when we point it out to them. Right? Wrong. Second, It reaches an important policy conclusion* and advances science based on a “statistically insignifcant” effect. Bettinger took the exact right approach to his estimated zero effects in the talk: he discussed testing them against other null hypotheses, not just zero. This isn’t done often enough. Zero is the default in most statistical packages, but it’s not really a sensible null hypothesis if we think an effect is likely to be zero. When we’re looking at possibly-zero effects, considering the top and bottom of the confidence interval – as Bettinger does – let’s us re-orient our thinking: given the data we’re looking at, what is the largest benefit the treatment could possibly bring? The biggest downside?

Null effects v2

Answering those questions shows us why this is a “good” zero: many statistically insignificant effects are driven by imprecision. They’re positive and pretty big, but the confidence intervals are really wide. The graph above, which I just made with some simulated data, illustrates the difference. On the left, we have a statistically insignificant, badly-measured effect. It could be anywhere from zero to two and a half. The right is a precise zero: the CI doesn’t let us rule out zero (indeed, the effect probably is about zero), but it does let us rule out any effects worth thinking about.

*Bettinger was careful to state that the information intervention alone cold still be cost-effective since it’s so damned cheap.

More on airline ticket pricing, now with (a tiny bit of) actual data

Joe Golden shared this Atlantic piece on the evolution of airfares, and the way airfare is priced now. It reaches the same conclusion I have: airline tickets are a very strange market, characterized by many constraints, limited competition between sellers, and, most important, nothing close to the classic “law of one price”. If a gas station tried to charge you twice as much as the next guy in line, you’d probably throw punches, but that kind of pricing is pretty typical on airplanes.

The article also has a nice graph that illustrates some of the random-looking price fluctuations that you see in airfares:

Screen Shot 2013-02-27 at 4.20.45 PM

The above chart is for a single route, departing on a single day. The time axis shows the date of a search for a fare on that route. Bearing in mind the human tendency to spot patterns in total noise, there appears to be a slowly-increasing regular fare of about $275, with a lot of large fluctuations from that level. Big downward spikes are definitely evident, and this might be masking their size (if what’s shown are daily averages, and the spikes are mixed with higher prices on a given day).

There are also upward spikes, consistent with anecdotes reported to me by a number of people. I’ve seen things like them myself. Michel, commenting on my last post, argues that these could be (false) signals sent by the airlines to indicate that flights are selling out. I agree that this is probably going on. The basic story is that you’re searching for a given flight, see a fare of $500, and decide to keep looking. Then you see a price of $1200. Crap! You start to freak out about the $700 you just lost. Maybe you open another browser or switch to incognito mode or delete your cookies. The next price you see is back down to $550 – thank god – and you buy it immediately to be safe. The airline has fooled you into taking their first offer, and even nudged you upward a bit. This is an interesting variation on price discrimination, one based on consumer psychology instead of income or demographics.

I differ with Michel’s conclusion that they are evidence for Valendr0s’s model (of exploiting cookies to track a given customer). An airline can try to scare potential customers by throwing in such upward spikes purely at random. It’s not clear that cookies give it any additional traction on this: if anything, a dedicated refresher like myself is signaling a lot of patience.

[Semi-technical aside: Michel also notes that my model has incomplete information on one side of the market. This is absolutely true, and in reality I think that incomplete information is the rule on both sides of the airline ticketing market. This drives all kinds of signaling by both sides. Sometimes I wonder about outright lies: when Kayak tells me there are only 2 seats left, are there any consequences if that’s not true?]

Random fluctuations around a fairly-high base price allow airlines to split the market into three segments, based on consumer preferences and psychology. The normal consumers will just take the base price or the low price as soon as they see it. Dedicated cheap types like myself will ride the refresh button until they get a low offer, while cautious cheap types will take the base price, or even a slightly higher one, if they see the high price offered first.  The story is getting richer and better-able to fit the facts, but we still need a data-scraping experiment (that randomly changes whether cookies are set) to test the different models on the table.