Getting the data to talk

by John Q on May 23, 2005

Among the many fascinating contributions in the latest contribution to the popular literature on economics is a chapter defending the (mildly surprising) conclusion that having a black-sounding name like DeShawn is not a disadvantage in the US, once you take account of the class, education and family backgrounds variables typically associated with such a name. Having named their own baby Freakonomics, economist Steven Levitt and journalist Stephen Dubner must be pretty confident on this point, and their high ranking on the New York Times bestseller list would appear to bear them out.

The subtitle of Freakonomics is A Rogue Economist Explores the Hidden Side of Everything and it’s a summary of some of the research contributions made by Leviit, the ‘rogue economist’ of the title. Indeed, the authors go to some pains to point out that Levitt is not really an economist at all, at least not in the ordinary sense of the term. He is quoted as telling his co-author, ‘I just don’t know very much about the field of economics. I’m not good at math. I don’t know a lot of econometrics, and I also don’t know how to do theory. If you ask me whether the stock market’s going to up or down, if you ask me whether the economy’s going to grow or shrink, if you ask me whether deflation’s good or bad, if you ask me about taxes – I mean, it would be total fakery if I said I knew anything about any of those things.’

It’s a surprising thing to say about someone who’s been awarded the John Bates Clark medal for the best American economist under the age of 40, but this does not appear to be mere false modesty. There’s very little economics in Freakonomics. Levitt is an incredibly smart social scientist, but most of his work could equally well have been done by someone trained in quantitative versions of sociology or poltical science.

Although he’s a product of Harvard and MIT and a full professor at Chicago, what Levitt has taken from the economics profession is not so much a body of theory to be applied, as a set of tools for empirical analysis and an unflinching willingness to look at social and policy issues without regard to social norms or received wisdom. More importantly, he’s combined all this with creative flair and an impressive capacity to see the right way of teasing compelling conclusions out of refractory data.

It’s not hard, for example, to see that, if teachers are rewarded and punished on the basis of scores from tests they administer themselves, that they’re likely to rig the results. What’s harder is to work out, as Levitt did, how cheating would manifest itself in answers to multiple choice tests, when the physical evidence that might have revealed cheating (the original test sheets) has long since been destroyed.

Similarly, it’s easy enough to see that real estate agents may be motivated to go for a quick sale and commission, rather than holding out for the best possible price, even though that would be in the interest of the sellers. but hard to see how you might test it. Levitt’s bright idea was to look at real estate agents selling their own homes. Sure enough, they waited longer and got more money. The disparity has diminished, but not disappeared, since the arrival of the Internet, and the sites like that allow anyone with the time and judgement to become experts on the state of the market.

In terms of unflinching willingness to follow the data where it leads, there can be few examples more striking than the paper by Levitt and Donohue arguing that the drop in crime rates experienced in the United States in the 1990s was due, in large measure to the legalisation of abortion two decades earlier which, Levitt and Donohue hypothesised, prevented the birth of unwanted children at high risk of becoming criminals.

This is a striking and unsettling finding, at at time when the debate over abortion law in the United States is hotter than ever. Naturally there have been vigorous critiques. The most prominent of these, by John Lott and John Whitley seems like a very weak reed in view of the subsequent discrediting of Lott’s major work, on guns and crime. As Levitt and Dubner note in the gun crime context, Lott’s supporters were embarrassed by the revelation that he posted favorable reviews of his own work, and attacks on his rivals, using a female pseudonym, Mary Rosh. (Lott himself was unabashed, as were his employers at the American Enterprise Institute). Less embarrassingly, but more seriously, Lott apparently invented a survey which he used to support his claims, and engaged in other dubious maniplulations. Given the ease with which statistical analyses can be selectively biased, no reliance can be placed on Lott’s results. However, there is still every possibility that subsequent work using different data and methods may support the conclusions of Lott and Whitley or at least fail to replicate that of Levitt and Donohue.

What’s notable, though, is that the central premise of the proposed explanation, that unwanted children are more likely to end up as criminals, comes from the realm of sociology. Economists have had little to say on this subject.

One point where Levitt sticks with the verities of the economics tribe is his repeated insistence that incentives matter. The empirical studies certainly give plenty of support to the view that human beings are purposive agents, pursuing their own ends and responding creatively to their environment, which includes deliberately constructed incentives and market price signals. The alternative, structuralist idea, that people are passive bearers of socially constructed roles, gets short shrift.

Yet, in nearly every case considered by Levitt, incentives turn out not to work the way that might have been expected by the people who designed them. Parents respond to a small charge for late pickups from a childcare centre by increasing the number of late pickups; apparently, the specification of a monetary price cancelled any feeling of moral obligation.

In other cases, incentives are too weak to produce the expected results. Real estate agents are paid for selling houses at the highest possible price, but their share of any increase in the sale price (the commission percentage) is too small to induce them to ‘go the extra mile’ for their customers. Sumo wrestlers are rewarded for wins, but nonlinear jumps in the reward schedule lead to matches being fixed when they are vital for one contestant but marginal for the other.

Most striking of all is Levitt’s analysis of the economics of drug dealing in Chicago housing estates, based on data collected, at significant personal risk by sociologist Sudhir Venkatesh. We begin with the irony that harsher sentences, designed to deter gang activity brought small-time gang members into contact with Mexican and Colombian cocaine importers, and thereby helped to fuel the crack epidemic of the early 1990s.

Of even more interest is the fact, documented in detail using the account books of a drug gang, that the average street-level dealer makes about $3.30 an hour, less than the minimum wage. Levitt plausibly explaines this as a tournament, in which the dealers accept low pay and high risk in return for a shot at the position of gang leader, earning more than $100 000/year, tax free.

It’s easy to see, however, that in a gang with more than 50 members, the expected payoff is not that great Suppose that the leader can expect perhaps five years in the job before falling to the hazards of prison, a rival gang, or an ambitious subordinate, and that the average gang member would have to work at least that long on the street corner before aspiring to the top job. Then the expected return from the leadership lotter is no more than $2000/year or around $1 an hour, only a marginal supplement to the miserable returns documented by Levitt.

There are plenty of plausible explanations for this. For example, explaining the fact that the chance of winning lotteries is routinely overweighted has been one of the major tasks for theorists of generalised expected utility models, and the same explanations would be applicable here.

The fact remains though, that, while people respond to incentives, they don’t respond the way the people who created the incentives might expect, or the way textbook economics might predict. Not that long ago, most economists would have said this was a problem for psychologists or sociologists and left it at that. Increasingly, however, the barriers between economics and the other social sciences are breaking down, with the rise of subfields like behavioural economics and economic sociology.

If we are going to make progress in understanding how human beings actually behave, rather than how idealised social science models say they should behave, skill in extracting meaningul patterns from refractory data will be crucial. Steve Levitt is showing us the way.



Steve Sailer 05.24.05 at 4:02 pm

Dr. Quiggins use of ad hominem reasoning to dismiss criticism of Dr. Levitt’s abortion-cut-crime theory is disappointing.

A quick Google search would show that this theory has more critics than just Dr. Lott, and that it disastrously fails basic tests of plausibility. For example, the first cohort born after the legalization of abortion committed about three times as many murders as 14-17 year olds as the last cohort born before legalization.

For instance, Dr. Ted Joyce, an economist with the National Bureau of Economic Research and Baruch College, City University of New York, has published an in-depth critique of the Levitt-Donohue theory.

And, in a not-yet-published paper, Joyce tries hard to remove the effects of crack crime from the data. Here’s part of his abstract:

“In this paper, I conduct a number of new analyses intended to address [Levitt and Donohue’s] criticisms of my earlier work.

“First, I examine closely the effects of changes in abortion rates between 1971 and 1974. Changes in abortion rates during this period were dramatic, varied widely by state, had a demonstrable effect on fertility, and were more plausibly exogenous than changes in the late 1970s and early 1980s. If abortion reduced crime, crime should have fallen sharply as these post-legalization cohorts reached their late teens and early 20s, the peak ages of criminal involvement.

“It did not.

“Second, I conduct separate estimates for whites and blacks because the effect of legalized abortion on crime should have been much larger for blacks than whites, since the effect of legalization of abortion on the fertility rates of blacks was much larger.

“There was little race difference in the reduction in crime.

“Finally, I compare changes in homicide rates before and after legalization of abortion, within states, by single year of age. The analysis of older adults is compelling because they were largely unaffected by the crack-cocaine epidemic, which was a potentially important confounding factor in earlier estimates.

“These analyses provide little evidence that legalized abortion reduced crime.”

And here is the abstract of the not-yet-published response paper by economists John R. Lott and John Whitley replying to Levitt and Donohue. They find that abortion increased the murder rate:

“Previous empirical work linking abortions and crime [i.e., Levitt and Donohue’s] has assumed, with the exception of five states, that no abortions took place prior to the Roe v. Wade decision in January 1973. In fact, abortion data from the Centers for Disease Control indicate that states which allowed abortions prior to the Roe v. Wade only when the life or health of the mother was in danger actually had higher abortion rates than some states where it was legal. The use of data from the Supplemental Homicide Report also allows the direct linkage between the current age of the murderer and the abortion rate when those murders were born.).

“One more abortion per 1,000 females age 15-44 (i.e., about four percent of the average) is associated with between a 0.12 to 0.9 percent increase in murders in any given year. Similar estimates are obtained using abortions per 1,000 live births. Linear estimates indicate increased annual victimization costs by at least $3.2 billion.”

One of the differences between Joyce’s approach and Lott-Whitley’s approach is that Joyce assumes the crack war of the late 1980s and early 1990s waged in sizable measure by urban teens born soon after the rise in urban abortion rates in the early 1970s was an “exogenous” event while Lott-Whitley are assuming the legalization of abortion and the subsequent growth in the murder rate during the crack wars might be related.

Both Joyce’s and Lott-Whitley’s approaches seem defensible ways to explore a hugely complex social phenomenon. What’s not reasonable is Dr. Levitt’s cherry-picking approach, in which he assumes that legalizing abortion isn’t responsible for any of the increase in murders at the beginning of the crack wars but is responsible for some of the decrease of murders at the end of the crack wars. That’s called having your cake and eating it too.

To learn more about the abortion-crime controversy, please see


Steve Sailer 05.24.05 at 8:43 pm

Dr. Quiggin rightfully states:

“If we are going to make progress in understanding how human beings actually behave, rather than how idealised social science models say they should behave, skill in extracting meaningful patterns from refractory data will be crucial.”

And that’s exactly why I’m disappointed that Dr. Levitt continues to push in his book and other venues a theory of how legal abortion would cut crime that Dr. Levitt himself knows is misleadingly simplistic.

For six years, Dr. Levitt has argued that abortion cuts crime by reducing the number of “unwanted” children who are born. Yet, the number of out-of-wedlock children soared, suggesting that wantedness by their fathers’ was definitely not increased. Indeed, legalizing abortion appears to have cut significantly the chance that a pregnant single woman can talk her boyfriend into marrying her.

Even worse for Levitt’s simplistic theory, he admits in “Freakonomics” that: “Conceptions rose by nearly 30 percent, but births actually fell by 6 percent …” In other words, legalizing abortions has led to tens of millions of unwanted pregnancies in America that wouldn’t have happened without legalization.

Levitt himself told Tim Harford, one of the contributors to this forum, that:

“One in four of the pregnancies which took place were just because people were lazy,” he says. “That’s a lot. That’s a lot of abortions.”

So, who ended up getting born out of all those unintended pregnancies. We really don’t know. The pro-choice Alan Guttmacher Institute reports that almost half of all pregnancies are unintended, and that out of every 100 unintended pregnancies, 13 end in miscarriages, 47 in abortions, and 40 in live births. So, Levitt’s tidy little model is radically unsatisfactory for understanding such a complicated and quasi-random process.

But he continues to push his model because it’s highly persuasive in getting people to believe his abortion-cut-crime theory, even though the historical evidence on abortion’s impact is thoroughly mixed.

One of my readers who was an inner city social worker before becoming a lawyer wrote to me:

A couple of further observations on abortion and crime:

First, it’s fascinating to see Conventional Wisdom taking shape right before my eyes. Usually the process is not nearly so obvious, and has to be pieced together after the fact.

Second, reading about Levitt’s theory that abortion cuts crime by culling unwanted babies reminds me of that old Elvis Presley song called “In the Ghetto.” It went:

As the snow flies
On a cold and gray Chicago morn
A poor little baby child is born
In the ghetto
And his mama cries
Cause if there’s one thing that she don’t need
It’s another hungry mouth to feed
In the ghetto

Remember that one? It’s the one where The King showed how sensitive and politically aware and stuff he was.

What the fans of Elvis and Levitt fail to get is that poor women don’t necessarily see their situation the same way middle class folks do. They may actually love their little bastard babies!

Middle class types see poor unwed teenage mothers as Scum of the Earth and a Terrible Social Problem. But poor women don’t see themselves that way. Instead, they think of themselves as human beings facing the age-old challenge of getting along in the world — and, if they’re lucky, passing their genes on to the next generation.

Unbelievable, I know. But bear with me for just a minute and try to see it from their point of view.

If you’re a young underclass woman, one of the first things you notice is that there are not many marriage-worthy men in your social milieu. A whole lot of them are unemployed or in prison or dead. [For example, in New York City, there are 36% more black women than black men.]

So even though you may want to get married, you figure your prospects are pretty dim. If you wait to marry before having children, you probably won’t have children.

You might as well have them now because, well, why wait? You’re not getting any younger. More to the point, your mother and other female relatives are not getting any younger. And since they’re the ones you’ll have to rely on for child care and support, it’s important to have your kids before they develop Type II diabetes and kidney failure and all the other health problems that tend to afflict black underclass folks more than white privileged types.

Will having kids hold back your career? Well, if you have an IQ of 80 and are looking for a reason to drop out of high school anyway, then no.

You’ve probably already figured out that your prospects of a good job are dim, and getting dimmer by the day, especially with immigrants flooding in by the millions to take the few jobs you’re qualified to do.

So for you, its not a choice of a ghastly life as a welfare mother or good life in the ‘burbs. Fate and the immigration mavens have already decreed that you will get mostly crumbs from America’s bounteous economic table. The only choice you have is between a crummy life with kids or a crummy life without kids.

Your lack of career prospects just makes having kids look that much more attractive. Children are about the only thing you can produce that people will view as being truly valuable.

Besides, if you can’t count on a spouse for love and companionship, kids become doubly important because they’ll be the only family you’ve got.

So becoming a single mother makes quite a bit of sense to you. You realize it’s a scary prospect and a hard life, but what are your options?

You may not exactly be looking to get pregnant, but when it happens — well, is it really all bad? Lots of others have done it before you. In fact, in your neighborhood, girls who have babies out of wedlock are becoming the norm.

The only people who can’t seem to grasp what is going on here are the Really Smart Guys. Even though it should be getting pretty obvious by now, especially since the black illegitimacy rate is close to 70 percent. Admittedly, most of these out of wedlock pregnancies may not have been “planned” or “intended” in any sense that a middle class observer could understand. But that doesn’t mean they’re necessarily “unwanted.”

Seen from this perspective, poor women who have abortions are likely to be the strivers and achievers. They’re the ones who see some prospect of improving their lives, and realize it may hold them back if they have five kids by four fathers. They’re the ones who are trying, in their own way, to make good.

Inability to grasp what is wrong with Levitt’s argument seems to be a case of “I’ll see it when I believe it.” Maybe all the bright guys who can’t believe what’s going on in the underclass world should ditch Elvis and listen to Fantasia Barrino [last year’s American Idol and an unwed mother herself] sing:

Nowadays it’s like a badge of honor
To be a baby mama…
Cause we the backbone of the ‘hood.


John Quiggin 05.25.05 at 12:56 am

Steve, reread the post. The point isn’t that there aren’t any other critiques, or that Levitt & Donahue have been proven right. In fact, I conclude “there is still every possibility that subsequent work using different data and methods may support the conclusions of Lott and Whitley or at least fail to replicate that of Levitt and Donohue.”

The point is that any research reported by Lott is worthless and only serves to muddy the water for future researchers.

If you’re not aware of the chapter and verse on this, go here.


Tim Lambert 05.25.05 at 1:10 am

Steve describes the Lott-Whitley paper as “not yet published”. However, it’s over four years since it was posted to SSRN, so a better description is likely to be “rejected by peer review”.

Lott and Whitley’s paper assumes that the crack cocaine epidimic was caused by legalized abortion. Some may not find this assumption plausible.


Steve Sailer 05.25.05 at 2:22 am

Sorry, Dr. Lambert, but you are confused. I quoted above the abstracts of new, not-yet-published papers from both Joyce and Lott-Whitley. They are both responses to Levitt-Donohue’s recent (2003?) response to their criticisms of Levitt-Donohue’s 2001 paper.

If you are interested in an objective party overview of the state of the debate after Levitt-Donohue’s 2003 publication (but before these two new papers), here is Jonathan Klick’s paper: “Econometric Analyses of U.S. Abortion Policy: A Critical Review”

Keep in mind that Klick’s paper was written with Levitt-Donohue serving last, so the fact that he didn’t find them to be winning raises severe doubts about how they will be seen as doing after Joyce’s and Lott-Whitley’s second responses are published.

No study that I am aware of has yet replicated Levitt-Donohue’s findings for American history. Levitt’s findings are highly dependent upon dubious assumptions to avoid having to face the fact that his theory fails disastrously basic tests of historical plausibility. Big assertions need big proof, and Levitt has so far failed to meet this burden of proof.

As I explained above, Joyce’s latest study is based on the assumption that the crack wave was a purely exogenous event. He tries to carefully remove the influence of the rise and fall of crack killings. When he does, he finds legalization of abortion had no impact on the murder rate.

Lott-Whitley, in contrast, assume that the murder rate during the crack crime wave that began as the first generation born after the legalization of abortion reached their crime-committing years may have been influenced by legalizing abortion. Under that assumption, they found that abortion drove up the murder rate.

Either assumption is questionable, but defensible.

What is not logically defensible is Levitt-Donohue’s assumption that legalizing abortion could only have driven _down_ the murder rate, and couldn’t possibly have driven it up. That’s having your cake and eating it too.

You’ll notice that in “Freakonomics,” Levitt only wants to talk about the decline in crime in mid-1990s in relation to abortion, even though the effects of legalizing abortion during the 1970-1973 period on teenage crime would begin showing up by the mid-1980s. He never wants to talk, in his chapter on abortion, about the huge rise in serious violent youth (age 17 and under) crime that preceded the decline, and that struck first in the metropolises and among the demographic group that first partook heavily of legalized abortion. In the weird world of Freakonomics, the _longer_ the lag between a putative cause (legalization of abortion) and its putative effect (a change in the crime rate), the more trustworthy is the theory of cause and effect.

The truth is that when Levitt and Donohue first wrote up a draft paper in 1999 announcing this theory, they failed to look at crime statistics for youths (for example, according to the FBI the worst year, both in absolute and relative terms, for “serious violent crime” by youths ages 12-17 was 1993), so they were totally unaware of the implausibility of their theory until our debate in that summer. ( )But, by that point, their draft paper had been leaked to the Chicago Tribune and Levitt’s name was unalterably linked to his theory. He has defended it like a bulldog ever since, but he hasn’t come much closer to meeting a reasonable burden of proof.


Tim Lambert 05.25.05 at 5:18 am

The only Lott-Whitley paper on abortion at SSRN dates from 2001. If there is another one, he hasn’t made it publically available yet.

And yes, John’s argument against Lott is an ad hominem argument, but it’s not an ad hominem fallacy. An ad hominem fallacy is when you offer some unflattering but irrelevant detail about a person in order to discredit them. But that’s not what is going on here. There are so many different specifications you can choose in econometrics that it is very easy to cheat and just show the ones that support your position. So any econometric paper requires to trust the author, and you can’t trust Lott.

Also, you seem to be utterly certain that Levitt is wrong, but you cite Klick as an objective commenter and he doesn’t agree with you since he reckons the jury is still out.


Steve Sailer 05.25.05 at 12:35 pm

“Also, you seem to be utterly certain that Levitt is wrong, but you cite Klick as an objective commenter and he doesn’t agree with you since he reckons the jury is still out.”

My position is that large assertions need large proof and Levitt hasn’t come anywhere close to meeting the burden of proof he has taken on. It appears to me that there is about as much historical evidence that legalizing abortion drove up serious violent crime and homicide as that it drove it down. And, I think, his theory that abortion would drive down crime is, while appealing on its face, is less persuasive the more you learn about the actual behavior of underclass women.

So, I would say that at present it’s about a wash. I’m skeptical that contemporary social science can actually answer this question with the data currently available, but I think that openly debating this question is an excellent learning tool. I’ve certainly learned far more about crime, abortion, and other important social issues in the half dozen years I’ve been thinking about this.

Regarding the new, not-yet-published papers by Joyce and Lott-Whitley, both Joyce and Lott emailed them to me and gave me permission to post on their abstracts. I would suggest asking them for copies.

Regarding Klick’s third party evaluation that the jury is still out, that came after, to use American baseball terminology, Levitt-Donohue had two at-bats (i.e., had issued their original paper and their response to criticism), while Joyce and Lott-Whitley had had only one (i.e., their latest papers are not considered by Klick). That, when Klick wrote, Levitt-Donohue had not pulled into a clear lead even though they had enjoyed two at-bats and their critics had only one suggests that the prospects of Levitt-Donohue ever meeting the burden of proof are not good.

But perhaps in Levitt-Donohue’s prospective third at-bat they will produce new, more persuasive evidence. For example, the American abortion rate fell in the 1990s, especially among non-Hispanic whites (from 19 in 1991 per 1,000 white women in their child-bearing years to only 11). This suggests, following Dr. Levitt’s logic, that the crime rate among 14-17 year old boys, especially among white boys, should start going up in the next few years. For some reason, though, I haven’t heard Dr. Levitt warning that we should start building new juvenile detention facilities to accomodate the coming crime wave from a generation who didn’t benefit as much from pre-natal culling as had previous cohorts.

As I’ve mentioned, you can learn a lot more about this important and informative subject at


shah8 05.26.05 at 2:26 pm

steve, your assertions are too long, and not concise in a way that really helps. So much of this is depending on what data set you look at. Murders or armed robbery, or maybe it’s tax evasion…It’s hard to read that stuff when you don’t give a strong model to contain the rebuttal you are giving…


Steve Sailer 05.26.05 at 11:18 pm

Quick summary:

Empirically, there is not much more historical evidence that legalizing abortion drove homicide or the broader category of “serious violent crime (including rape, armed robbery, etc.) down than that it drove it up. (The crack crime wave began, geographically and demographically, among those who were the heaviest users of legal abortion in the 1970s.)

Theoretically, Levitt’s initially appealing model that legalizing abortion reduces unwanted births is heavily undermined by his own admission that the biggest effect of legalization was to increase unwanted pregnancies: as he says in Freakonomics, the birthrate fell by only 6% while the conception rate went up by almost 30%.

In summary, over the last six years, the Levitt-Donohue assertion theory has proven enormously appealing at first glance to academics and writers (as shown by all the softball reviews “Freakonomics” has gotten in prestigious publications by prominent reviewers who do zero independent research on his abortion-crime theory and just summarize Levitt’s arguments). On the other hand, those who have dug deeper into the data and thought harder about Levitt’s model of human behavior have tended to become increasingly skeptical.

Large assertions require large evidence, and Levitt hasn’t come close to meeting the burden of proof.


meika 05.28.05 at 8:06 am

so, incentives then, next buzz word, mmmh, questions, questions, if incentives includes a mix of, well, incentives, called x, y, z etc, how are they mixed, at the individual level, and the the gang level and the socious/economy itself, how does each level (I guess they are fractal) influence the mix in the other levels (top-down, down-top, middle-out),

Also, can an agent choose their own mix, can we re-mix ourselves, or are we not free to choose?

for examples, does studying economics turn one into an arseholes, or are sociopaths attracted to it to excuse their inner arsehole?

who needs hard data, who? its obvious?

I actively rejected Mr Economy as a youth, while the econommists can point at me now as a free loading parasite, I no longer have any choice in the matter.

So, then, how did I ever think I could choose to make the world bettter by refusing Mr Economy?

Does it mean those who choose to choose and refuse, are irrelevant, that the only way to move is to choose an iPod like a consuming robot, and download the fileshared songs as some sort of compromise??

Who are the fools who choose virtue, and how did they get to be so dumb?

What was the incentive??

Comments on this entry are closed.