Not as smart as I thought I was

My education is clearly sadly lacking

Meanwhile, as a break from the hysterical, obsessive and politicised world of weblog disputes, I decided to have another look at an uncontroversial, scientific topic like John Lott’s research into gun control. And I discovered that I have been quite appalingly conned by two institutions that I thought I could trust. Instapundit has printed a letter from someone called Benjamin Zycher, a “Senior Economist”[1] at the Rand Corporation, supported by Raymond Sauer, a professor at Clemson University. Zycher says, and Sauer supports him in saying that the Ayres and Donohue paper on Lott’s work is all wet.

Specifically, he says that:

To put it bluntly: Any undergraduate student receiving a B or better in introductory Econometrics would be able to pick the Ayres/ Donohue work apart. This is for a number of reasons, the most fundamental of which is—and this is an error more appropriate for freshman Statistics 1—that their own interpretation of their estimated coefficients is simply wrong. They discuss two variables purporting to measure the effect of concealed-carry laws, but then fail to understand that it is the joint effect of the two variables, rather than merely one of them, that represents the estimated effect in the model.

This is worrying to me for the following reason. I’ve taken not only an undergraduate course in Econometrics (for which I did indeed receive a B), but also a postgraduate course in Econometrics (for which I got a Distinction, yes, thank you, we’re all very proud). I got ’em at two of the country’s most prestigious institutes of learning; the undergraduate one was paid for by my parents and the UK taxpayer, but the postgraduate one cost me a pretty significant chunk of my own money (£18,000 to be precise). And I don’t understand what the hell Zycher’s going on about. Have I been made the victim of grade inflation and fobbed off with a couple of noddy preparatory courses that wouldn’t get me a passing grade at Clemons University?

In the shoddy, downscale version of econometrics taught at British universities, the multivariate regression model exists to separate out the effect of different variables. The estimated coefficients and associated standard errors measure the partial effect of each variable taken separately. Least squares estimation (which is the technique Lott used) doesn’t have the characteristic of delivering coefficient estimates that have to be taken two at a time.

Furthermore, I’ve read the A ‘n’ D paper, and I don’t understand what is meant by saying that “they discuss two variables puporting to measure the effect of concealed carry laws”. Throughout the paper, they discuss the effect of the “Concealed Carry Dummy” and whether it’s significant or not. That’s one variable, not two. Ayres & Donohue discuss two types of models (trend models and level models), but that’s not the same as “two variables” in a model, and I don’t see how the two types of model taken together might show significant effects if neither does on its own.

Mr Zycher, I think, overestimates his undergraduate students. He also overestimates his readers when he later writes: “There is no need here to delve into a mini-course on econometrics, however lacking for sleep your readers may be.”, because I’m both a chronic insomniac and apparently in dire need of a mini-course in econometrics. And he overestimates Lott’s co-authors when he goes on to say “Anyone interested simply can read the paper by Plassmann and Whitley, utterly devastating in its critiques of the Ayres/Donohue paper”, because the Plassman & Whitley paper doesn’t include anything resembling the critique that he’s made himself. (It does have one rather bizarre argument that you should ignore large discontinuous jumps in Lott’s model because they aren’t so big if you smooth them out with the following year’s trend, but that’s not really the same thing).

I was never a good study in econometrics – I found it dull and difficult – but I don’t think I was ineducable. If someone read the Instapundit letter and understood it, I’d be very grateful for help. Otherwise, Prof Sauer may wrong wrong in stating that “And he [Zycher – dd] knows that his argument prior to making that statement, if incorrect, will torpedoed in an instant by one of Lott’s critics who is also skilled at econometrics”. Speaking as one of them (I hope), it’s unlikely to be torpedoed by me any time soon because I don’t understand what it means.

[UPDATE]: I had hoped that the Ultimate Lott Trainspotter would be on this one and he is. As Tim says in comments, it’s likely that Zycher was referring to the “hybrid model” that A&D used in their paper. I personally think that the hybrid model is a terrible way to try to achieve anything in time series analysis, but the fact is that A&D only introduced the bloody thing for the purpose of addressing exactly the issue that Zycher might be raising. Anyway, read Tim’s post, it’s better than mine if you care about this sort of thing.

[1] “Senior Economist” put in scare-quotes because the meanings of titles vary from institution to institution and I don’t know exactly how senior an economist one would have to be to be titled thus at Rand. [2]
[2] I’m using this style for footnotes rather than superscripts because somebody told me that superscripts don’t fit into the CT template too well.