Not as smart as I thought I was

by Daniel on October 2, 2003

My education is clearly sadly lacking

Meanwhile, as a break from the hysterical, obsessive and politicised world of weblog disputes, I decided to have another look at an uncontroversial, scientific topic like John Lott’s research into gun control. And I discovered that I have been quite appalingly conned by two institutions that I thought I could trust. Instapundit has printed a letter from someone called Benjamin Zycher, a “Senior Economist”[1] at the Rand Corporation, supported by Raymond Sauer, a professor at Clemson University. Zycher says, and Sauer supports him in saying that the Ayres and Donohue paper on Lott’s work is all wet.

Specifically, he says that:

To put it bluntly: Any undergraduate student receiving a B or better in introductory Econometrics would be able to pick the Ayres/ Donohue work apart. This is for a number of reasons, the most fundamental of which is—and this is an error more appropriate for freshman Statistics 1—that their own interpretation of their estimated coefficients is simply wrong. They discuss two variables purporting to measure the effect of concealed-carry laws, but then fail to understand that it is the joint effect of the two variables, rather than merely one of them, that represents the estimated effect in the model.

This is worrying to me for the following reason. I’ve taken not only an undergraduate course in Econometrics (for which I did indeed receive a B), but also a postgraduate course in Econometrics (for which I got a Distinction, yes, thank you, we’re all very proud). I got ’em at two of the country’s most prestigious institutes of learning; the undergraduate one was paid for by my parents and the UK taxpayer, but the postgraduate one cost me a pretty significant chunk of my own money (£18,000 to be precise). And I don’t understand what the hell Zycher’s going on about. Have I been made the victim of grade inflation and fobbed off with a couple of noddy preparatory courses that wouldn’t get me a passing grade at Clemons University?

In the shoddy, downscale version of econometrics taught at British universities, the multivariate regression model exists to separate out the effect of different variables. The estimated coefficients and associated standard errors measure the partial effect of each variable taken separately. Least squares estimation (which is the technique Lott used) doesn’t have the characteristic of delivering coefficient estimates that have to be taken two at a time.

Furthermore, I’ve read the A ‘n’ D paper, and I don’t understand what is meant by saying that “they discuss two variables puporting to measure the effect of concealed carry laws”. Throughout the paper, they discuss the effect of the “Concealed Carry Dummy” and whether it’s significant or not. That’s one variable, not two. Ayres & Donohue discuss two types of models (trend models and level models), but that’s not the same as “two variables” in a model, and I don’t see how the two types of model taken together might show significant effects if neither does on its own.

Mr Zycher, I think, overestimates his undergraduate students. He also overestimates his readers when he later writes: “There is no need here to delve into a mini-course on econometrics, however lacking for sleep your readers may be.”, because I’m both a chronic insomniac and apparently in dire need of a mini-course in econometrics. And he overestimates Lott’s co-authors when he goes on to say “Anyone interested simply can read the paper by Plassmann and Whitley, utterly devastating in its critiques of the Ayres/Donohue paper”, because the Plassman & Whitley paper doesn’t include anything resembling the critique that he’s made himself. (It does have one rather bizarre argument that you should ignore large discontinuous jumps in Lott’s model because they aren’t so big if you smooth them out with the following year’s trend, but that’s not really the same thing).

I was never a good study in econometrics – I found it dull and difficult – but I don’t think I was ineducable. If someone read the Instapundit letter and understood it, I’d be very grateful for help. Otherwise, Prof Sauer may wrong wrong in stating that “And he [Zycher – dd] knows that his argument prior to making that statement, if incorrect, will torpedoed in an instant by one of Lott’s critics who is also skilled at econometrics”. Speaking as one of them (I hope), it’s unlikely to be torpedoed by me any time soon because I don’t understand what it means.

[UPDATE]: I had hoped that the Ultimate Lott Trainspotter would be on this one and he is. As Tim says in comments, it’s likely that Zycher was referring to the “hybrid model” that A&D used in their paper. I personally think that the hybrid model is a terrible way to try to achieve anything in time series analysis, but the fact is that A&D only introduced the bloody thing for the purpose of addressing exactly the issue that Zycher might be raising. Anyway, read Tim’s post, it’s better than mine if you care about this sort of thing.

[1] “Senior Economist” put in scare-quotes because the meanings of titles vary from institution to institution and I don’t know exactly how senior an economist one would have to be to be titled thus at Rand. [2]
[2] I’m using this style for footnotes rather than superscripts because somebody told me that superscripts don’t fit into the CT template too well.



Tim Lambert 10.02.03 at 3:27 pm

I think I figured out what Zycher is on about. A&D’s most general model combines the dummy and trend models into a hybrid model which has a variable for the trend effect and one for the dummy effect.

However, Zycher is not just wrong, but wrongity wrong when he claims that they didn’t consider the joint effect, when they have about ten pages on it. Details are here.


nofundy 10.02.03 at 3:41 pm

As a fellow member of the fraternity of those whose education is clearly sadly lacking (great line BTW), I need a little clarification Daniel.
Are we intimating that InstaPundit, John Lott and some Rand guys may be presenting misleading information for consumption while posing as ‘experts?’
Say it ain’t so! (Oops, there I go exhibiting my lack of education again.)
Shall we inform Mary Rosh of your brazen and groundless charges so she might defend the honor of these obviously superior intellects?


dsquared 10.02.03 at 3:45 pm

Not actually accusing anyone of quite such a serious crime. For reference, here’s my charge sheet:

Zycher: Writing incomprehensible babble, plus one misdemeanor count of cliche for the “undergraduate” reference.
Sauer: One count of being a me-too.
Reynolds: Posting impressive-looking gobbledigook without understanding what it means.

I think that Tim would add “being full of it” to Zycher’s charges, and I am currently pondering adding “trying to baffle people with science in order to make an issue look less cut and dried than it is” to Reynolds corpus delicti.



Michael 10.02.03 at 3:47 pm

To be fair to InstaPundit, nofundy, he didn’t buy Zycher’s hack piece, either.


dsquared 10.02.03 at 3:51 pm

“To be fair to InstaPundit”

WHo do you think I am? Kevin bloody Drum?


Zizka 10.02.03 at 3:56 pm

What nofundy said.

If I had seen the article you linked in a different context, I would probably have moved Lott/Rosh’s study off the “Debunked” shelf onto the “Undetermined” shelf. I certainly wouldn’t have begun the study of econometrics.

What the Rand guys did is technically known as “making shit up”, and it’s a prime right-wing debating technique these days. (Left-wingers use it too, but I’m a Stalinist, so I’m cool with that). Other valuable techniques include “throwing shit at the wall” and the “George Chuvalo” tactic: you remain standing while the other guy hits you in the face over and over again.

In the blogosphere there’s no referee, no scorekeeper, and you can’t physically hit the sons of bitches, so bad theories and untrue facts are immortal. People are still talking about Saddam’s mobile biological warfare vans, for example, months after they were shown to be used to produce weather ballons.


Michael 10.02.03 at 4:01 pm

“WHo do you think I am? Kevin bloody Drum?”

Heh. Indeed.


Mary Frosh 10.02.03 at 4:04 pm

Yeah, I was just reading this at Reynold’s site, and I thought he was quite unfriendly to Zycher. Okay, “unfriendly” may not be the correct word. But I hardly think one can reasonably conclude that Reynolds is supporting Zycher.


Michael 10.02.03 at 4:49 pm

I think my favourite bit was this:

“Does Lindgren believe that Lott would invent an entire survey for the purpose of adding a sentence or two to a book of well over 200 pages?”

Er… Does Zycher believe that Lott conducted the survey — at personal expense, no less — for the purpose of adding a sentence or two to a book of well over 200 pages? Seems like an awful lot of effort for a throwaway line. And then, strangely, Lott forgot about the survey and attributed the 98% to “national surveys,” remembering his survey only after the number was called into question.

Yeah, I think we’re perfectly justified in believing that Lott would invent an entire survey for this.


Alan 10.02.03 at 5:07 pm

months after they were shown to be used to produce weather ballons

Yes, but they were evil weather balloons, full of malice for their neighbors.


Doug Turnbull 10.02.03 at 5:13 pm

I’m pretty sure Hurican Isabel was linked to Iraqi weather balloons.


Keith M Ellis 10.02.03 at 5:33 pm

“They were evil weather balloons, full of malice for their neighbors.”

That’s a fine first line for a novel. Or something.


dsquared 10.02.03 at 5:55 pm

I’d also suggest that Reynolds “I’m not sure I agree with this” disclaimer struck me as a bit “disinglennuous“. I operate on a basis of strict liability for this sort of thing; you print it, you own it.


dipnut 10.02.03 at 8:32 pm

You print it, you own it.

Well, you printed:

To put it bluntly: Any undergraduate student receiving a B or better in introductory Econometrics would be able to pick the Ayres/Donohue work apart.

Do you own that? Now let’s look at what Disinglennuous Reynolds said:

I don’t find this very persuasive, I’m afraid, since it consists mostly of assertions that people are idiots, but without much actual exposition.

I find Jim Lindgren quite credible, and I don’t think that assertions that he is biased are very persuasive. Assertions that he doesn’t understand elementary statistics would be more persuasive if accompanied by explanations.

I find Lindgren credible; Ayres and Donohue, too, though they’re anti-gun.

Reynolds admits his bias:

I would, of course, prefer to have it turn out that Lott is correct and that his critics are mistaken.

Finally, Reynolds gives Lindgren about ten-thousand words of rebuttal inline, links to Tim Lambert, and so on: hardly the actions of someone who wants to make “untrue facts…immortal”. Here’s a challenge for you: find a pro-gun blogger citing Zycher’s little flame-fest as a “plus” argument.

I won’t hold my breath.


FDL 10.03.03 at 1:59 am

“Kevin bloody drum”?

No. I think you’re a “fat young man without a good word for anybody” (or something along those lines.)

And on the weighty topic of weight, was is it about you economists? you admit to being “fat” and Brad’s picture on his website shows someone who appears to be able to afford to lose a few pounds. Try Atkins, or something. You and he are far too entertaining to lose to diabetes, stroke etc.



Zizka 10.03.03 at 5:31 pm

Dipnut: Whether or not Reynolds is culpable, the authors and publishers of the article are. I try to avoid 2nd Amendment discussions, but I’m not always successful, and I expect to see Zycher’s piece again.


dipnut 10.03.03 at 8:21 pm


Reynolds is the only “publisher” of Zycher’s diatribe, as far as I know. And he strangled it in the cradle, despite being sympathetic to the author on the issues. Zycher simply didn’t measure up, analytically or rhetorically.

You won’t be seeing it again, except maybe in some penny-ante high-school blog. And this particular 2nd-Amendment zealot is just as happy.

Lott has not weathered the attacks on him gracefully. He may well be a fraud and/or hack. As a result of the cloud hanging over him, Lott’s “facts” are anything but immortal. He is hardly ever cited by gun-rights proponents anymore, and his scholarship is not missed. We simply don’t need him.

On the other hand, there are any number of “untrue facts”, far easier to debunk than any of Lott’s work, which have achieved immortal status on the anti-gun side of the debate.


Jason McCullough 10.05.03 at 2:31 am

On a completely unrelated note, here’s some stuff I happened on Daniel will probably like:

Is America the republic or the galactic empire?.’s libertarian analysis of Star Wars.

The Von Mises institute has a lot more search results for “Star Wars” than I’d expect. The best is The Political Economy of Star Wars.


dsquared 10.06.03 at 12:43 am

Ahhhh I remember those. I still think nothing beats the VMI analysis of WWF wrestling as deeply anti-capitalist propaganda.

Comments on this entry are closed.