My education is clearly sadly lacking
Meanwhile, as a break from the hysterical, obsessive and politicised world of weblog disputes, I decided to have another look at an uncontroversial, scientific topic like John Lott’s research into gun control. And I discovered that I have been quite appalingly conned by two institutions that I thought I could trust. Instapundit has printed a letter from someone called Benjamin Zycher, a “Senior Economist”[1] at the Rand Corporation, supported by Raymond Sauer, a professor at Clemson University. Zycher says, and Sauer supports him in saying that the Ayres and Donohue paper on Lott’s work is all wet.
Specifically, he says that:
To put it bluntly: Any undergraduate student receiving a B or better in introductory Econometrics would be able to pick the Ayres/ Donohue work apart. This is for a number of reasons, the most fundamental of which is—-and this is an error more appropriate for freshman Statistics 1—-that their own interpretation of their estimated coefficients is simply wrong. They discuss two variables purporting to measure the effect of concealed-carry laws, but then fail to understand that it is the joint effect of the two variables, rather than merely one of them, that represents the estimated effect in the model.
This is worrying to me for the following reason. I’ve taken not only an undergraduate course in Econometrics (for which I did indeed receive a B), but also a postgraduate course in Econometrics (for which I got a Distinction, yes, thank you, we’re all very proud). I got ‘em at two of the country’s most prestigious institutes of learning; the undergraduate one was paid for by my parents and the UK taxpayer, but the postgraduate one cost me a pretty significant chunk of my own money (£18,000 to be precise). And I don’t understand what the hell Zycher’s going on about. Have I been made the victim of grade inflation and fobbed off with a couple of noddy preparatory courses that wouldn’t get me a passing grade at Clemons University?
In the shoddy, downscale version of econometrics taught at British universities, the multivariate regression model exists to separate out the effect of different variables. The estimated coefficients and associated standard errors measure the partial effect of each variable taken separately. Least squares estimation (which is the technique Lott used) doesn’t have the characteristic of delivering coefficient estimates that have to be taken two at a time.
Furthermore, I’ve read the A ‘n’ D paper, and I don’t understand what is meant by saying that “they discuss two variables puporting to measure the effect of concealed carry laws”. Throughout the paper, they discuss the effect of the “Concealed Carry Dummy” and whether it’s significant or not. That’s one variable, not two. Ayres & Donohue discuss two types of models (trend models and level models), but that’s not the same as “two variables” in a model, and I don’t see how the two types of model taken together might show significant effects if neither does on its own.
Mr Zycher, I think, overestimates his undergraduate students. He also overestimates his readers when he later writes: “There is no need here to delve into a mini-course on econometrics, however lacking for sleep your readers may be.”, because I’m both a chronic insomniac and apparently in dire need of a mini-course in econometrics. And he overestimates Lott’s co-authors when he goes on to say “Anyone interested simply can read the paper by Plassmann and Whitley, utterly devastating in its critiques of the Ayres/Donohue paper”, because the Plassman & Whitley paper doesn’t include anything resembling the critique that he’s made himself. (It does have one rather bizarre argument that you should ignore large discontinuous jumps in Lott’s model because they aren’t so big if you smooth them out with the following year’s trend, but that’s not really the same thing).
I was never a good study in econometrics - I found it dull and difficult - but I don’t think I was ineducable. If someone read the Instapundit letter and understood it, I’d be very grateful for help. Otherwise, Prof Sauer may wrong wrong in stating that “And he [Zycher - dd] knows that his argument prior to making that statement, if incorrect, will torpedoed in an instant by one of Lott’s critics who is also skilled at econometrics”. Speaking as one of them (I hope), it’s unlikely to be torpedoed by me any time soon because I don’t understand what it means.
[UPDATE]: I had hoped that the Ultimate Lott Trainspotter would be on this one and he is. As Tim says in comments, it’s likely that Zycher was referring to the “hybrid model” that A&D used in their paper. I personally think that the hybrid model is a terrible way to try to achieve anything in time series analysis, but the fact is that A&D only introduced the bloody thing for the purpose of addressing exactly the issue that Zycher might be raising. Anyway, read Tim’s post, it’s better than mine if you care about this sort of thing.
[1] “Senior Economist” put in scare-quotes because the meanings of titles vary from institution to institution and I don’t know exactly how senior an economist one would have to be to be titled thus at Rand. [2]
[2] I’m using this style for footnotes rather than superscripts because somebody told me that superscripts don’t fit into the CT template too well.
I think I figured out what Zycher is on about. A&D’s most general model combines the dummy and trend models into a hybrid model which has a variable for the trend effect and one for the dummy effect.
However, Zycher is not just wrong, but wrongity wrong when he claims that they didn’t consider the joint effect, when they have about ten pages on it. Details are here.
As a fellow member of the fraternity of those whose education is clearly sadly lacking (great line BTW), I need a little clarification Daniel.
Are we intimating that InstaPundit, John Lott and some Rand guys may be presenting misleading information for consumption while posing as ‘experts?’
Say it ain’t so! (Oops, there I go exhibiting my lack of education again.)
Shall we inform Mary Rosh of your brazen and groundless charges so she might defend the honor of these obviously superior intellects?
Not actually accusing anyone of quite such a serious crime. For reference, here’s my charge sheet:
Zycher: Writing incomprehensible babble, plus one misdemeanor count of cliche for the “undergraduate” reference.
Sauer: One count of being a me-too.
Reynolds: Posting impressive-looking gobbledigook without understanding what it means.
I think that Tim would add “being full of it” to Zycher’s charges, and I am currently pondering adding “trying to baffle people with science in order to make an issue look less cut and dried than it is” to Reynolds corpus delicti.
dd
To be fair to InstaPundit, nofundy, he didn’t buy Zycher’s hack piece, either.
“To be fair to InstaPundit”
WHo do you think I am? Kevin bloody Drum?
What nofundy said.
If I had seen the article you linked in a different context, I would probably have moved Lott/Rosh’s study off the “Debunked” shelf onto the “Undetermined” shelf. I certainly wouldn’t have begun the study of econometrics.
What the Rand guys did is technically known as “making shit up”, and it’s a prime right-wing debating technique these days. (Left-wingers use it too, but I’m a Stalinist, so I’m cool with that). Other valuable techniques include “throwing shit at the wall” and the “George Chuvalo” tactic: you remain standing while the other guy hits you in the face over and over again.
In the blogosphere there’s no referee, no scorekeeper, and you can’t physically hit the sons of bitches, so bad theories and untrue facts are immortal. People are still talking about Saddam’s mobile biological warfare vans, for example, months after they were shown to be used to produce weather ballons.
“WHo do you think I am? Kevin bloody Drum?”
Heh. Indeed.
Yeah, I was just reading this at Reynold’s site, and I thought he was quite unfriendly to Zycher. Okay, “unfriendly” may not be the correct word. But I hardly think one can reasonably conclude that Reynolds is supporting Zycher.
I think my favourite bit was this:
“Does Lindgren believe that Lott would invent an entire survey for the purpose of adding a sentence or two to a book of well over 200 pages?”
Er… Does Zycher believe that Lott conducted the survey — at personal expense, no less — for the purpose of adding a sentence or two to a book of well over 200 pages? Seems like an awful lot of effort for a throwaway line. And then, strangely, Lott forgot about the survey and attributed the 98% to “national surveys,” remembering his survey only after the number was called into question.
Yeah, I think we’re perfectly justified in believing that Lott would invent an entire survey for this.
months after they were shown to be used to produce weather ballons
Yes, but they were evil weather balloons, full of malice for their neighbors.
I’m pretty sure Hurican Isabel was linked to Iraqi weather balloons.
“They were evil weather balloons, full of malice for their neighbors.”
That’s a fine first line for a novel. Or something.
I’d also suggest that Reynolds “I’m not sure I agree with this” disclaimer struck me as a bit “disinglennuous”. I operate on a basis of strict liability for this sort of thing; you print it, you own it.
You print it, you own it.
Well, you printed:
To put it bluntly: Any undergraduate student receiving a B or better in introductory Econometrics would be able to pick the Ayres/Donohue work apart.
Do you own that? Now let’s look at what Disinglennuous Reynolds said:
I don’t find this very persuasive, I’m afraid, since it consists mostly of assertions that people are idiots, but without much actual exposition.I find Jim Lindgren quite credible, and I don’t think that assertions that he is biased are very persuasive. Assertions that he doesn’t understand elementary statistics would be more persuasive if accompanied by explanations.
…
I find Lindgren credible; Ayres and Donohue, too, though they’re anti-gun.
Reynolds admits his bias:
I would, of course, prefer to have it turn out that Lott is correct and that his critics are mistaken.
Finally, Reynolds gives Lindgren about ten-thousand words of rebuttal inline, links to Tim Lambert, and so on: hardly the actions of someone who wants to make “untrue facts…immortal”. Here’s a challenge for you: find a pro-gun blogger citing Zycher’s little flame-fest as a “plus” argument.
I won’t hold my breath.
“Kevin bloody drum”?
No. I think you’re a “fat young man without a good word for anybody” (or something along those lines.)
And on the weighty topic of weight, was is it about you economists? you admit to being “fat” and Brad’s picture on his website shows someone who appears to be able to afford to lose a few pounds. Try Atkins, or something. You and he are far too entertaining to lose to diabetes, stroke etc.
cheers
Dipnut: Whether or not Reynolds is culpable, the authors and publishers of the article are. I try to avoid 2nd Amendment discussions, but I’m not always successful, and I expect to see Zycher’s piece again.
Zizka,
Reynolds is the only “publisher” of Zycher’s diatribe, as far as I know. And he strangled it in the cradle, despite being sympathetic to the author on the issues. Zycher simply didn’t measure up, analytically or rhetorically.
You won’t be seeing it again, except maybe in some penny-ante high-school blog. And this particular 2nd-Amendment zealot is just as happy.
Lott has not weathered the attacks on him gracefully. He may well be a fraud and/or hack. As a result of the cloud hanging over him, Lott’s “facts” are anything but immortal. He is hardly ever cited by gun-rights proponents anymore, and his scholarship is not missed. We simply don’t need him.
On the other hand, there are any number of “untrue facts”, far easier to debunk than any of Lott’s work, which have achieved immortal status on the anti-gun side of the debate.
On a completely unrelated note, here’s some stuff I happened on Daniel will probably like:
Is America the republic or the galactic empire?. Free-market.net’s libertarian analysis of Star Wars.
The Von Mises institute has a lot more search results for “Star Wars” than I’d expect. The best is The Political Economy of Star Wars.
Ahhhh I remember those. I still think nothing beats the VMI analysis of WWF wrestling as deeply anti-capitalist propaganda.
Daniel Davies criticises Zycher for the obscurity of Zycher’s allegation about a fundamental error in Ayres and Donohue. And he dubs me “the Ultimate Lott Trainspotter”...
Read more at John Lott's Unethical Conduct
À Gauche
Jeremy Alder
Amaravati
Anggarrgoon
Audhumlan Conspiracy
H.E. Baber
Philip Blosser
Paul Broderick
Matt Brown
Diana Buccafurni
Brandon Butler
Keith Burgess-Jackson
Certain Doubts
David Chalmers
Noam Chomsky
The Conservative Philosopher
Desert Landscapes
Denis Dutton
David Efird
Karl Elliott
David Estlund
Experimental Philosophy
Fake Barn County
Kai von Fintel
Russell Arben Fox
Garden of Forking Paths
Roger Gathman
Michael Green
Scott Hagaman
Helen Habermann
David Hildebrand
John Holbo
Christopher Grau
Jonathan Ichikawa
Tom Irish
Michelle Jenkins
Adam Kotsko
Barry Lam
Language Hat
Language Log
Christian Lee
Brian Leiter
Stephen Lenhart
Clayton Littlejohn
Roderick T. Long
Joshua Macy
Mad Grad
Jonathan Martin
Matthew McGrattan
Marc Moffett
Geoffrey Nunberg
Orange Philosophy
Philosophy Carnival
Philosophy, et cetera
Philosophy of Art
Douglas Portmore
Philosophy from the 617 (moribund)
Jeremy Pierce
Punishment Theory
Geoff Pynn
Timothy Quigley (moribund?)
Conor Roddy
Sappho's Breathing
Anders Schoubye
Wolfgang Schwartz
Scribo
Michael Sevel
Tom Stoneham (moribund)
Adam Swenson
Peter Suber
Eddie Thomas
Joe Ulatowski
Bruce Umbaugh
What is the name ...
Matt Weiner
Will Wilkinson
Jessica Wilson
Young Hegelian
Richard Zach
Psychology
Donyell Coleman
Deborah Frisch
Milt Rosenberg
Tom Stafford
Law
Ann Althouse
Stephen Bainbridge
Jack Balkin
Douglass A. Berman
Francesca Bignami
BlunkettWatch
Jack Bogdanski
Paul L. Caron
Conglomerate
Jeff Cooper
Disability Law
Displacement of Concepts
Wayne Eastman
Eric Fink
Victor Fleischer (on hiatus)
Peter Friedman
Michael Froomkin
Bernard Hibbitts
Walter Hutchens
InstaPundit
Andis Kaulins
Lawmeme
Edward Lee
Karl-Friedrich Lenz
Larry Lessig
Mirror of Justice
Eric Muller
Nathan Oman
Opinio Juris
John Palfrey
Ken Parish
Punishment Theory
Larry Ribstein
The Right Coast
D. Gordon Smith
Lawrence Solum
Peter Tillers
Transatlantic Assembly
Lawrence Velvel
David Wagner
Kim Weatherall
Yale Constitution Society
Tun Yin
History
Blogenspiel
Timothy Burke
Rebunk
Naomi Chana
Chapati Mystery
Cliopatria
Juan Cole
Cranky Professor
Greg Daly
James Davila
Sherman Dorn
Michael Drout
Frog in a Well
Frogs and Ravens
Early Modern Notes
Evan Garcia
George Mason History bloggers
Ghost in the Machine
Rebecca Goetz
Invisible Adjunct (inactive)
Jason Kuznicki
Konrad Mitchell Lawson
Danny Loss
Liberty and Power
Danny Loss
Ether MacAllum Stewart
Pam Mack
Heather Mathews
James Meadway
Medieval Studies
H.D. Miller
Caleb McDaniel
Marc Mulholland
Received Ideas
Renaissance Weblog
Nathaniel Robinson
Jacob Remes (moribund?)
Christopher Sheil
Red Ted
Time Travelling Is Easy
Brian Ulrich
Shana Worthen
Computers/media/communication
Lauren Andreacchi (moribund)
Eric Behrens
Joseph Bosco
Danah Boyd
David Brake
Collin Brooke
Maximilian Dornseif (moribund)
Jeff Erickson
Ed Felten
Lance Fortnow
Louise Ferguson
Anne Galloway
Jason Gallo
Josh Greenberg
Alex Halavais
Sariel Har-Peled
Tracy Kennedy
Tim Lambert
Liz Lawley
Michael O'Foghlu
Jose Luis Orihuela (moribund)
Alex Pang
Sebastian Paquet
Fernando Pereira
Pink Bunny of Battle
Ranting Professors
Jay Rosen
Ken Rufo
Douglas Rushkoff
Vika Safrin
Rob Schaap (Blogorrhoea)
Frank Schaap
Robert A. Stewart
Suresh Venkatasubramanian
Ray Trygstad
Jill Walker
Phil Windley
Siva Vaidahyanathan
Anthropology
Kerim Friedman
Alex Golub
Martijn de Koning
Nicholas Packwood
Geography
Stentor Danielson
Benjamin Heumann
Scott Whitlock
Education
Edward Bilodeau
Jenny D.
Richard Kahn
Progressive Teachers
Kelvin Thompson (defunct?)
Mark Byron
Business administration
Michael Watkins (moribund)
Literature, language, culture
Mike Arnzen
Brandon Barr
Michael Berube
The Blogora
Colin Brayton
John Bruce
Miriam Burstein
Chris Cagle
Jean Chu
Hans Coppens
Tyler Curtain
Cultural Revolution
Terry Dean
Joseph Duemer
Flaschenpost
Kathleen Fitzpatrick
Jonathan Goodwin
Rachael Groner
Alison Hale
Household Opera
Dennis Jerz
Jason Jones
Miriam Jones
Matthew Kirschenbaum
Steven Krause
Lilliputian Lilith
Catherine Liu
John Lovas
Gerald Lucas
Making Contact
Barry Mauer
Erin O'Connor
Print Culture
Clancy Ratcliff
Matthias Rip
A.G. Rud
Amardeep Singh
Steve Shaviro
Thanks ... Zombie
Vera Tobin
Chuck Tryon
University Diaries
Classics
Michael Hendry
David Meadows
Religion
AKM Adam
Ryan Overbey
Telford Work (moribund)
Library Science
Norma Bruce
Music
Kyle Gann
ionarts
Tim Rutherford-Johnson
Greg Sandow
Scott Spiegelberg
Biology/Medicine
Pradeep Atluri
Bloviator
Anthony Cox
Susan Ferrari (moribund)
Amy Greenwood
La Di Da
John M. Lynch
Charles Murtaugh (moribund)
Paul Z. Myers
Respectful of Otters
Josh Rosenau
Universal Acid
Amity Wilczek (moribund)
Theodore Wong (moribund)
Physics/Applied Physics
Trish Amuntrud
Sean Carroll
Jacques Distler
Stephen Hsu
Irascible Professor
Andrew Jaffe
Michael Nielsen
Chad Orzel
String Coffee Table
Math/Statistics
Dead Parrots
Andrew Gelman
Christopher Genovese
Moment, Linger on
Jason Rosenhouse
Vlorbik
Peter Woit
Complex Systems
Petter Holme
Luis Rocha
Cosma Shalizi
Bill Tozier
Chemistry
"Keneth Miles"
Engineering
Zack Amjal
Chris Hall
University Administration
Frank Admissions (moribund?)
Architecture/Urban development
City Comforts (urban planning)
Unfolio
Panchromatica
Earth Sciences
Our Take
Who Knows?
Bitch Ph.D.
Just Tenured
Playing School
Professor Goose
This Academic Life
Other sources of information
Arts and Letters Daily
Boston Review
Imprints
Political Theory Daily Review
Science and Technology Daily Review