Unless there was fraud in the original work. To say that results cannot be replicated in the strictest possible sense is an accusation of fraud, to say that they are not robust to different models or slightly different data sets is another thing.

I am also an economist, and that would be my understanding of “replication” as well. My take on the issue is that this is yet another example of John Lott attempting to stifle any and all legitimate criticism of his research.

For example, assume that the data was manifestly flawed. Assume that the mathematical model was equally flawed. Obviously, the product of such a model would be crap. And yet you could “replicate” the analysis by applying the flawed model on flawed data.

If scientific “replication” requires accepting manifestly flawed data and methodologies, then the whole enterprise is worthless.

He’s claiming that they’re cherry picking, in other words.”

Nope. The “failure to replicate” statement came with a citation to an article that used Lott’s methodology and dataset but that corrected some of the coding errors he made. If “replicate” unambiguosly means: run the analysis with the same method on the same data if these contain errors, then Lott has a case.

Brett Bellmore: 50 states is all the data? Hmm, I guess now it’s my turn to restore the fundamental equilibrium of the universe, where by “universe” I mean “America”.

However, his objection to people using different data sets, appears to be that he went out and collected the largest possible data set, all 50 states, and that anybody who is using a different set of data can only be using a subset of the data he used.

He’s claiming that they’re cherry picking, in other words.

It isn’t, is it? In Lott’s sense wouldn’t this mean re-running the analysis of the experiment using the original data.

ahhh I see what you mean, although given the crap that Lott has produced with simple linear regressions, the idea of him getting access to maximum likelihood estimation fills me with terror. I seem to remember he did actually develop an enthusiasm for the twice censored tobit once actually, although not in the guns n crime work.

Mostly, replication in biology means repeating the experiment with new subjects, but the same protocols. The analogous case in economics is fitting the same model to a new or expanded data set.

(After writing this, I see I’m replicating eudoxis).

Hmmm. In that case, the method is the same but the data are collected independently. Lott claims that both the data and the method must be the same.

But clearly this fool Lott just keeps digging himself deeper and deeper.

It is a very pretty hobby horse that he likes to ride, isn’t it? Especially with the after-market shotgun rack.

Given all the self-inflicted damage to Lott’s reputation, how could “replicate,” even if used falsely and maliciously, hurt Lott’s reputation further?