My namesake, Tom Quiggin has been in the news lately, debunking the idea that Al Qaeda cultivates sleeper agents and also tracing to its source the urban myth that Osama Bin Laden used a private fortune of $300 million to promote the group.

He’s sent me some reflections on the sloppy research that’s been used to promote some of these ideas, noting

. A disconnect between the statement in the body of the article and the sources in the footnotes which do not back up the statement being made,
2. Strong statements which are made, but which are built on weak foundations or on assumptions which cannot be shown to be valid,
3. Information from two different situations is overlapped or mixed together, leaving the reader with a false impression about the nature of a particular problem or situation,
4. In a limited number of cases, information provided in articles is simply false.

The faults he points out are, I think, found to some extent in every field (I’ve certainly found plenty of instances in economics, though the prevailing flaws are a bit different), but fields like the study of security issues have the added problem that replication and verification are particularly difficult. Processes such as peer review, replication and empirical testing aren’t panaceas, and errors will always slip through, but they work pretty well in the long run.