“Laura Rozen”:http://www.warandpiece.com/blogdirs/002423.html on revelations that Able Danger contractors lost their jobs after fingering Condoleeza Rice and William Perry as part of a web of relationships between China and US defence/security types.

bq. Able Danger’s data mining results seemed more all over the board, a kind of tinfoil hat producing adventure better left to freepsters and google?

Not necessarily so. There’s a lot of confusion about what data mining can and cannot do. Both its proponents (who want to get fundng for it), and its opponents (who want to conjure up images of Big Brother) have an interest in hyping up its capabilities. The fact that Able Danger or other data mining programs may throw up false positives doesn’t mean that data mining isn’t potentially useful. The _most_ that data mining can do (and should be expected to do) is sometimes to highlight interesting and non-obvious relationships that might otherwise have escaped people’s attentions. In the words of Mary DeRosa’s “CSIS report”:http://www.csis.org/tech/2004_counterterrorism.pdf on data mining and counter-terrorism (the best thing I’ve read on the topic), data mining may provide a set of ‘power tools’ for law enforcement and intelligence, which may suggest interesting further lines of investigation. Inevitably, however, it’s going to provide a lot of entirely spurious leads (indeed, if it doesn’t provide some dead-ends, its filters are probably set too narrowly). Thus, it shouldn’t be treated as providing smoking gun evidence the one way or the other – all that it does is to analyse sets of relationships in a network of actors, and highlight some relationships that might otherwise have been non-obvious.

So the important question isn’t whether Able Danger and related programs came up with some network connections that seemed on the face of it to be ridiculous (although in the unlikely event that the Able Danger people portrayed Rice as some class of a Manchurian candidate it would obviously be a serious problem). In order to figure out the underlying merits and defects of Able Danger, we’d need to have a lot more information than seems to be publicly available at the moment. How good was Able Danger _overall_ at filtering out the wheat from the chaff? What was the overall ratio of false positives to genuine positives? Was the data mining exercise that spat out Atta’s name (assuming that the Able Danger people are telling the truth) one of a whole bunch of data mining exercises, most of which came up with garbage? Did the specific exercise that came up with Atta’s name highlight him as playing a central role in the network, or at least a role that merited further investigation, or did it have him on the periphery of the network? At the moment, we simply don’t know enough to evaluate – instead, we seem to be in a wilderness of mirrors, with “conflicting leaks”:http://www.washingtonmonthly.com/archives/individual/2005_08/006988.php from pro- and anti-Able Danger types, all with their own agendas. The quick take as best as I can make out – if Able Danger singled out Atta as one of a small group of individuals who merited substantial further investigation, then the Pentagon has a problem. If Atta’s name was one of hundreds or thousands, the rest of whom were mostly false positives, or if the network analysis didn’t highlight Atta as someone who merited further investigation, then the Pentagon’s decision to close down the program is far more easily defensible _ex post_.

Don’t miss the “Guardian’s profile of evolutionary biologist Robert Trivers”:http://books.guardian.co.uk/review/story/0,12084,1556482,00.html . A nervous breakdown after reading too much Wittgenstein, friendship with Huey Newton of the Black Panthers and the following priceless comment on Richard Dawkins: “My first wife, a wonderful woman, used to refer to Dick as the Selfish Gene, just because of the way he acts. ” Definitely worth a look.