Comments on: Selection bias

By: Jurjen

Jurjen — Mon, 14 Jul 2003 10:07:08 +0000

The problem with any non-natural science is that it's a lot easier to determine what not to do than it is to determine what is the right thing to do; the old saw of "we cannot determine what is true, we can only identify waht is demonstrably not true and eliminate it." In 1994, doing my national service, I took part in a "computer-aided exercise" (CAX); the command posts at brigade level and above were in the field, with the actual combat units only taking part in a computer simulation. My brigade, and most of the rest of the Dutch army for that matter, was (theoretically) annihilated in the exercise. I asked our operations officer why this was, and he replied that the purpose of any military exercise is to learn from your mistakes, and the best way to achieve that is by losing. To me, this made sense. As a result, it strikes me that when one feels the urge to publish an article or book with a title like "What really works," the best thing one can do is go and lie down until the urge passes. Better to establosh (and eliminate) what doesn't work, methinks.

By: dsquared

dsquared — Mon, 14 Jul 2003 06:54:21 +0000

Indeed; but there are worse offenders. What about those medical schools who spent years and years ignoring the vast numbers of people who smoked cigarettes and didn’t die of lung cancer? I don’t think that sampling on dependent variables is necessarily always and everywhere a mortal sin.

By: Kieran Healy

Kieran Healy — Mon, 14 Jul 2003 03:48:59 +0000

Repeat after me, everyone: do not sample on the dependent variable.

By: pathos

pathos — Mon, 14 Jul 2003 02:29:03 +0000

The problem with business school courses and their unfortunate offspring—management consultants, is unfortunately hard to determine due to its wifty terminology. I believe, thought, that the problem can better be viewed as a weak version of what Malcolm Gladwell recently discussed in the New Yorker in the context of foreign affairs—“Creeping Determinism.” With creeping determinism, everything become immediately obvious in retrospect. http://www.gladwell.com/2003/2003_03_10_a_dots.html In the 1990s, when some companies became successful by “breaking all the rules,” that became the mantra. OBVIOUSLY, the way to succeed is to not do what everyone else is doing. When the rule-breakers turned out to be law-breakers, too, it suddently became obvious that breaking all the rules was not the way to go. The rules were there for a reason, after all. So, this generation of analysis consists of stuff that you find “bloody obvious.” This is because we’re starting with the conclusion and working backwards. You can’t do a real scientific study because what if the results show that successful firms are random? Or that they disproportionately benefit from government contracts or regulations? Or that they have a business plan that reads to close to Enron’s? B-schools can’t do real research because of the real risk that it will be determined that teaching “management” is no different from teaching astrology. Or maybe not. The key to good health is “eat reasonable portions, eat sufficient fruits and vegetables, and exercise regularly.” Everyone knows that. No one can debate it. But billions of dollars go into new research to determine if high carb foods cause more heart attacks than high sugar foods or some such nonesense.

By: lawrence krubner

lawrence krubner — Mon, 14 Jul 2003 02:16:17 +0000

Also, the money splashing around inside of business and law schools distorts the social reality in which the researchers operate. Any grad student who is getting a Ph.d in philosophy must have a genuine passion for philosophy. They obviously aren’t doing it for the money. At business and law schools, the opposite presumption is reasonable. Concerns about socials status also limit some of the informality that poverty forces on researchers, especially grad students, in many other fields. Excellent research can be done despite concerns with social status, a lack of informality, and by people whose only interest is money. However, these factors do explain why the researchers in these fields operate in an environment so much different than most of other university research departments.