The Economist has a long article asking whether or not companies are too risk averse to take proper advantage of new opportunities and a changing marketplace. Explaining the roots of corporate advantage is tricky stuff; conventional economic theory isn’t very good at telling us when efforts to innovate are going to be successful, and when they’re not. Economic sociologists do a slightly better job, but they still have difficulty in providing useful lessons for business people. Which opens the way for all sorts of cranks and quacks, who offer dubious nostrums for business success, with all the fervid enthusiasm of a 19th century medicine show charlatan. I’m referring of course to management “theorists.”
Now, I don’t want to dismiss them all out of hand - some interesting and serious work does get done at business schools. But management theory has more than its fair share of fakers, especially towards the populist end of the market. Grand, sweeping claims are made on the basis of very iffy and dubious research and case studies which aren’t chosen for any better reason than that they can be shoe-horned into the analytic boxes prescribed by the prevailing wisdom.
Which brings us back to the Economist article. It gives a guided tour of business school research on creative ways to deal with changing circumstances. The survey gives the impression that the field is a mess: different management theorists shouting bland (but somehow mutually contradictory) business prescriptions at each other. For example, one major five year research program comes to the conclusion that successful companies are distinguished from their peers by four factors - flawless execution, a company culture based on aiming high, a structure that is flexible and responsive, and a clear and focused strategy. Bloody obvious stuff in other words, which any half way intelligent senior executive could jot down on the back of a beermat in five minutes.
Why is this stuff so bad? Two possible reasons I can think of. The first is that management theory is less interested in the patient accumulation of real knowledge, than in providing training-fodder for future managers. Probably not all that much that can be done about this - b-school professors get whopping salaries for training MBAs, and are likely disinclined to change their ways. But the second reason is a problem that they could address without too much difficulty - their research methodology is often just awful.
The Economist article gives an apparent example of this in its discussion of two highly influential books - Built to Last by Jim Collins and Jerry Poras, and Good to Great by Jim Collins on his own. Now I haven’t read these books, so I’m going on the Economist’s description of what they have to say - it could be that they’re innocent of all the charges laid below, and are serious pieces of research. They’re almost certainly a considerable cut above the Who Ate My Cheese genre of populist business nonsense. But it still sounds as though these books commit a serious social-science sin - “selecting on the dependent variable.”
What does this mean? Social scientists tend to believe that if you want to find out if a causes b by studying different cases, you need to be quite careful in choosing the cases. For example, if you want to argue that risk taking leads to business success, you want to look at cases of firms that are risk takers, and firms that are risk averse, and you also want to have cases of firms that are successful, and cases of firms that are failures. If you only study successful risk-taking firms, you’re cooking the books. It could be that there are many more risk-taking firms that are failures out there than successes - but because you’ve only chosen to look at the successes, you have no way of knowing this. You can thus end up providing pretty bad advice.
As far as I can tell from the summary in the Economist , Built to Last makes a mistake of just this sort, when it tries to figure out the source of business success by looking
at a small sample of companies (18) that had been persistently great over time. It suggested that endurance and performance were linked.
Apparently, so does Good to Great which looks at companies which spectacularly outperformed the stock-market over a three year period, and argues that the source of their success is quietly determined CEOs who believe in high standards.
The problem is that studies which concentrate on successful firms can’t tell us anything very useful about the differences between successful and unsuccessful firms, because they haven’t looked at the latter at all. We don’t know whether or not other companies, which also endured a long time, had below average performance. And very likely, many did (at least that’s the conclusion of another piece of research that The Economist also cites, which sounds a little more satisfactory). Equally, companies that tanked may also have had quietly determined CEOs who believe in high standards (which in any event sounds like a very hazy and subjective judgement on the researcher’s part - how can you really tell the one kind of CEO from the other).
Case-studies play a big part in business-school education; they’re a useful way to make students work through the consequences of real life decisions. And they can, indeed, contribute to our more general stock of knowledge, if scholars use certain methodologies (such as ‘process tracing’) carefully. But management theory not only relies heavily on case studies - it all too often uses vague and wuffly concepts, that reflect the pre-conceived biases of the researcher rather than grounded theories. Thus its rather extraordinary noise-to-signal ratio - it’s less suited to providing new insights than new jargon to be wielded by McKinsey consultants in order to terrify staid managers into submission.
Also, the money splashing around inside of business and law schools distorts the social reality in which the researchers operate. Any grad student who is getting a Ph.d in philosophy must have a genuine passion for philosophy. They obviously aren’t doing it for the money. At business and law schools, the opposite presumption is reasonable.
Concerns about socials status also limit some of the informality that poverty forces on researchers, especially grad students, in many other fields.
Excellent research can be done despite concerns with social status, a lack of informality, and by people whose only interest is money. However, these factors do explain why the researchers in these fields operate in an environment so much different than most of other university research departments.
The problem with business school courses and their unfortunate offspring — management consultants, is unfortunately hard to determine due to its wifty terminology.
I believe, thought, that the problem can better be viewed as a weak version of what Malcolm Gladwell recently discussed in the New Yorker in the context of foreign affairs — “Creeping Determinism.”
With creeping determinism, everything become immediately obvious in retrospect.
http://www.gladwell.com/2003/2003_03_10_a_dots.html
In the 1990s, when some companies became successful by “breaking all the rules,” that became the mantra. OBVIOUSLY, the way to succeed is to not do what everyone else is doing. When the rule-breakers turned out to be law-breakers, too, it suddently became obvious that breaking all the rules was not the way to go. The rules were there for a reason, after all.
So, this generation of analysis consists of stuff that you find “bloody obvious.” This is because we’re starting with the conclusion and working backwards. You can’t do a real scientific study because what if the results show that successful firms are random? Or that they disproportionately benefit from government contracts or regulations? Or that they have a business plan that reads to close to Enron’s?
B-schools can’t do real research because of the real risk that it will be determined that teaching “management” is no different from teaching astrology.
Or maybe not. The key to good health is “eat reasonable portions, eat sufficient fruits and vegetables, and exercise regularly.” Everyone knows that. No one can debate it. But billions of dollars go into new research to determine if high carb foods cause more heart attacks than high sugar foods or some such nonesense.
Repeat after me, everyone: do not sample on the dependent variable.
Indeed; but there are worse offenders. What about those medical schools who spent years and years ignoring the vast numbers of people who smoked cigarettes and didn’t die of lung cancer? I don’t think that sampling on dependent variables is necessarily always and everywhere a mortal sin.
The problem with any non-natural science is that it’s a lot easier to determine what not to do than it is to determine what is the right thing to do; the old saw of “we cannot determine what is true, we can only identify waht is demonstrably not true and eliminate it.”
In 1994, doing my national service, I took part in a “computer-aided exercise” (CAX); the command posts at brigade level and above were in the field, with the actual combat units only taking part in a computer simulation. My brigade, and most of the rest of the Dutch army for that matter, was (theoretically) annihilated in the exercise. I asked our operations officer why this was, and he replied that the purpose of any military exercise is to learn from your mistakes, and the best way to achieve that is by losing. To me, this made sense.
As a result, it strikes me that when one feels the urge to publish an article or book with a title like “What really works,” the best thing one can do is go and lie down until the urge passes. Better to establosh (and eliminate) what doesn’t work, methinks.
À Gauche
Jeremy Alder
Amaravati
Anggarrgoon
Audhumlan Conspiracy
H.E. Baber
Philip Blosser
Paul Broderick
Matt Brown
Diana Buccafurni
Brandon Butler
Keith Burgess-Jackson
Certain Doubts
David Chalmers
Noam Chomsky
The Conservative Philosopher
Desert Landscapes
Denis Dutton
David Efird
Karl Elliott
David Estlund
Experimental Philosophy
Fake Barn County
Kai von Fintel
Russell Arben Fox
Garden of Forking Paths
Roger Gathman
Michael Green
Scott Hagaman
Helen Habermann
David Hildebrand
John Holbo
Christopher Grau
Jonathan Ichikawa
Tom Irish
Michelle Jenkins
Adam Kotsko
Barry Lam
Language Hat
Language Log
Christian Lee
Brian Leiter
Stephen Lenhart
Clayton Littlejohn
Roderick T. Long
Joshua Macy
Mad Grad
Jonathan Martin
Matthew McGrattan
Marc Moffett
Geoffrey Nunberg
Orange Philosophy
Philosophy Carnival
Philosophy, et cetera
Philosophy of Art
Douglas Portmore
Philosophy from the 617 (moribund)
Jeremy Pierce
Punishment Theory
Geoff Pynn
Timothy Quigley (moribund?)
Conor Roddy
Sappho's Breathing
Anders Schoubye
Wolfgang Schwartz
Scribo
Michael Sevel
Tom Stoneham (moribund)
Adam Swenson
Peter Suber
Eddie Thomas
Joe Ulatowski
Bruce Umbaugh
What is the name ...
Matt Weiner
Will Wilkinson
Jessica Wilson
Young Hegelian
Richard Zach
Psychology
Donyell Coleman
Deborah Frisch
Milt Rosenberg
Tom Stafford
Law
Ann Althouse
Stephen Bainbridge
Jack Balkin
Douglass A. Berman
Francesca Bignami
BlunkettWatch
Jack Bogdanski
Paul L. Caron
Conglomerate
Jeff Cooper
Disability Law
Displacement of Concepts
Wayne Eastman
Eric Fink
Victor Fleischer (on hiatus)
Peter Friedman
Michael Froomkin
Bernard Hibbitts
Walter Hutchens
InstaPundit
Andis Kaulins
Lawmeme
Edward Lee
Karl-Friedrich Lenz
Larry Lessig
Mirror of Justice
Eric Muller
Nathan Oman
Opinio Juris
John Palfrey
Ken Parish
Punishment Theory
Larry Ribstein
The Right Coast
D. Gordon Smith
Lawrence Solum
Peter Tillers
Transatlantic Assembly
Lawrence Velvel
David Wagner
Kim Weatherall
Yale Constitution Society
Tun Yin
History
Blogenspiel
Timothy Burke
Rebunk
Naomi Chana
Chapati Mystery
Cliopatria
Juan Cole
Cranky Professor
Greg Daly
James Davila
Sherman Dorn
Michael Drout
Frog in a Well
Frogs and Ravens
Early Modern Notes
Evan Garcia
George Mason History bloggers
Ghost in the Machine
Rebecca Goetz
Invisible Adjunct (inactive)
Jason Kuznicki
Konrad Mitchell Lawson
Danny Loss
Liberty and Power
Danny Loss
Ether MacAllum Stewart
Pam Mack
Heather Mathews
James Meadway
Medieval Studies
H.D. Miller
Caleb McDaniel
Marc Mulholland
Received Ideas
Renaissance Weblog
Nathaniel Robinson
Jacob Remes (moribund?)
Christopher Sheil
Red Ted
Time Travelling Is Easy
Brian Ulrich
Shana Worthen
Computers/media/communication
Lauren Andreacchi (moribund)
Eric Behrens
Joseph Bosco
Danah Boyd
David Brake
Collin Brooke
Maximilian Dornseif (moribund)
Jeff Erickson
Ed Felten
Lance Fortnow
Louise Ferguson
Anne Galloway
Jason Gallo
Josh Greenberg
Alex Halavais
Sariel Har-Peled
Tracy Kennedy
Tim Lambert
Liz Lawley
Michael O'Foghlu
Jose Luis Orihuela (moribund)
Alex Pang
Sebastian Paquet
Fernando Pereira
Pink Bunny of Battle
Ranting Professors
Jay Rosen
Ken Rufo
Douglas Rushkoff
Vika Safrin
Rob Schaap (Blogorrhoea)
Frank Schaap
Robert A. Stewart
Suresh Venkatasubramanian
Ray Trygstad
Jill Walker
Phil Windley
Siva Vaidahyanathan
Anthropology
Kerim Friedman
Alex Golub
Martijn de Koning
Nicholas Packwood
Geography
Stentor Danielson
Benjamin Heumann
Scott Whitlock
Education
Edward Bilodeau
Jenny D.
Richard Kahn
Progressive Teachers
Kelvin Thompson (defunct?)
Mark Byron
Business administration
Michael Watkins (moribund)
Literature, language, culture
Mike Arnzen
Brandon Barr
Michael Berube
The Blogora
Colin Brayton
John Bruce
Miriam Burstein
Chris Cagle
Jean Chu
Hans Coppens
Tyler Curtain
Cultural Revolution
Terry Dean
Joseph Duemer
Flaschenpost
Kathleen Fitzpatrick
Jonathan Goodwin
Rachael Groner
Alison Hale
Household Opera
Dennis Jerz
Jason Jones
Miriam Jones
Matthew Kirschenbaum
Steven Krause
Lilliputian Lilith
Catherine Liu
John Lovas
Gerald Lucas
Making Contact
Barry Mauer
Erin O'Connor
Print Culture
Clancy Ratcliff
Matthias Rip
A.G. Rud
Amardeep Singh
Steve Shaviro
Thanks ... Zombie
Vera Tobin
Chuck Tryon
University Diaries
Classics
Michael Hendry
David Meadows
Religion
AKM Adam
Ryan Overbey
Telford Work (moribund)
Library Science
Norma Bruce
Music
Kyle Gann
ionarts
Tim Rutherford-Johnson
Greg Sandow
Scott Spiegelberg
Biology/Medicine
Pradeep Atluri
Bloviator
Anthony Cox
Susan Ferrari (moribund)
Amy Greenwood
La Di Da
John M. Lynch
Charles Murtaugh (moribund)
Paul Z. Myers
Respectful of Otters
Josh Rosenau
Universal Acid
Amity Wilczek (moribund)
Theodore Wong (moribund)
Physics/Applied Physics
Trish Amuntrud
Sean Carroll
Jacques Distler
Stephen Hsu
Irascible Professor
Andrew Jaffe
Michael Nielsen
Chad Orzel
String Coffee Table
Math/Statistics
Dead Parrots
Andrew Gelman
Christopher Genovese
Moment, Linger on
Jason Rosenhouse
Vlorbik
Peter Woit
Complex Systems
Petter Holme
Luis Rocha
Cosma Shalizi
Bill Tozier
Chemistry
"Keneth Miles"
Engineering
Zack Amjal
Chris Hall
University Administration
Frank Admissions (moribund?)
Architecture/Urban development
City Comforts (urban planning)
Unfolio
Panchromatica
Earth Sciences
Our Take
Who Knows?
Bitch Ph.D.
Just Tenured
Playing School
Professor Goose
This Academic Life
Other sources of information
Arts and Letters Daily
Boston Review
Imprints
Political Theory Daily Review
Science and Technology Daily Review