How Professors Think shows that across all disciplines, evaluations depend on taste and expertise. Because tastes are often idiosyncratic (what is “fascinating” is what looks like me or reinforces my own line of scholarship), funding agencies should explicitly instruct panelists to value the quality of the proposal (expertise) more than what excites (a matter of taste).
How Professors Think also shows that economists and historians have very different views concerning where excellencet resides – in the object being evaluated or in the eyes of the beholder. While economists think that excellence is objective and is to be found in the proposal itself (that a clear line separates what is first rate from the rest), scholars hailing from more interpretive fields believe that evaluators play a central role in giving value to the proposals – indeed, that they are engaged in the coproduction of excellence. While participating in panel deliberations, they produce what they hope will be convincing arguments about what is good work. They don’t think that their views – their subjectivity – corrupt the process. Instead, they think it is essential to the process, because they are asked to serve in their quality as connoisseurs, as experts who have spent many years developing a very refined classification system for understanding what the field has already produced and what is new and promising.
My book discusses in some detail the evaluative culture of historians. Interestingly, and perhaps surprisingly to some, I characterize history as presenting a relatively high degree of consensus about what constitutes quality and how to recognize it. At least, this is what historians told me about their field.
When historians compare their field with English literature – a contrast that could not be starker – they say that in their field “the disciplinary center holds.” According to one interviewee,
History hasn’t been politicized in the way some fields have, in a kind of roughly post-modern sort of approach to history. In the wider field you get a range, but the range is reasonably narrow. There are not so many people who would be writing in the language that would seem empty jargon, [that would be dismissed as a] bunch of junk by people who consider themselves empirical historians. You don’t have such a dominant group of people who are very engaged in cultural theory, who would just simply dismiss arbitrarily work that is narrowly empirical. The middle is pretty big, pretty calm, not overtly politicized, and the ends I think are relatively small [my emphasis]. The idea that evidence does matter, that giving attention to theory at the same time is a good thing, I think both of those do probably hold.
This peaceful state of affairs is not based on a notion that the field is (or can be) unified around a common theory. Rather, in the opinion of a particularly distinguished early American historian, what is shared is agreement on what constitutes good historical craftsmanship, a sense of “careful archival work.” A European historian concurs:
We are neither English, nor political science . . . We see ourselves as an interpretive, empirically grounded social science. . . I think that grounding in [the] empirical is something strong that makes historians sort of have more of an idea of “what’s new here?” [my emphasis]. Research is oriented toward getting results. Theory is useful, but not paramount. Those disciplines that tend to have less agreement are based more on rhetoric, on personalism, that is, “I worked with da, da, da, or this is my theory,” and they have no tangible way to judge excellence..
How Professors Think examines closely the evaluative cultures of six disciplines and only economics reaches a degree of consensus comparable to that of history – but in this case, disciplinary consensus is based on a shared culture of mathematical formalism, rather than in a shared recognition of standards for good craftsmanship (the other disciplines considered are philosophy, English, anthropology and political science). Accordingly, in several of the competitions I studied, historians were perceived as receiving the lion’s share of awards. This is explained, in part, because they apply in such large numbers, and are always represented on panels. But their success is also attributed to disciplinary consensus about how to recognize quality. The disciplinary fault lines might be deeper were I comparing tensions within subfields, such as American history or Chinese history. Were the discipline smaller, it might be characterized by more conflict. In any case, it is significant to know that various types of disciplinary consensus exist across the social sciences and the humanities. And that a shared sense of what defines good craftsmanship can act as a common disciplinary matrix of evaluation that is as powerful as shared mathematical tools.