On The Governance of LLMs, and The University (of Chicago)

Sometimes people that know and like each other, and that would never employ snark with each other, can still talk entirely past each other online. Carlo Ludovico Cordasco (Sheffield) wrote a fruitful and prudent sub-stack post (here) on the ‘longstanding debate on AI and deskilling.’ As he notes it was prompted by my Kvetching about a 2 June announcement by The University of Chicago’s President that it has a contract with Anthropic to give all of its students, and all of its staff and faculty, full access to Claude Enterprise.

Now, I viewed that announcement by Paul Alivisatos (the University President) the way I interpret many of that university’s public announcements during the last two decades: as a cynical, branding ploy aiming to keep the university in the eyes of influencers who may alert full tuition paying parents that they should send their kids to the UofC. In my view, in its public communication, the university has stopped trying to be the academy for would-be-academics and those closely adjacent to it.

Anyway, when I first participated in howling about Alivisatos’ announcement, the fate of skill was far from my mind. I viewed the UofC as vice-signaling its path away from all that is noble and beautiful about higher education. (I explain my position on that near the end of this post.) But I was interpreted as a techno-luddite about the role of AI. I ruminated a bit on Carlo’s essay. In response, I focus more on governance than de-skilling.

I was never an undergraduate at The UofC. But on my second day of graduate school there, I remember walking through the undergraduate library (yes, there was an undergrad library!) surrounded by kids who clearly had pulled an all-nighter mostly with Homer, but a smattering of other thick books—some were asleep with the books on the desks, while others were engaged in intense discussions with each other. The most popular T-Shirt in my time there, was ‘Where Fun Goes to Die’ (and closely followed ‘Hell Freezes Over.’) The place had an ethos centered on its undergraduate core that involved reading, discussion, and writing and that permeated it. (Graduate students often taught in The Core so it wasn’t just for the kidz.)

While I was contemplating a response to Carlo’s essay, I decided to got back to a familiar passage in the Phaedrus. I read it with fresh eyes.[1] I quote the full paragraph:

I heard, then, that at Naucratis, in Egypt, was one of the ancient gods of that country, the one whose sacred bird is called the ibis, and the name of the god himself was Theuth. He it was who invented numbers and arithmetic and geometry and astronomy, also draughts and dice, and, most important of all, letters. Now the king of all Egypt at that time was the god Thamus, who lived in the great city of the upper region, which the Greeks call the Egyptian Thebes, and they call the god himself Ammon. To him came Theuth to show his inventions, saying that they ought to be imparted to the other Egyptians. But Thamus asked what use there was in each, and as Theuth enumerated their uses, expressed praise or blame, according as he approved or disapproved. The story goes that Thamus said many things to Theuth in praise or blame of the various arts, which it would take too long to repeat; but when they came to the letters, “This invention, O king,” said Theuth, “will make the Egyptians wiser and will improve their memories; for it is an elixir of memory and wisdom that I have discovered.” But Thamus replied, “Most ingenious Theuth, one man has the ability to beget arts, but the ability to judge of their usefulness or harmfulness to their users belongs to another; and now you, who are the father of letters, have been led by your affection to ascribe to them a power the opposite of that which they really possess. For this invention will produce forgetfulness in the minds of those who learn to use it, because they will not practice their memory. Their trust in writing, produced by external characters which are no part of themselves, will discourage the use of their own memory within them. You have invented an elixir not of memory, but of reminding; and you offer your pupils the appearance of wisdom, not true wisdom, for they will read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant and hard to get along with, since they are not wise, but only appear wise.—Phaedrus, 274-275, translated by Harold N. Fowler.

Naucratis was an open commercial port on the Nile that was the unique point of entry into Egypt. By  Socrates’ time had, somewhat unusually, been populated, as Herodotus attests, by Greeks for several generations. So, it is fascinating that Socrates makes the God of the Underworld, and patron of scribes, Thoth/Theuth, dwell there in that cosmopolitan port. One is made to wonder whether this God didn’t pick up the use of letters from some travelers (say Phoenicians), or invented it to mediate or judge a commercial dispute.

As an aside, I would be amazed if Socrates (or Plato) were not alluding to Herodotus in this narrative. For, according to Herodotus, Cadmus brought the Phoenician alphabet to Greece and founded Boeotian Thebes. And the fact that there are two Thebes means one must presuppose a wider world to keep them apart. That is to be distinguished, of course, from entities with multiple names.

Now, while it is tempting to rush ahead to the part of the story where deskilling is discussed, I want to pause for a second at the division of labor between Theuth and Thamus/Ammon. Theuth invents, but Ammon is the regulator who decides whether an invention can be circulated and to what extent. Wisely, these ancient Egyptians don’t let the inventor himself decide on issues of public safety and public utility. This part of the myth is usually skipped because governance is nowhere near as sexy as debating the possible deskilling effects of a new technology. Carlo’s piece nicely illustrates this, all the hard social and political challenges get solved magically in it.2

What is most notable about Ammon in his dealing with Theuth is that he focuses on two features of an invention before he gives permission to circulate it: first, its practical use. And here he is quite clearly guided by the inventor’s views. And, second, the potential harms (or, let’s be cheeky, inductive risks) involved. And notably, when it comes to foreseeing harms Ammon’s own views are more decisive than any inventors not even a God. And the reason for this is explicitly spelled out: would be inventors tend to misperceive the true nature of their inventions, primarily seeing the benefits only. I return to this below.

Obviously, Socrates is simplifying the division of labor a bit by using Ammon. Most kings may (if they know their Hobbes) wish to overawe all the citizens in a godlike fashion, but they will not be in the same epistemically advantageous decision situation as (let’s stipulate) Ammon. So, how they will judge the possible harms of new technology will be, in fact, a rather challenging governance question. And it is by no means obvious that one can foresee the harms antecedently, or that one will evaluate them with the same measures after the fact. If the technology transforms the users’ or community’s identity this becomes a serious issue. Below I suggest that Socrates points toward this possibility.

My point here is not that Socrates presents us with the right informal governance model. But importantly, he presents such a model that will allow us to think about the various considerations involved. In fact, lurking here is a more important point: the existence of art of governance is itself the effect of the invention of the very writing-technology that we are discussing here. For writing makes bureaucracy possible. (It is helpful for my interpretation that Theuth is the God of scribes.) And this raises new problems, as Plato’s Socrates alerts us.

Conveniently enough (for my new debate with Carlo), the harms Ammon is alert to in the invention of writing do involve, alongside the acquisition of a new skill (writing), some de-skilling, including the art of memory. Now, in light of the contemporary debate, one might have thought that Ammon would have made a simple cost-benefit analysis to the individual or to the population of the gaining of one skill (writing) and the loss of another (worse memory). Is writing more efficient than relying on memory? Could we create performance standards? (etc.)

But what’s interesting about Ammon’s stance is that he diagnoses an externality, actually two externalities, that are especially important to a King-Ruler. And the externality has everything to do, I think, with the fact that writing is the tool of bureaucracy. As a close reader of Plato, Rousseau (recall here), reminds us (in the Third Discourse) bureaucracies are invented when the domain to be managed cannot be surveilled by single person or family. A bureaucracy is a division of labor that is spatially and temporarily extended, and relies on proper record-keeping to manage socially complex affairs. It is introduced because a ruler cannot surveil his/her domain all at once.

But this bureaucratic managing also introduces a useful fiction: that the recorded ‘data’ convey the facts, and that the person reading the report on the data understands what they signify (up the bureaucratic chain of command). Writing makes a form of impersonal acquaintance possible. But the risk that Ammon (correctly) identifies is that the bureaucrats and those they serve (governors like Ammon himself) will “read many things without instruction and will therefore seem to know many things, when they are for the most part ignorant.” The reader of the report will end up with knowledge of the reported reality, but not (without further instruction) with its connection to the underlying facts. And if they forget this distinction, as the written world takes on its own reality, they will seem wise when they may not be so. How to manage a bureaucracy that systematically generates this kind of vulnerability, is a central problem in the art of governance. When things remain on the rails, this challenge is usually invisible to most. But when it goes off the rails — public infrastructure is not repaired, corruption goes unpunished, etc. — it is often too late for minor fixes (not the least in circumstances of emergency that Carlo acknowledges).

The written word, when used as a technology of governance, inevitably creates legibility (of the population and society) and simultaneously all kinds of new opacities. As I have noted before (recall here), this is a design feature of modern LLMs, whose internal architecture and the process by way which they convey content, remains largely opaque to end-users (especially in real time). This is especially important because the error rates of LLMs remains non-trivial. This output externality is only visible if you look, expert enough, and care to notice.

I could stop here. But I promised a second externality as noted by Ammon. And this is a more important social externality: the users of a new technology will feel more confident. And, let’s stipulate, their confidence is not misplaced. Like Carlo’s ‘off-loaders’ they get more important things done in less time. Unfortunately, human nature being what it is, it will encourage boasting and self-aggrandizement. And, even if this self-satisfaction were fully merited, as Ammon recognizes this is a dynamic that predictably generates social instability. Not the least because off-loading boasters refuse to be bothered by the real challenges new technology introduces. The ethos of a polity may change for the worse with new technology—and it doesn’t take much imagination to discern that Ammon is worried about new idols and new modes. This may be inevitable, and some communities are willing to pay the price.

Now, Carlo treats the university as a site of skill-transference, and in which Claude is one tool of many. And this is no surprise, because if you return to the press release of Alivisatos’ statement, you will see that the key word in it is ‘tool’ which is repeated over and over. Tools are used. And that’s how President Alivisatos sells his message.

But he doesn’t convey at all that he has evaluated the risks (a word he does not use which is notable for a modern executive) and externalities a new tool predictably generates for the ‘community’ (his word) that he has been charged to lead. All he notes are the “astonishing and inspiring ways” of the new tool. He sounds more like enthusiastic Theuth than prudent Ammon. Hence my complaint that he is vice-signaling.

In fairness, to Alivisatos (who was an undergraduate at the UofC): when you read the full press release, he is setting up a quite elaborate governance structure for new AI. So, it is quite possible that he has assimilated the first lesson I wished to derive from Socrates’ myth. But I don’t think a fair reading of his text suggests he is signaling concern over or apt care about what might be lost in the process (which is what motivated my original lament).

Even so, I used the language of governance throughout this post, because I was most struck by his message to his students:

To our students: Many of you already are reflecting on how a skeptical, ethical, and ambitious approach to AI applies to your education. Your faculty and instructors are developing practices in the classroom within a philosophical framework to support you in opening and sharpening your mind during your journey here. That framework will prepare you well to meet the opportunities of a changing world. You can understand more about how your faculty are thinking about helping you to learn by reading about a recent series of workshops sponsored by the Center for Teaching and Learning. Policies vary by course, and you are required to follow them closely; but the sum across your full curriculum will give you diverse learning experiences with and without AI assistance. Instructors at all levels are navigating a fast-evolving landscape, and the University has a duty of care to ensure that the education offered to you is responsive to these technological developments by teaching you how to think with machines, how to think without them, and how to think about them.

There are a lot of action words in this paragraph (e.g., reflecting, developing, reading, navigating, and teaching not to least how to think) and a few technocratic terms (e.g., framework, opportunities, policies, diverse learning experiences, landscape)), including a few commands (required). What’s notably absent in this message are reading, interpretation, and (most notably) discussion. That’s not just a matter of vocabulary. Something has gone off the rails here, and by this I don’t mean the total absence of reference to the world of books and intellectual culture in his message.

Back in the day (and I amplified with some twists), Agnes Callard argued that “A university is a place devoted to the problem of how to make serious use of free time.” It would be wrong to suggest that the proper use of intellectual tools has no place in this vision of the academy. But as Callard notes such a university presupposes what she calls ‘inquisitive leadership.’ President Alivisatos’ rhetoric is very far from it. He leads not by teaching and learning here, but by cheerleading and commanding.3 And this is why I called it, vice-signaling.

• An earlier version of this post was first published at D&I.