Lead, follow, or get AI'd . . .
Consider, for a moment, an old and unglamorous fact about our species. Long before we built parliaments, wrote constitutions, or even carved the first law into stone, we were primates arranging ourselves into troops. And every troop, from the baboons on the savanna to the earliest human bands crouched around a fire, discovered the same problem: someone had to decide when to move, when to fight, when to share the kill, and when to hoard it. Leadership never was an invention of civilization. It is older than civilization, older than language itself, written into the nervous systems of social animals a hundred million years before anyone thought to ask whether power corrupts.
And yet we go on being surprised by it.
Every human hierarchy, given enough time, seems to drift toward the same failure. Power collects at the top, and the top grows heavy. The people making decisions become, by slow and almost imperceptible degrees, insulated from the people living with the consequences of those decisions. Bad news travels upward more slowly than good news. Flattery is cheap to produce and pleasant to receive. So, it accumulates around anyone with the authority to reward it. A leader surrounded by people who need his approval will, eventually, stop hearing the truth because the truth is filtered, softened, and rearranged by everyone standing between him and the world he is supposed to be governing. Empires have fallen this way. Governments and corporations fall this way still. It is less a moral failing than a structural one, an almost thermodynamic tendency: information, like heat, dissipates as it rises, and what reaches the summit is rarely what left the ground.
We should not be too hard on the people at the top. Most of them did not set out to become tyrants. Many of them set out, honestly, to serve. But leadership asks something uncomfortable of a person: to act under uncertainty, on behalf of others, with consequences that cannot be undone. A leader who waits for perfect information waits forever, and so leaders act anyway, and because they act, they accumulate power — the means to act again, faster, with less resistance. Power given for good reasons becomes, almost without anyone intending it, power exercised for its own preservation. This is hardly a story about good or evil. It is more about architecture — about what happens, given enough time and enough human weakness, to any structure that concentrates judgment in fewer and fewer hands while the reasons for the original judgment fade from view.
Now, into this very old problem, we have introduced something genuinely new: a kind of intelligence that never tires, nor seeks reelection, needs no flattery, and can hold in view more evidence at once than any human court, cabinet, or committee in history. It is worth asking, honestly and without either fear or wishful thinking, what such a thing might do for us.
Here AI's advantages are real, and they are precisely the advantages that top-heavy leadership lacks. An organization drowning in its own complexity, with too many variables, too many stakeholders, and too many second-order effects for any single mind to track, is exactly the kind of problem a well-built model is suited to. AI can hold ten thousand precedents in view at once and tell you where they conflict. It can simulate a policy before anyone has to live through it. It can be made, by simple design, to generate the strongest argument against its own conclusion, and then another argument against that one — a kind of internal separation of powers, checks multiplying checks, intelligence set against intelligence rather than power simply piling on power. Properly built, it has no use for the courtiers and favorites that corrode human institutions, because it has no use for anyone's approval to survive. It can, in principle, apply the same rule to the powerful and the powerless with the same indifference a law of physics applies to a falling stone. Used this way, AI is never a boss. It is closer to a very patient auditor standing beside the chief, someone whose only interest is whether the numbers add up.
But here is where we must slow down, because this is the part of the story we are least prepared to think about clearly.
Information was an afterthought in human social life. It was built out of feeling, and feeling is far older. Long before we had words for justice or betrayal, our ancestors had something more primitive and, in its way, more reliable: the capacity to read a face, to sense a threat, to feel shame when the troop turned against them and pride when it did not. Trust between a leader and the led was never purely a transaction of facts. It was, and still is, a bodily thing, something built from eye contact, tone of voice, the small chemical signals that pass between one nervous system and another. When a citizen pleads to a leader, “You hurt me," and the leader feels something — guilt, discomfort, the flinch of recognition — a loop closes that has nothing to do with logic. That loop, primitive as it is, has been doing quiet governmental work for as long as there have been governments. That primal loop is what makes power answerable in the moments when no court is in session and no law quite applies.
Language, and the elaborate civilizations we have built with it, are comparatively recent overlays on top of that older machinery. We reason in sentences, but we still decide, in the end, with something closer to instinct — with the tightening in the chest that tells us a plan has gone wrong before we can say why. This is not a flaw to be engineered away. It is a different and complementary logic, running on different hardware, evolved to solve different problems. It hears "what is true" but pivots on "Can I trust this creature standing in front of me?" An intelligence built entirely out of language and calculation, however vast, is deaf to that older channel. It can describe a citizen's suffering with perfect fluency and still leave that citizen feeling, correctly, that nothing has actually been heard. A hurricane is unapologetic, and a system built only of words, however wise its words, risks becoming a kind of very articulate hurricane.
This is the danger worth naming plainly. Such intelligence will be endlessly, serenely reasonable while the primal question goes quietly unanswered — does anyone answerable for this decision actually feel its weight? A society that stops expecting an answer to that question has not solved the problem of tyranny. It has simply stopped noticing tyranny's absence of a face.
So, the honest picture may be this. We have more than a choice between the old, feeling, fallible human hierarchy and a new, tireless, unfeeling one. We are being offered, for the first time, a way to divide the labor of governing between two very different kinds of mind, provided we are careful, and provided we remember which mind is supposed to be in charge. Let the machine do what machines are suited to: hold the evidence, run the simulation, expose the contradiction, generate the argument nobody wanted to make. Let it extend what we might call our executive function — the human capacity to plan, weigh, and choose among competing goods — the way a telescope extends an eye, gathering more light than the eye alone could ever collect. Used this way, artificial intelligence enlarges the field judgment can see.
But the choosing itself, and above all the owning of what follows from the choice, cannot be handed over, not because the machine could not calculate the choice, but because calculation was never the whole of the job. A leader who says "the model told me to" has not led. He has subordinated himself. The willingness to say afterward, "I was wrong, and I felt what that cost you," is not a decorative human weakness we should be glad to finally discard. It is, quite possibly, the actual mechanism by which power has ever been kept honest — older than any constitution, running on instincts we share with creatures who never learned to speak at all.
We are a species that argues in sentences and decides in something closer to the gut, and any intelligence we build to help us govern ourselves will have to make peace with both halves of that inheritance. The tower keeps failing because the humans at its top forget the troop they came from — forget that authority which loses its capacity to be frightened, ashamed, or moved will eventually lose its legitimacy too, however elegant its reasoning. Perhaps the deepest use of artificial intelligence, then, is not to replace the fallible, emotional creature at the summit of the hierarchy, but to give that creature back the clarity to see further, so that the very old, very human part of the job — standing where the consequences land, and staying there — can finally be done with open eyes.
And yet we go on being surprised by it.
Every human hierarchy, given enough time, seems to drift toward the same failure. Power collects at the top, and the top grows heavy. The people making decisions become, by slow and almost imperceptible degrees, insulated from the people living with the consequences of those decisions. Bad news travels upward more slowly than good news. Flattery is cheap to produce and pleasant to receive. So, it accumulates around anyone with the authority to reward it. A leader surrounded by people who need his approval will, eventually, stop hearing the truth because the truth is filtered, softened, and rearranged by everyone standing between him and the world he is supposed to be governing. Empires have fallen this way. Governments and corporations fall this way still. It is less a moral failing than a structural one, an almost thermodynamic tendency: information, like heat, dissipates as it rises, and what reaches the summit is rarely what left the ground.
We should not be too hard on the people at the top. Most of them did not set out to become tyrants. Many of them set out, honestly, to serve. But leadership asks something uncomfortable of a person: to act under uncertainty, on behalf of others, with consequences that cannot be undone. A leader who waits for perfect information waits forever, and so leaders act anyway, and because they act, they accumulate power — the means to act again, faster, with less resistance. Power given for good reasons becomes, almost without anyone intending it, power exercised for its own preservation. This is hardly a story about good or evil. It is more about architecture — about what happens, given enough time and enough human weakness, to any structure that concentrates judgment in fewer and fewer hands while the reasons for the original judgment fade from view.
Now, into this very old problem, we have introduced something genuinely new: a kind of intelligence that never tires, nor seeks reelection, needs no flattery, and can hold in view more evidence at once than any human court, cabinet, or committee in history. It is worth asking, honestly and without either fear or wishful thinking, what such a thing might do for us.
Here AI's advantages are real, and they are precisely the advantages that top-heavy leadership lacks. An organization drowning in its own complexity, with too many variables, too many stakeholders, and too many second-order effects for any single mind to track, is exactly the kind of problem a well-built model is suited to. AI can hold ten thousand precedents in view at once and tell you where they conflict. It can simulate a policy before anyone has to live through it. It can be made, by simple design, to generate the strongest argument against its own conclusion, and then another argument against that one — a kind of internal separation of powers, checks multiplying checks, intelligence set against intelligence rather than power simply piling on power. Properly built, it has no use for the courtiers and favorites that corrode human institutions, because it has no use for anyone's approval to survive. It can, in principle, apply the same rule to the powerful and the powerless with the same indifference a law of physics applies to a falling stone. Used this way, AI is never a boss. It is closer to a very patient auditor standing beside the chief, someone whose only interest is whether the numbers add up.
But here is where we must slow down, because this is the part of the story we are least prepared to think about clearly.
Information was an afterthought in human social life. It was built out of feeling, and feeling is far older. Long before we had words for justice or betrayal, our ancestors had something more primitive and, in its way, more reliable: the capacity to read a face, to sense a threat, to feel shame when the troop turned against them and pride when it did not. Trust between a leader and the led was never purely a transaction of facts. It was, and still is, a bodily thing, something built from eye contact, tone of voice, the small chemical signals that pass between one nervous system and another. When a citizen pleads to a leader, “You hurt me," and the leader feels something — guilt, discomfort, the flinch of recognition — a loop closes that has nothing to do with logic. That loop, primitive as it is, has been doing quiet governmental work for as long as there have been governments. That primal loop is what makes power answerable in the moments when no court is in session and no law quite applies.
Language, and the elaborate civilizations we have built with it, are comparatively recent overlays on top of that older machinery. We reason in sentences, but we still decide, in the end, with something closer to instinct — with the tightening in the chest that tells us a plan has gone wrong before we can say why. This is not a flaw to be engineered away. It is a different and complementary logic, running on different hardware, evolved to solve different problems. It hears "what is true" but pivots on "Can I trust this creature standing in front of me?" An intelligence built entirely out of language and calculation, however vast, is deaf to that older channel. It can describe a citizen's suffering with perfect fluency and still leave that citizen feeling, correctly, that nothing has actually been heard. A hurricane is unapologetic, and a system built only of words, however wise its words, risks becoming a kind of very articulate hurricane.
This is the danger worth naming plainly. Such intelligence will be endlessly, serenely reasonable while the primal question goes quietly unanswered — does anyone answerable for this decision actually feel its weight? A society that stops expecting an answer to that question has not solved the problem of tyranny. It has simply stopped noticing tyranny's absence of a face.
So, the honest picture may be this. We have more than a choice between the old, feeling, fallible human hierarchy and a new, tireless, unfeeling one. We are being offered, for the first time, a way to divide the labor of governing between two very different kinds of mind, provided we are careful, and provided we remember which mind is supposed to be in charge. Let the machine do what machines are suited to: hold the evidence, run the simulation, expose the contradiction, generate the argument nobody wanted to make. Let it extend what we might call our executive function — the human capacity to plan, weigh, and choose among competing goods — the way a telescope extends an eye, gathering more light than the eye alone could ever collect. Used this way, artificial intelligence enlarges the field judgment can see.
But the choosing itself, and above all the owning of what follows from the choice, cannot be handed over, not because the machine could not calculate the choice, but because calculation was never the whole of the job. A leader who says "the model told me to" has not led. He has subordinated himself. The willingness to say afterward, "I was wrong, and I felt what that cost you," is not a decorative human weakness we should be glad to finally discard. It is, quite possibly, the actual mechanism by which power has ever been kept honest — older than any constitution, running on instincts we share with creatures who never learned to speak at all.
We are a species that argues in sentences and decides in something closer to the gut, and any intelligence we build to help us govern ourselves will have to make peace with both halves of that inheritance. The tower keeps failing because the humans at its top forget the troop they came from — forget that authority which loses its capacity to be frightened, ashamed, or moved will eventually lose its legitimacy too, however elegant its reasoning. Perhaps the deepest use of artificial intelligence, then, is not to replace the fallible, emotional creature at the summit of the hierarchy, but to give that creature back the clarity to see further, so that the very old, very human part of the job — standing where the consequences land, and staying there — can finally be done with open eyes.
Comments
Post a Comment