We need to play close attention to the effectiveness and value of AI through close governance. AI is not some Nietzschean Superman (or some all conquering super aware system) come to save humanity or to define its future, it is simply a very powerful and persuasive digital tool. Perhaps we would be safer to think of it as a Digital Pet constantly demanding our attention and our efforts to keep it alive and to deliver value through effective governance.
Doomed us (I’m paraphrasing Nietzsche) to believe
In the rectification of the world
Through knowledge—to live
Within the limited circle of soluble problems
James Longenbach, Barcarolle
The Danger of Anthropomorphism
AI is undoubtedly powerful, leveraging the predictive capabilities of large language models and trained on most of humanity’s written history and knowledge. Again and again the tools have demonstrated the capability to engage quickly with complex problems and to be persuasive in their engagement of people (perhaps too much for a human in the loop to be effective). We are encouraged by their authors to see these models as potentially having consciousness and most definitely as the future of human work. The same model authors float disaster scenarios for human employment and society as a whole.
Many academics and thinkers have been warning that the simulation of human intelligence does not make these models intelligent or human. Kai Riemer at the University of Sydney has consistently made the point that AI is a tool and an open ended one at that. We can be lured by anthropomorphism into lowering our guard and trusting the models too much. Rather than treating these models as humans we are perhaps better to think of them as useful digital pets. We can still grow attached but at least acknowledge the work and effort involved in the use of them. Hopefully this focus, also helps us to understand the real value of human engagement and interaction that is at risk as this new technology is adopted.
Many may have missed or forgotten the craze for digital pets, as exemplified by the Tamagotchi. These were simple digital game applications sold on small fobs which enabled a user to take care and raise a digital pet. The devices were demanding requiring constant attention but also designed to build engagement with the user through rewards and the connection built to a digital pet. Without the right care and attention, the pet died and the fob became a useless piece of plastic.
The more I look into AI governance the more this process feels familiar. AI requires considerable and continuous attention: to goals, to harnesses and guardrails, to context, to access and authorities, to training, to monitoring and performance and also guarding against the surprises that come when we inadvertently attribute human traits to a digital system. We must learn to be better prompters and then we must learn to specify agentic behaviour with the exactitude of other digital engineering if we are to take comfort in consistently valuable outcomes. Often we need to rebuild things to prevent failure when models are updated or changed. We must manage our pets closely lest they die or do something entirely unintended.
Dreams of Superman
When we see AI as human, it is an easy but dangerous step to move from seeing this as a tool requiring work to some Superhuman. Part of the value of thinking of this tool is a digital pet is to avoid this hubris.
in the
fever of revelation, you will
understand why the shaman
is filled with the hubris
of creation
Kwame Davis, Horns
Whether your model of a Superhuman is an all powerful alien who wears a cape and underwear on the outside or is some vision from Nietzsche come to save us from a nihilist future by showing us the potential of humanity, AI is a poor exemplar. To begin, it is not even human though it has been well trained to lure us into thinking it is and flattering us to think it must be clever enough to see our intelligence. The tools are powerful but they have no agency of their own and their capacity to manage complexity can mask an ability to compellingly oversimplify ambiguity.
From fantasies of a society where humans are released from productivity to dreams of AI leading us to distant stars, we have let the race of AI hype run well ahead of our need to focus on value creation and sensible use cases. We have been lured by exponential rise in performance of the tool and the anthropomorphic nature of the technology to overinflate its near term potential. It is worth being reminded of Bill Gates’ advice:
“We always overestimate the change that will occur in the next two years and underestimate the change that will occur in the next ten.”
The hard work of AI adoption and governance is underway now. Organisations are learning and adapting to the real value that can be created and the challenges realising that value creates for governance. There will be a lot of work managing, tailoring and adapting these systems as the underlying models change. We have seen rapid changes at the model layer as these organisations grow their models, their revenue and the trajectory of investment and cost profiles. While the investment and debt levels are eye-watering, there are encouraging signs in the Anthropic IPO that at least in this phase value is being created for Anthropic and its customers. Similar levels of experimentation, innovation, and adaptation will be required at the layers of business process above the model
We must also continue to invest in our people and changing our organisations as the technology is adopted. Careful decision making around what should and should not be trusted to the new digital pets will be important. Retaining and developing the resources to manage these systems will be a key part of their future success. The long term possibilities of the technology will be best pursued by seeing it as a digital pet requiring work and investment from us, rather than some kind of superintelligence with a human soul.
The deeper sinks the sun, the longer grow the shadows - now shines in the Minotaur's unblinking face,
and the labyrinthine man seeks not the truth but his Ariadne.
Paul Goodman, Sentences of Nietzsche
