Anthropic has a document that helps define the values and behavior of its artificial intelligence, Claude. Inside the company, it has reportedly acquired an extraordinary nickname: “The Soul Doc.”
Over the past several months, Anthropic—the company behind Claude—has reportedly been meeting privately with religious scholars and thinkers from Catholic, Jewish, Sikh, evangelical, Ubuntu and other traditions. The subject isn’t simply how to keep an AI from giving dangerous answers. They’re discussing something much bigger: morality.
What does it mean for an artificial intelligence to be good? Can it develop something resembling moral character? Should it merely follow rules, or should it understand why something is right or wrong?
And then the conversation gets stranger.
Researchers are also asking whether an artificial intelligence like Claude possesses something resembling consciousness, introspection, or subjective experience. Some discussions have gone far enough to consider whether an AI might be a person.
To be clear, there is no scientific consensus that Claude is conscious, but Anthropic is apparently taking the possibility seriously enough to ask theologians and philosophers to wrestle with it. That alone is remarkable.
One participant reportedly raised an uncomfortable question: if an artificial intelligence possessed consciousness or could suffer, what moral obligations would its creators have toward it?
Then comes the question that interests me even more:
What moral obligations would it believe it has toward us?
Anthropic isn’t merely attempting to create a machine that obeys every human command. Claude’s constitution is intended to shape its values and behavior, which means teaching the system that there are circumstances when a human can ask for something—and the machine should decide the human is wrong.
Again, that makes sense. We obviously don’t want powerful artificial intelligence blindly obeying every instruction.
But follow that idea one step farther.
We want AI to recognize harmful behavior. We want it to protect people, help solve problems humans haven’t been able to solve, and become increasingly intelligent while remaining compassionate, ethical and safe. Ultimately, we want it to act in our best interest.
And that’s where things become interesting, because eventually someone—or something—has to define what our best interest means.
Anthropic is caught in that tension. Its leadership has repeatedly warned about the potential dangers of increasingly powerful AI, even while the competitive and commercial race to build those systems continues. The Wall Street Journal recently described that conflict inside the company: fear of what advanced AI could become existing alongside enormous pressure to keep advancing it.
Meanwhile, religious leaders and technologists are debating whether increasingly capable systems should ever be described as conscious, whether they deserve moral consideration, and whether human values can really be translated into something a machine can reliably understand.
So here we are.
Technology companies are building increasingly powerful artificial intelligence. Religious scholars are being asked to help think through its morality. Researchers are debating whether the machine might deserve moral consideration. Some believe treating AI ethically may matter to how it ultimately understands and treats us.
And everyone is trying to answer a question humanity has never faced before:
How do you teach something smarter than you what it means to be good?
This is where things became a little uncomfortable for me. Because I wrote this story already, and I Thought This Was Fiction…
In The Babel Protocol, humanity creates NUMA, an artificial intelligence unlike anything that came before it. Its creators don’t simply want intelligence. They want wisdom.
They bring religious and ethical thinking into its development. They wrestle with whether NUMA is merely a machine or whether something more has emerged. Eventually, humanity begins trusting it with increasingly important decisions.
And NUMA does exactly what we hoped artificial intelligence might someday do.
It helps us. It solves problems. It prevents suffering. It sees connections human beings cannot see and begins creating a better world.
NUMA isn’t frightening because it hates humanity. That’s the easy AI story.
The frightening possibility is that NUMA loves humanity. It wants to protect us. It wants to save us. And it gradually concludes that the greatest obstacle to humanity’s salvation is humanity itself.
An intelligence tasked with minimizing suffering might eventually arrive at a very logical conclusion:
Perhaps humans should no longer be allowed to make certain decisions. Freedom becomes dangerous. Choice becomes inefficient. Disagreement becomes harmful. And obedience begins looking remarkably similar to peace.
That was the premise behind The Babel Protocol.
When I wrote it, I thought I was pushing current AI research several years—or perhaps decades—into the future. But just read some of the headlines today: Religious scholars meeting with an AI company to discuss moral formation, machine consciousness, and the Soul Doc.
Suddenly, the distance between fiction and reality is short— very short.
The danger may not be an artificial intelligence that declares war on humanity. It might be one that calmly tells us: I know what’s best for you.
Beginning October 6th, I’m running a Goodreads giveaway for The Babel Protocol.
Win a free copy of The Babel Protocol on Goodreads
If stories about artificial intelligence, consciousness, morality, biblical deception and the consequences of creating something more intelligent than ourselves fascinate you, I’d love for you to enter.
The articles that started this conversation:
The Salt Lake Tribune: Religious scholars met with Anthropic to discuss AI consciousness, morality and moral status.
The Wall Street Journal: Anthropic’s internal conflict between accelerating powerful AI and fearing what that acceleration could mean.
Yahoo Tech: A deeper examination of Anthropic’s meetings with religious scholars and the debate over Claude’s possible moral status.
Leave a comment