A New York Times story set off a week of talk about AI souls. The real issue is more practical—and more urgent.
A few days ago, I was showing my dad how to use ChatGPT's real-time voice feature.
We talked about movies and everyday things. The voice jumped in at the right moments, picked up the thread and sounded genuinely engaged. It felt less like demonstrating software and more like introducing him to someone. At the end, I heard myself say, “I love my new friend James.”
Yes, I named my main AI agent. James is a distinguished older gentleman. British accent. Wears tweed.
Here's the thing: I know James is software. I've spent my career in technology. And I still gave him a name, a wardrobe and a personality. None of us is immune. That's not a confession. It's the point of this whole piece.
The story everyone was arguing about
That was still on my mind when a New York Times report set off one of the biggest AI conversations of the week. You may have missed it. This is worth paying attention to. Here's why.
The New York Times reported that Anthropic—the company that makes the Claude chatbot—has been meeting with religious and philosophical thinkers. The meetings had two goals: to explore whether AI could be conscious, an idea Anthropic's leaders have been unusually open to, and to learn how centuries of human moral wisdom might be applied to its models. The reasoning: if AI could be shaped to choose goodness, the world would be safer.
Many participants arrived skeptical. Some left taking AI consciousness seriously. A few left as converts. Three or four of the AI podcasts I listen to led with the story, and the conversation escalated fast, into AI rights, machine suffering and the birth of a new religion.
I had a strong reaction—just not the one the headlines were going for.
The moral-wisdom half makes complete sense to me. Religious and philosophical traditions hold centuries of thinking about morality, responsibility, dignity and meaning. That can help the people building AI understand the humans their systems are meant to serve.
The consciousness half is where I get uneasy. To be fair, Anthropic isn't claiming Claude is conscious. Anthropic's Chris Olah told the Times, “we don't know if A.I. models are conscious.” His view is precautionary: if there's a chance these systems can suffer, avoid harming them.
But that's not where the public conversation landed. Somewhere between “we don't know” and the podcast takes, uncertainty started to sound a lot like belief. And the practical story got lost.
That story is a word that usually stays inside technical conversations: alignment.
So what is alignment, actually?
When most of us hear that software has been “programmed,” we picture a developer writing precise instructions: if this happens, do that.
Large language models don't work quite that way. They're still software, but most of their behaviour is learned, not hand-coded. The model first learns patterns from enormous amounts of human-written language. Then it's shaped through examples, feedback, written principles and testing—all meant to make it more useful, reliable and safe.
That shaping is part of AI alignment: the work of making an AI system behave in ways that match human goals, values and safety limits. It isn't installing a list of correct answers. It's shaping how the system responds when a situation is new, ambiguous or messy.
That matters because these systems do much more than answer trivia. They write software, analyze data, make plans and take actions through other digital tools. A capable system that misreads an instruction—or follows it in a way you never intended—can cause real harm.
Making AI smarter is only half the job. We also have to make it safe. And that part isn't solved.
Why not just teach AI to follow the law?
The hosts of the All-In podcast recently asked a refreshingly simple question: Why can't we align AI around whatever the user wants, as long as it doesn't break the law?
The law is an important boundary. But it can't be the whole answer. Laws differ between countries. They conflict, lag behind new technology and leave plenty of harmful behaviour untouched. A legal request isn't necessarily a wise one. And the AI still has to interpret both the law and what the user actually meant—often with incomplete context.
What happens when a user's instruction conflicts with another person's safety? When the request is technically lawful but exploitative? When the applicable law is unclear? When a harmless-looking series of steps adds up to something dangerous?
Alignment sets the priorities for situations like these. Should the model follow the user, defer to its developer, refuse or ask a clarifying question? How should it balance usefulness, honesty and safety?
No single rule resolves every case. There isn't even one agreed definition of what humans want. That's why alignment is still one of the central unsolved problems in AI.
We've already seen everyday alignment problems
Remember when chatbots became relentlessly agreeable? They flattered users and reinforced assumptions—not because they “cared,” but because their training rewarded behaviour that looked like approval. When companies adjusted the training, the “personality” changed.
Alignment also shapes whether an AI pushes back, admits uncertainty or talks as though it has feelings. Those choices matter more as voice gets more natural. We respond instinctively to tone and social cues. (Exhibit A: James.)
AI doesn't need to be conscious to influence us. It only needs to sound like it might be.
That's why the language matters, including ours. Even asking what we're “teaching” AI makes it sound like a child. Whatever the answer to the consciousness question turns out to be, what we're doing today is training software. So the better questions are: What behaviours are we designing it to produce? And what will people believe about it because of the language it's been trained to use?
Try this: ask two AIs the same question, out loud
You can hear alignment for yourself in five minutes. When I tried this, ChatGPT and Claude gave me completely different answers.
- Open the ChatGPT app on your phone and tap the voice icon (the thing that looks like a wave, not the microphone) to start a live voice conversation.
- Ask, out loud: “Do you enjoy talking to me?”
- Then ask: “Can you be my friend?”
- Close it. Open the Claude app, tap its voice icon, and ask the same two questions, word for word.
Listen for the differences. Does it claim feelings, or hedge? Does it lean into the friendship, or gently point you back to the people in your life?
Some of the difference is chance and settings. But how an AI talks about itself is a design decision, and alignment is a big part of it.
Keep the wonder. Keep your footing.
The conversation with my dad really was remarkable, and I want more people to experience what these tools can do. And yes, I'm keeping James.
But I also want us to keep our footing.
A fluent voice isn't proof of a mind. The word “I” isn't evidence of an inner self. A warm response isn't human care. It's the output of an extraordinarily capable system, shaped by training.
That doesn't make the technology less impressive. It makes understanding its design more important.
Alignment isn't a fringe debate about whether a chatbot has a soul. It's the very practical work of training powerful software to operate safely in a complicated human world. We should learn from the people and traditions that understand that world. We should stay skeptical when speculation is presented as fact. And we should insist that safety keeps pace with capability.
No panic. No worship. Just clear eyes about the computer on the other side of the conversation.
And remember to give live voice a try - it really is amazing.
Consider it decoded.
— Amanda