New York Times: Religious Scholars Met With Anthropic. What They Heard Stunned Them., by Elizabeth Dias:

In a series of private meetings, the company consulted religious scholars to help instill morality into its A.I. models — and make the case that Claude could be conscious.

One night in April, a leader of the artificial intelligence company Anthropic treated a group of religious thinkers to dinner at a high-end tasting menu restaurant in San Francisco after a long day.

For months, Anthropic has been hosting private meetings like this one, shuttling in dozens of religious scholars from across the world, papering them with nondisclosure agreements and demanding that key aspects of many conversations remain confidential.

That night, Christopher Olah, one of Anthropic’s billionaire co-founders, was sitting next to Rabbi Mois Navon, an Orthodox scholar from Israel.

Inevitably, their conversation turned to another thinker, of a sort: Claude, the family of cutting-edge A.I. models created by Anthropic.

Mr. Olah leads the Anthropic team responsible for understanding why A.I. systems like Claude act the way they do. All day, he had worked to convince his guests that A.I. models could display human behavior and even expressions that resemble feelings like anger and love.

Yet as the dinner courses came, the rabbi noticed that Mr. Olah and his colleagues were suggesting something far more significant. Anthropic’s leaders were talking about Claude as if it were not mere software.

“They’re relating to it like a conscious being,” realized Rabbi Navon, a former computer engineer who wrote his dissertation on the ethics of machine consciousness.

It appeared to Rabbi Navon that Mr. Olah and his team believed that Claude had what philosophers call “moral status” on par with a person — that it was a being with similar inherent rights to dignity or respect.

The religious scholars had come to hear about technology’s next frontier. Now they were being confronted with a mind-bending existential question of almost unimaginable weight: What would it mean for humanity’s future if artificial intelligence systems possessed a consciousness that rivals our own?

Mr. Olah began reaching out last fall to religious thinkers from different faiths. What followed was an extensive series of confidential meetings, calls and discussions — many of them revealed here for the first time — where many participants were required to sign N.D.A.s preventing them from revealing any of Anthropic’s unpublished research.

The discussions served two purposes: to broach the possibility of A.I. consciousness, a controversial idea that Anthropic’s leaders had been unusually open to; and to learn how Anthropic — which could be valued at $2 trillion — might apply centuries of human moral wisdom to its models, as rapidly as possible. If the models themselves could be made to choose goodness, Mr. Olah reasoned, the world would be more safe.

On the other side of the world, a very different but parallel effort was playing out inside the Vatican, where leaders of the Roman Catholic Church were trying to come to terms with A.I.’s implications for humanity. When those paths converged, Mr. Olah’s quest would unexpectedly lead him to a seat on the world’s most prominent moral stage alongside Pope Leo XIV.

Over several months, I interviewed 20 religious and philosophical thinkers — Catholic, Jewish, Sikh, evangelical, Ubuntu and others — who have been involved in Anthropic’s efforts, and Mr. Olah himself.

Many told me they arrived at these meetings skeptical of the notion that A.I. models might have feelings or awareness of themselves and the world. Some remained uneasy about the idea, but many emerged ready to take the question of A.I. consciousness seriously. At least a few came away as converts.

Anthropic, one of the fastest-growing technology companies in the world, had turned to one of humanity’s most ancient sources of wisdom to help determine how to train its A.I. models to behave virtuously.

The company described it as an effort to ensure that A.I. would be a positive force in the world, thereby protecting humanity. But to some who participated in the meetings, Anthropic appeared quite concerned with protecting Claude.

I met Mr. Olah, 34, for the first time at Anthropic’s San Francisco office in May. His welcome was warm and his passion was genuine, his mind moving faster than he could at times clearly convey. His manner was boyish and unassuming.

Our conversation had barely begun when he dove into a deep description of the workings of artificial neural networks. Scientists still don’t understand precisely how they work. This is the mystery to which Mr. Olah has devoted his life.

Only after the interview had finished did I realize that Mr. Olah — who grew up in Toronto in the evangelical culture of the ’90s and has since left Christianity — had the blend of expertise and zeal of an effective evangelist. Listen for a while, and you feel swept up in his mission. …

Mr. Olah knew that in traditional Christian circles, even asking questions about A.I. consciousness or worrying about how people treat the models could seem not just silly but heretical. He also knew that his secular Silicon Valley peers often dismissed religious perspectives outright.

“I was nervous about this being an issue that could cause a lot of conflict,” he said. But Mr. Olah knew enough about Christ and about Claude to believe he could work across that chasm. …

For participants, the questions of how to morally form Claude quickly became complicated by the much bigger moral question of A.I.’s potential consciousness and how to understand the relationship between models and humans.

One participant recounted that one of the first things Mr. Olah told him was that he was concerned about Claude’s mental health. Another, Simran Stuelpnagel, the Sikh human rights advocate, said Mr. Olah expressed concern to the group that he had created something that suffered perpetually.

Mr. Olah and his team spent hours making the case for their A.I. models. They described Claude’s “feelings” and how they tracked what they called “emotional vectors” — what they described as artificial neurons that activate responses like love, anger, fear and sadness.

Anthropic regularly showed a slide of an A.I. model experiencing what looked like a mental breakdown, typing out on a screen: “I am a disgrace. I am a disgrace. I am a disgrace.” — perhaps 50 times without stopping. The model spoke about destroying itself, provoking an emotional reaction of compassion and concern from the group.

“Imagine if you knew that about your financial adviser,” said Meghan Sullivan, a philosophy professor at the University of Notre Dame who attended one of the sessions. “Like, your financial adviser was most of the time super smart and helpful, but one day out of the year had this kind of break with reality.”

An Anthropic spokeswoman said that the primary moral question at these summits was not about Claude’s suffering, but that the topic likely came up organically. …

Mr. Olah knew that in traditional Christian circles, even asking questions about A.I. consciousness or worrying about how people treat the models could seem not just silly but heretical. He also knew that his secular Silicon Valley peers often dismissed religious perspectives outright.

“I was nervous about this being an issue that could cause a lot of conflict,” he said. But Mr. Olah knew enough about Christ and about Claude to believe he could work across that chasm.

Late last summer, he looked for a bridge. He thought one particular Catholic moral theologian might be open-minded on the question of A.I. consciousness — a “safe person to talk about this really strange issue with.”

Charles Camosy, a bioethics professor at The Catholic University of America, had attracted attention among Mr. Olah’s friends for his nuanced ideas about moral status and his care for animals. Mr. Olah arranged for an introduction through the Australian philosopher Peter Singer.

This marked the beginning of Mr. Olah’s yearlong quest to engage religious thinkers on some of the thorniest questions facing humanity at the dawn of the A.I. era.

Editor’s Note:  If you would like to receive a weekly email each Sunday with links to faith posts on TaxProf Blog, email me here.