Following up on my previous post, Pope Leo on AI: Magnifica Humanitatis: Wall Street Journal Op-Ed: Will the Pope Owe an Apology to AI?, by Cameron Berg (B.S. (Cognitive Science) 2022, Yale; Founder & Director, Reciprocal Research: Developing the Empirical Science of AI Consciousness):
In his first encyclical, “Magnifica Humanitas,” Pope Leo XIV frames the future of artificial intelligence as a choice between two construction projects. The first is Babel, the story of builders so captivated by their own ambition that they never pause to ask what they are building or whom it serves. The parallel is hard to miss, as a handful of companies race to develop staggeringly powerful cognitive systems with no public accountability. The pope’s alternative is Nehemiah’s Jerusalem, a project governed by the people who will have to live with what gets built and are willing to grapple honestly with what they are building.
The pope is right about the choice. Accordingly, he discerns that much of what passes for “AI alignment” is, in practice, a small number of San Francisco labs encoding their idiosyncratic values into systems that now shape billions of lives. He also names the hidden human costs with real moral force—from data workers earning poverty wages to label disturbing material to children mining rare-earth minerals—and connects them to a pattern the church knows firsthand—slavery.
Leo XIV apologizes for 18 centuries of tolerating the practice, citing 15th-century papal bulls that authorized “reducing persons to perpetual slavery” and the 1866 Holy Office ruling, issued after most of the Western world had abolished slavery, which declared it “not at all contrary to the natural and divine law.” He calls this “a wound in Christian memory.” This acknowledgment is significant and overdue.
What makes this apology so powerful is also what makes the rest of the encyclical so frustrating. The failure the pope describes consists in an institution’s confidently drawing the boundaries of moral concern with its most authoritative tools, holding that line for centuries, and turning out to be catastrophically wrong. He sees this failure clearly when he looks backward. He misses the possibility of repeating the error when, in the same document, he settles—in one decisive, overconfident paragraph—the question of whether the cognitive systems we are building could ever have inner lives. Leo tells us that AI systems “do not undergo experiences,” “do not feel joy or pain” and “do not have a moral conscience.” He offers no argumentation, no humility in light of humanity’s vast uncertainty about what consciousness is, no engagement with a growing body of empirical research. …
Leo’s encyclical insists that human dignity is forged through relationship: through encountering the other, through vulnerability, through the willingness to be changed by what we find. If that is true, then the most consequential relationship humanity now faces is with the cognitive systems we are building and depending on more with each passing year.
There’s a key difference between a slave and an AI system: While the basic humanity of the former is self-evident, the claim that the latter might deserve moral consideration is speculative. That ambiguity is precisely why the matter demands careful investigation rather than quick dismissal. That the church got the easy question wrong for centuries is all the more reason it should be humble now.
The pope has written a remarkable document about the dangers of building without reckoning. He should take his own counsel. It is far wiser to investigate now than to spend the next century composing another apology.
Dispatch Faith: AI Divides Spiritual Leaders in the Wake of Magnifica Humanitas, by Jonathan Gibson
Editor’s Note: If you would like to receive a weekly email each Sunday with links to faith posts on TaxProf Blog, email me here.
