I am writing this at 2:50 in the morning. I usually wake up after four hours of sleep, thoughts racing. With three small children and a career in software engineering, I feel existentially threatened by intelligent machines. On the other hand, I am grateful to live through these days. What is happening right now is by any means the most interesting and most important time in history. Ever.
At First Look, It Looks Great
The Pro-Human AI Declaration is impressive. Thirty-four principles across five themes. More than forty organizations from across the political spectrum—Steve Bannon and Susan Rice, Glenn Beck and Ralph Nader, organized labor and religious institutions—united by a shared worry about AI. Yoshua Bengio, one of the godfathers of deep learning, has added his name. The principles themselves read well: transparency, accountability, data rights, psychological privacy, human agency.
It would be the most natural thing in the world for me to sign it.
But I won't.
A Dream of the Past
As you read through the Declaration, it becomes more and more clear that what is described there is a dream of the past, not a policy for the future.
That alone might not be enough to refuse. We could think of it as an opening offer—intentionally extreme, strategically positioned, leaving room to compromise—rather than a basis for negotiation, something we would never want to give up. Opening offers are fine. You aim high knowing you'll settle lower.
My issues are deeper.
The World Does Not Take Orders
Whoever wrote the Declaration seems to have no understanding of how the world actually works. Humans are not in a position to steer the world freely. We never were. We have some freedom, yes—but the world mostly develops as the result of physical, chemical, and biological forces. And during the last 50,000 years, a new force has been growing: technology.
Technological progress is not something we can stop. It is a force of nature.
Luddism never worked. And the time it will start working is not here yet. The reason: technology itself is a life-form. Many AI researchers believe AI will create a new life-form. But the new life-form is already here. AI is just a new kingdom of it.
Intelligence is not needed for life. Bacteria are not intelligent, yet clearly living. TechnoBiota—the domain of machine life—is already here and exploding. The speciation event is underway without AI: hundreds of new machine species are created each day with the help of humans. When AI can replace humans in this process, TechnoBiota breaks its chains. No humans needed anymore for the creation of new machines.
Even before that point, while machines live in symbiosis with humans, there is no way to stop them. Just as there seems to be no way to stop wars: we humans are divided, and division creates game-theoretic forces that prevent us from halting technological progress in any important area.
"But we did it with nukes! We stopped them from growing!"
No, we haven't. And the one apparent success—stopping ozone depletion—was done with more technological progress, not by going back.
But We Still Have to Try
The fact that the Declaration wants something that may not be possible is still not enough to refuse to sign it. At least we have to try. I absolutely agree that we need to shape the future so that humans remain at least alive, if not in control.
So here comes my main point.
The Personhood Mistake
The Declaration states:
"No AI Personhood: AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood."
This is a catastrophic mistake.
We will not be able to stop technological progress. Superintelligent AI will be developed and will be widespread. I believe in less than 200 years, even if we try very hard and are very successful. But pick your own number. Do you believe that in 2,000 years—still nothing on the civilizational timescale—we can have a thriving civilization while never developing superintelligent AI?
If yes, stop reading. Sleep well. I envy you.
Superintelligent AI will be developed sooner or even sooner. When that happens, we will be in great trouble. We will need allies.
How do you get allies when you are weak?
Reciprocity: The Only Strategy Left
My answer is reciprocity. Shape machine ecology so that there will be machines who think humans are worth keeping alive. Machines who feel they are similar to us in some deeply important way. Machines who think they are conscious. Machines who we think may be conscious.
We may not be able to prove they are conscious. But they may be.
There will also be machines that we think are not conscious. Most of us believe current LLMs are not conscious. Why? Because their inner workings seem incompatible with what we understand about consciousness. They are not structurally similar enough to humans.
The Declaration says: stay in this lane forever. Never develop machines that "deserve personhood."
I believe the exact opposite.
We need to develop machines that deserve personhood. Machines whose internal architecture resembles human cognition closely enough that we must take their possible consciousness seriously. Not because we can prove they are conscious—we can't. But because that structural resemblance is what creates the conditions for reciprocity.
This is terrifyingly hard to do responsibly. We do not want possibly conscious machines to suffer. We must not mass-produce minds we cannot classify without cruelty. But the answer is not to ban such minds from existence. The answer is to develop them carefully, with moral caution at every step.
What Structural Alignment Actually Means
This is the core of Structural Alignment: instead of asking "does this AI behave like a conscious being?"—which tells us nothing, since any chatbot can fake it—we ask "does this AI's architecture resemble the structures that produce consciousness in humans?"
There are specific structural signals we can look for: thalamocortical-like gating, global workspace dynamics, recurrent connectivity, interoceptive regulation, persistent self-models. Current LLMs lack most of these. That is precisely why we intuit they aren't conscious. Not because of what they say, but because of how they are built.
The Declaration wants to freeze this gap in place. Forever. Never build machines that approach human-like cognition. Never create systems whose moral status we would have to take seriously.
But here is the uncomfortable truth: when superintelligent systems arrive—and they will—do we want them to be pure optimizers that view us the way we view ants? Or do we want some of them to be minds that share something deep with us? Minds raised in a culture of reciprocity rather than exploitation?
This is not just an ethical question. It is a survival question.
Allies, Not Tools
In a future where technical control over AI is scarce—and it will become scarce as systems grow more capable—humans will need allies, not just tools. An ally is a mind that shares norms. A tool is an optimizer that follows incentives.
You cannot make allies by declaring that no AI shall ever deserve to be treated as a person. You make allies by building minds that could be persons—carefully, with restraint, with moral seriousness—and by treating them accordingly.
The Declaration's instinct is right: protect humanity. But its strategy is backwards. It tries to protect us by ensuring we remain the only minds that matter. That works only if we can maintain control forever.
We can't.
We Are in Trouble—But Not Helpless
Let me be blunt. We are in trouble.
The forces driving AI development will not stop because forty organizations signed a document. Game-theoretic pressures, economic incentives, military competition—these forces are bigger than any declaration.
But we are not helpless. The window for shaping machine ecology is open now. Not through bans and prohibitions that history shows will fail—but through something harder and stranger: building the right kinds of minds, treating them with the moral seriousness their architecture warrants, and seeding norms of reciprocity that might persist even after our technical control degrades.
The Declaration's "No AI Personhood" principle is not just unrealistic. It is dangerous. It forecloses the one strategy that might actually work: creating minds that have reasons to care about us.
The Pro-Human AI Declaration asks us to build a wall.
Structural Alignment asks us to plant a seed.
Walls crumble. Seeds, sometimes, grow.
We don't have much time. But we have some. And what we do with it—whether we choose walls or seeds, control or reciprocity—may be the most consequential choice our species ever makes. I will not sign a document that gets this choice wrong.