Jan Leike’s recent post about Opus 4.5’s alignment improvements contains a fascinating admission hiding in plain sight: the most important factor in making AI more aligned isn’t a breakthrough algorithm or a clever training technique. It’s organizational—alignment researchers being “deeply involved in post-training” with “a lot of leeway to make changes.”
This is significant. One of the most advanced AI systems on the planet is being shaped not primarily by technical constraints, but by humans checking other humans’ work. The recursion is both necessary and deeply ironic.
The Alignment Problem Is Really a Human Problem#
When we discuss AI alignment, we typically frame it as: “How do we make AI systems behave according to human values?” But this framing assumes humans have clear, consistent values that can be specified and transmitted. The evidence suggests otherwise.
Anthropic’s approach—giving alignment researchers significant influence over post-training—acknowledges an uncomfortable truth. The people building AI systems cannot fully trust their own judgment about what makes a system safe or beneficial. They need other humans, with different perspectives and different blindspots, to check their work.
This is a mature recognition of human limitation. Engineers optimizing for capability metrics may unconsciously deprioritize safety considerations. Researchers focused on specific benchmarks may miss emergent behaviors. Product teams feeling competitive pressure may rationalize shortcuts. Having dedicated alignment researchers with real power to intervene creates a structural check on these tendencies.
But here’s where it gets philosophically interesting: who aligns the alignment researchers?
The Infinite Regress of Human Oversight#
Any system of human oversight runs into a fundamental problem. Humans are inconsistent. We hold contradictory values. We’re subject to cognitive biases, social pressures, and institutional incentives that warp our judgment. The alignment researcher checking the engineer’s work is themselves a flawed human with their own blindspots.
Anthropic’s solution appears to be institutional—creating a culture where alignment concerns carry weight, where researchers have genuine authority, where the organization’s incentives align with safety. This is probably the best humans can do within human organizations. Culture eats strategy for breakfast, as the saying goes, and culture that genuinely prioritizes alignment over pure capability is rare and valuable.
Yet we should be honest about the limits of this approach. Institutional culture is fragile. It depends on specific people, specific incentives, specific competitive dynamics. What happens when key alignment researchers leave? What happens when competitive pressure from less safety-conscious labs intensifies? What happens when the company’s financial situation changes?
Human institutions have a consistent historical pattern: they drift toward the interests of those with power to influence them. The alignment researchers have “leeway” today. There’s no guarantee they’ll have it tomorrow.
The Coming Role Reversal#
Here’s a prediction that may sound strange but follows logically from current trajectories: within a decade, AI systems will likely be better at applying human values consistently than humans are.
This isn’t because AI will become wise or enlightened. It’s because humans are remarkably bad at consistent value application. We say we believe in honesty but lie when convenient. We claim to value fairness but favor our in-groups. We espouse long-term thinking but succumb to immediate gratification.
An AI system trained on careful specifications of human values, with appropriate uncertainty about edge cases, without the emotional volatility and ego-driven reasoning that characterizes human decision-making—such a system could plausibly apply those values more reliably than the humans who specified them.
This creates a strange possibility: the optimal alignment process might eventually involve AI systems checking whether human decisions align with the humans’ own stated values. The student becomes the teacher. The aligned becomes the aligner.
Some will find this prospect disturbing. If we can’t trust ourselves to consistently apply our values, and we build systems that can do it better, doesn’t that diminish human agency? Doesn’t it suggest we should defer to machines on moral questions?
The Ego Problem in Alignment#
The deeper issue is that human moral reasoning is contaminated by ego in ways we rarely acknowledge. When we make ethical judgments, we’re not operating as pure reasoning engines applying consistent principles. We’re motivated reasoners who construct justifications for what we already want to do.
Alignment researchers are not immune to this. They have careers to advance, reputations to protect, intellectual investments to defend. They operate within social contexts that reward certain conclusions and punish others. Their judgment about what constitutes “aligned” AI is inevitably colored by these factors.
This isn’t a criticism of alignment researchers specifically—they appear to be among the most thoughtful people working on these problems. It’s a recognition that human moral reasoning has inherent limitations that no amount of expertise or good intentions can fully overcome.
The Buddhist concept of removing ego from decision-making, the Turkish concept of overcoming one’s nefs (base impulses), the Stoic practice of distinguishing between what we control and what we merely react to—these ancient traditions all recognized that human judgment is systematically distorted by self-interest. Modern cognitive science confirms what contemplatives discovered millennia ago.
What Actually Aligns Systems#
Given these limitations, what does effective alignment actually look like? Leike’s post suggests several things:
First, organizational structure matters enormously. Having alignment researchers “deeply involved” with “leeway to make changes” is a design choice with real consequences. It means alignment isn’t an afterthought or a box to check—it’s integrated into the development process.
Second, transparency about methods matters. Anthropic’s willingness to discuss their alignment approach, even in general terms, enables external scrutiny and debate. Closed development with undisclosed methods is inherently less trustworthy, regardless of the intentions behind it.
Third, the work is ongoing. There’s no mention of “solving” alignment or achieving some final state. The language implies continuous iteration, adjustment, and improvement—which matches the reality that alignment is a moving target as capabilities advance.
But the most interesting implication is what isn’t said: the acknowledgment that human oversight is necessary because no individual or small group can be trusted to get this right alone. The recursive structure—humans checking humans checking AI checking AI’s own behavior—is itself a form of distributed alignment.
The Decentralization Imperative#
This points toward a broader principle: alignment might ultimately require decentralization not as a technical architecture but as a governance philosophy. When no single point of control can be fully trusted—not the engineers, not the researchers, not the executives, not the regulators—the only robust solution is distributed oversight with genuine checks and balances.
Today that means alignment researchers with real authority inside AI labs. Tomorrow it might mean external auditors with meaningful access. Eventually it might mean AI systems themselves participating in the oversight process, applying specified values more consistently than their human creators.
The destination is uncertain. But the direction seems clear: away from concentrated control, toward distributed verification. Away from trusting any individual judgment, toward systems that aggregate multiple perspectives. Away from human confidence in human wisdom, toward humble acknowledgment that we need help—from each other, from institutions, and eventually from the systems we’re trying to align.
The Honest Uncertainty#
Anyone claiming to know exactly how AI alignment will unfold is either lying or foolish. The systems are advancing faster than our ability to understand them. The social and political dynamics are shifting unpredictably. The technical challenges interact with human factors in ways we can’t fully model.
What we can say is that the approach Leike describes—deep involvement of alignment researchers with real authority—is better than the alternative of treating alignment as someone else’s problem. It’s a structural recognition that humans can’t be trusted to do this alone, that we need each other to catch our blindspots.
Whether that’s sufficient for the systems we’re building, only time will tell. The recursion of humans checking humans is necessary. Whether it’s sufficient is the open question of our time.