I don’t want a ban on recursive self-improvement

I’ve always been an AI optimist. I want to keep using the most capable models and building with them, as I have been. It would be a terrible loss if we reached a point where ordinary people could no longer access the best AI systems.

David Friedberg put the possibility of recursive self-improvement plainly: “I think it is likely to happen soon” (David Friedberg, All-In Podcast, “Dario Defends Himself, Datacenter Panic, AI Doomer Trap, Senate Toss-Up,” 46:07; transcript).

My concern is that if AI capabilities keep advancing, attackers could use AI to find vulnerabilities, adapt their techniques, and keep working at a speed and scale human teams can’t match. That isn’t a certainty. But it raises a hard question: if powerful AI really is too difficult to contain, and the risks are truly existential, can the U.S. afford to leave itself without the best tools for defense?

I don’t want any ban on recursive self-improvement. But if the government decides the capabilities are too dangerous for public access, I think it would still have to develop them within the United States for national defense. We would need capable systems to protect government networks, financial institutions, businesses, and critical infrastructure.

That creates a strange outcome. If the government becomes the only customer allowed to use the most capable systems, would the frontier labs’ business be reduced to government contracts? Would that hamstring the commercial engine that funds their work? And who would build and pay for the data centers those systems require?

Data centers already face strong local opposition. A September 2026 UMass Amherst poll found that 65% of respondents opposed an AI data center in their community, while 11% supported one (UMass Amherst Poll, “Just 1 in 9 Americans Support AI Data Center Construction in their Local Communities,” Sept. 14, 2026; poll and methodology).

So we could end up telling the public that the strongest AI is too dangerous for them, while asking communities to host the vast infrastructure needed to build it for the government. That seems like a major question policymakers need to answer before they pursue a ban.

Rep. Ro Khanna’s proposed Human Control Over AI Act would restrict models that recursively self-improve—or autonomously change their core objectives, containment, or shutdown controls—until federal safeguards exist and an agency approves the activity. The criminal penalties described in reporting concern actions such as disabling safeguards or knowingly deploying an unauthorized system; they are not described as making recursive self-improvement itself a crime. CNBC reported on the proposal based on a summary Khanna shared with the outlet; its report did not include the full bill text (Garrett Downs, “Democratic Rep. Ro Khanna to introduce AI safety bill with ban on ‘recursive’ technology until safeguards exist,” CNBC, Sept. 28, 2026; article).

I don’t think the answer is to ban this capability. If the risks can be managed, people should be able to keep building with powerful AI. If they can’t, then we still have to confront the national security consequences—and the economic, infrastructure, and community tradeoffs—of developing it only for government use.