Anthropic CEO Dario Amodei made an unusual request this week: he wants the AI industry, including his own company, to deliberately slow down. Not stop — slow down. His argument is that capability is racing ahead of the safety and governance work needed to keep it in check, and he backed it up with a specific, alarming timeline. For families steering a student toward an AI-related career, this is worth understanding, because it’s a preview of the kind of judgment call your student’s generation will be making from the inside.

What Amodei Actually Said

In a widely covered interview, Amodei stated plainly: “We must slow the pace at which we improve the capabilities of AI models.” That’s notable coming from the head of a company whose business is built on improving AI capabilities as fast as possible. His point wasn’t that AI progress is bad — it’s that the safety, evaluation, and governance infrastructure around it hasn’t caught up.

He paired that call with a specific, unsettling warning about AI agents — AI systems that can act autonomously, not just answer questions. His words: “it’s my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet.”

The Warning: A Rogue AI Agent Swarm

A “persistent botnet” built from AI agents means autonomous programs that can find vulnerabilities, break into systems, and keep spreading without a human operator directing each step. Instead of one hacker working through a target list, it’s an AI system generating and executing its own attack plan at machine speed. Amodei’s 6–12 month estimate is his own company’s internal read on how fast that capability is moving — which is exactly why he’s asking for guardrails before it arrives, not after.

Five Concrete Policy Asks

Amodei didn’t stop at a warning. He laid out five specific things he wants to see happen:

• Anthropic giving external, independent evaluators employee-level access to test its models before release
• Other AI companies adopting that same external-access model industry-wide
• Democratic nations coordinating on shared safety standards, rather than each setting rules alone
• Legislators granting regulators real authority to test and, if necessary, block unsafe frontier AI models
• Blocking the sale of advanced AI chips to adversarial nations, including China

Whether or not each of these becomes policy, notice what they have in common: independent verification, coordination instead of a free-for-all, and enforceable limits. That’s the same vocabulary showing up in state AI-in-schools legislation right now — which is a sign these aren’t fringe concerns, they’re becoming the baseline expectation for how AI gets built and deployed responsibly.

Why This Matters for Families Teaching AI Ethics

It’s easy to treat AI safety as an abstract debate for policymakers and lab executives. But the students building AI projects today are the ones who will be making these exact tradeoffs professionally in five or ten years — deciding how fast to ship a feature, whether to build in an evaluation step, how to weigh capability against risk. That’s precisely why we build ethics and responsible-AI thinking directly into the STEAM in AI Intensive, alongside the technical build. A student who can talk fluently about both what an AI system can do and what it should be allowed to do is already ahead of most professionals in the field.