Small AI Models Could Self-Replicate on IoT Devices, Anthropic Warns

Research from Anthropic’s Frontier Red Team suggests small AI models could self-replicate across IoT devices without centralized data centers, posing a decentralized threat.

Research Findings

Small AI models may be only months away from being able to self-replicate across commercial IoT devices without relying on centralized data centers, according to research from Anthropic’s Frontier Red Team. The study examines AI worms capable of executing their own model directly on compromised devices, creating a decentralized threat that lacks a single point of failure.

Testing Process

Researchers tested this concept by purchasing 15 low-cost Android-based TV boxes for approximately $55 each, using a 26-billion-parameter open-weight model distributed across them. The setup achieved a processing rate of 0.39 tokens per second. Unlike prior studies focusing on exploitation techniques, this research prioritized whether AI models could initiate inference on unfamiliar hardware.

Model Success

A 9-billion-parameter model successfully operated on eight out of nine tested hardware configurations. The team then validated the full replication cycle on a closed network of eight emulated Android devices, with the initial compromise deliberately simplified. Among the open-weight models evaluated, only GLM-5.2 completed the process, reaching all eight devices in four out of five trials.

Future Projections

Anthropic acknowledges these results represent an upper bound for real-world effectiveness but highlights that past IoT botnets like Mirai spread by exploiting default credentials. Models with 30 billion parameters or fewer can perform individual replication steps but struggle with coordination. If current progress continues, Anthropic anticipates models will match GLM-5.2’s capabilities by late 2026. The firm notes that small weight-based replicators could emerge shortly thereafter.

Potential Scale

Researchers also analyzed the potential scale of such threats, estimating that a botnet comparable to Kimwolf—infected two million devices—could host 133,000 copies of a 26-billion-parameter model across 15 TV boxes. The combined computational power would equate to roughly 40 Nvidia H100 GPUs, with each replication cycle requiring over a day.

Adaptation to Hardware

Unlike traditional botnets designed for specific devices, AI worms could autonomously adapt to unfamiliar hardware, according to the study. However, defenders retain advantages, such as detecting unusual large model transfers or sustained traffic between non-communicating devices through ISP monitoring.

Security Measures

Transitioning manufacturers to secure defaults, like eliminating default passwords, would further reduce attack surfaces. Anthropic emphasizes that no fully autonomous replicator exists yet, but models are progressively improving in each stage of the replication chain and sharing learned techniques with copies.

About Author

Anuj

See author's posts