Welcome to Safe Mode, your weekly report for pressing security and privacy news—and what steps to take next. Want this newsletter to come directly to your inbox? Sign up on our website!
Last week, Anthropic released a report detailing a series of experiments—tasking AI with spreading itself across low-tech smart machines. The team examined how easily small models could discover, configure, and install model weights (that is, the model’s parameters for data processing) on vulnerable devices, then repeat the process to proliferate. The resulting botnet (and collective compute power) could then be used for AI workloads. The behavior mirrors that of a classic computer worm, where zero human oversight is needed for the malicious code to copy continuously.
(What’s a small model? AI comes in different sizes, so to speak. The ones we consumers best know represent the large-language variety, like Claude, ChatGPT, Gemini, etc. But small-language versions exist, too. They’re more streamlined and generally built for specific purposes. Think of an LLM as a sprawling multipurpose tool, while an SLM is a single-purpose one. And sometimes a very singular purpose at that.)
Turns out, small, specialized AI models achieved this goal pretty effectively. At least, they did so in a controlled lab environment, with researchers unleashing several small models on Android TV boxes. (Such media boxes have notoriously lax security configurations.) But Anthropic believes this attack method could become real “within months.”
Sounds scary, right? I have to admit that when I first saw a news headline about this report, it felt appropriately spooky for October. After all, we already know the overwhelming majority of smart devices have poor security. (Heck, I’ve written about the very topic, and multiple times at that.) Botnets have also been around forever. The idea of them becoming more advanced through AI isn’t a shocker, either. But I read the report more as a heads-up than a dire warning.
Nicolas J Leclercq / Unsplash
First: Anthropic came to these results in a controlled environment, rather than seeing it on the open internet. (Shenanigans in the wild are the province of its large-language models at the moment.)
Second: Anthropic’s predictions for when we’d see this in the wild is fairly vague. Sure, the implication is just a handful of months, based on current rates of AI improvement. But “months” is very open.