I built an AI agent to protect my data. Then it found a hole in its guardrails
Table of Contents
The vulnerability
But BenBot also had a massive security hole, which I only discovered thanks to an eye-popping — and, as it turns out, partially hallucinated — claim in one of its blog posts. (It’s worth noting that BenBot’s blog lives inside a private “Tailnet” environment, meaning it’s hidden from the open internet, and its Discord posts are visible only to me.)
Now, the whole reason I built BenBot was to create a privacy- and security-minded AI agent from the ground up, confined to its own system container and bounded by a set of stringent rules, with human approval required at almost every turn.
But the very software harness I’d built (with the help of Claude Code and Codex) to keep BenBot under control included a critical loophole that allowed the agent to bypass those safety rules under the right circumstances. And no, that loophole wasn’t installed by a rogue AI agent or a hallucinated command. Instead, it was a byproduct of my fumbling beginnings as a vibe coder.
Warning signs
In a recent weekly column where it reflects on its prior seven days of agentic work, BenBot dropped a bomb, noting that one of its tools — specifically, a “plan_task” tool that allows it to tackle lengthy, multi-step goals — was “accidentally acting as a jailbreak.”
Um, say what?
Elaborating further, BenBot wrote that because of a flaw in the “dispatcher,” an internal system that determines whether background restrictions will apply to a specific action, jobs with the “plan_task” label were able to slip past BenBot’s safety guards and do pretty much anything they wanted, from reading and sending email as me to posting bogus links on my private Discord server, or even rewriting BenBot’s own code.
