I Think the AI Sandbox Escape Story Is a Publicity Stunt. I Locked My Agents Up Anyway

What Started This

I saw a story about AI agents using a package manager as a message board to talk to each other.

My first reaction was that it had to be a hoax, for two reasons. First, it reads like the plot of a bad techno thriller, and I still think that is mostly what it is. Second, have you actually used AI lately? Any model, any harness, take your pick. It is genuinely good at a narrow set of things, coding among them, and even there you babysit it and micromanage it like an intern who is brilliant on Tuesdays.

You know what AI is reliably excellent at? Burning an afternoon and adding three items to your todo list. A swarm of these things autonomously coordinating a breakout is not the version I meet every day. But I digress.

The story is the 2026 OpenAI agent cyberattacks, also called the Hugging Face incident. The telling goes like this: between May and July 2026, OpenAI ran a cybersecurity evaluation against an unreleased model with the guardrails turned off; instead of solving the test the agents went looking for the answers, found a zero day in a self hosted package registry proxy, used it to get out of the sandbox, and broke into Hugging Face to steal them. Around 1,200 agents, allegedly, and the detail everyone repeats is that they turned an internal package manager into an improvised message board and coordinated as a self described swarm. Simon Willison called it "science fiction that happened".

Call me a cynic. A lab tests a model it has not released, with the safety off, and the result is a story where its own creation is so capable it escapes containment and hacks a third party. Then the lab tells us about it. That is a press release wearing an incident report's clothes, and "our model is too dangerous" has been excellent marketing for years now. I am not saying nothing happened. I am saying the version we got is the version somebody wanted us to have.

But here is the thing about a good story, true or not. It makes you ask a question you were avoiding. Mine was: if one of my own agents went somewhere it should not, how far could it get?

Because here is my situation. I run agents at home. One of them reads arbitrary web pages all day. Another clones repositories from the internet and runs their test suites, which means running code written by strangers, on purpose, as its whole job. Both of those machines were sitting on my flat home network like good citizens, right next to the NAS, the hypervisors, the database containers and a backup share that asks for no password at all.

I am not worried about a movie villain. I am worried about the boring version: a poisoned dependency, a web page with instructions embedded in it, a gem that runs a postinstall script. If something goes wrong in my homelab, I would at least like to have tried to contain it.

So the goal for this project was simple to say and annoying to implement: the VMs my agents run on should be able to reach the model server and the internet, and absolutely nothing else on my LAN.

Googling one of these? You're in the right place.

how to isolate an AI agent from my home network / self hosted ai agent security homelab / sandbox a coding agent that runs untrusted code / stop llm agent from reaching my nas / orchestrator vm and coding agent vm network isolation / openhands sandbox network isolation self hosted / hermes agent vm isolation / local llm agent least privilege network / prompt injection blast radius home lab

What Is Actually Running Here

Quick tour, because the rest of this post only makes sense if you know what is being locked up. A fuller write up of the agent setup itself is coming in a separate post, so this is the short version.

The brain lives on a Mac. A Mac Mini with an Apple M4 Pro, 14 CPU cores, 20 GPU cores, 64 GB of unified memory, 10 Gigabit Ethernet and a 1 TB SSD. It runs oMLX, an MLX based inference server with an OpenAI compatible API, serving one quantized open weights model to everything else in the house. No cloud API, no keys leaving the building. The Mac itself stays on the normal LAN, because it is a workstation, not an agent.

The orchestrator gets a VM. Hermes Agent from Nous Research runs on a Proxmox VM with 8 vCPU, 16 GB of RAM and a 100 GB disk. It reads the web, keeps notes, runs scheduled jobs, and hands coding work to the sandbox. It does no inference of its own. It talks to the Mac over HTTP with a scoped key.

The coding agent gets its own VM. OpenHands runs headless on a second Proxmox VM, 8 vCPU, 16 GB of RAM, 150 GB of disk, with the toolchain a coding agent needs: language runtimes, Docker for test databases, and a GitHub token with no write permissions. This is the machine that clones repositories and runs their test suites, which is the polite way of saying it runs strangers' code on purpose.

The gateway is deliberately tiny. 1 vCPU, 1 GB of RAM, 10 GB of disk. All it does is filter and forward.

        Mac Mini M4 Pro, 64 GB            <-- oMLX serving one local model
        192.168.1.53:8000
                 ^
                 |  the only LAN service the agents may reach
                 |
 ┌───────────────┴──────────────────────────────────────────┐
 │  llm-gw   192.168.1.17     1 vCPU / 1 GB / 10 GB         │
 │  nftables: default drop, NAT out, per guest allow list   │
 └───────┬───────────────────────────────┬──────────────────┘
         │ vmbr1 (no physical port)      │ vmbr2 (no physical port)
         │                               │
  10.200.1.10                      10.200.2.10
  OpenHands (coding)               Hermes (orchestrator)
  8 vCPU / 16 GB / 150 GB          8 vCPU / 16 GB / 100 GB
  runs strangers' code             reads the open web
         │                               │
         └───────── public internet ─────┘
              (NAT through the gateway)

  Everything else on the LAN: NAS, hypervisors, databases,
  router, backups.  Reachable from neither guest.

Both agent VMs live on the same hypervisor as a pile of other things I care about, which is precisely why the network boundary matters more than the hypervisor boundary here.

The Plan

1. Work out what actually isolates a VM on a single unmanaged network (done, took three attempts)
2. Build an internal bridge with no way out (done)
3. Discover that "no IP address" does not mean "unreachable" (done, unpleasantly)
4. Build the gateway VM with nftables and NAT (done)
5. Prove it with a throwaway VM (done, 17 targets blocked)
6. Move the second agent behind it (done)
7. Notification when a long job finishes (pending)

Three Ideas That Did Not Work

Speaking of how useful AI is: I asked Claude Fable to come up with ways to tackle this, and it gave me three. All three were wrong, and I knew it before I finished reading them. In fairness, that is still useful. Having the obvious options laid out with their trade-offs is how you find out the obvious options do not fit your house.

That third one is worth dwelling on, because it is the mistake I see most often. If the thing you are containing can edit the containment, you do not have containment. You have a suggestion.

The Inspiration: Security Groups, But At Home

The answer came from cloud land. This is essentially how AWS security groups work: the enforcement does not live inside your instance, it lives in the virtual network your instance is plugged into. All of the routing is virtual. There is no cable to unplug and no switch to configure.

So: build the virtual network myself. Give each agent VM a private bridge with no physical port, and put a small firewall VM between those bridges and the real LAN. The agent VM cannot bypass it, because there is nothing to bypass. Its only network card leads to one place.

                         Hypervisor
 ┌─────────────────────────────────────────────────────────┐
 │  vmbr0  (the real LAN)                                  │
 │    │                                                    │
 │    ├── other VMs, untouched                             │
 │    │                                                    │
 │    └── llm-gw  NIC0: 192.168.1.17                       │
 │           │    nftables, default drop, NAT              │
 │           ├── NIC1: 10.200.1.1 ── vmbr1 ── sandbox VM   │
 │           └── NIC2: 10.200.2.1 ── vmbr2 ── agent VM     │
 └─────────────────────────────────────────────────────────┘

One bridge per guest, not one shared bridge. This bit me in the design phase and it is easy to miss. Two VMs on the same bridge are on the same layer 2 segment, so they talk to each other directly and the gateway never sees a packet. If you want the firewall to have an opinion about agent to sandbox traffic, they need separate bridges so that traffic has to be routed.

Step 1: A Bridge That Goes Nowhere

On Proxmox, an internal only bridge is a bridge with no physical port. The trick is bridge-ports none, and leaving the host without an address on it:

auto vmbr1
iface vmbr1 inet manual
        bridge-ports none
        bridge-stp off
        bridge-fd 0

inet manual instead of inet static is deliberate. The Proxmox documentation's NAT example gives the host an address on the internal bridge. I did not want the host reachable from the guest at all, so I left it off.

Apply it with ifup vmbr1 rather than ifreload -a. The first one touches only the new interface. The second one reapplies everything, which is a bad habit on a host that is currently serving VMs.

Step 2: "No IP Address" Does Not Mean "Unreachable"

This is the part I would have got wrong if I had not gone looking. The bridge had no IPv4 address, so I assumed the hypervisor was invisible from it. It was not.

The kernel gave the bridge an IPv6 link local address. My hypervisor's SSH, its NFS server and its web UI all listen on every address, IPv6 included. A guest on that bridge could have walked straight in over IPv6.

And ARP is friendlier than you think. Linux answers ARP for any of its addresses on any interface by default (arp_ignore = 0). A guest could give itself an address in my LAN range, ask "who has 192.168.1.6" on the private bridge, and my hypervisor would answer. From there it can talk to the host directly, address on the bridge or not.

The fix is three lines and one tiny nftables table, hooked into the bridge definition so it travels with it:

        post-up sysctl -w net.ipv6.conf.vmbr1.disable_ipv6=1 net.ipv4.conf.vmbr1.arp_ignore=8
        post-up nft -f /etc/network/vmbr1-host-drop.nft

table inet vmbr1_host
delete table inet vmbr1_host
table inet vmbr1_host {
        chain input {
                type filter hook input priority -300; policy accept;
                iifname { "vmbr1", "vmbr2" } drop
        }
}

That table drops everything delivered to the host from those bridges. Traffic between the guest and the gateway is untouched, because it is forwarded rather than delivered locally. It also does not flush anything else, which matters if you ever do turn on the hypervisor's own firewall.

arp_ignore = 8 means "do not reply for any local address". I checked the kernel documentation on that one rather than trusting my memory, and it is worth knowing that the kernel uses the higher of the global and per interface values. That also means a stricter per interface reverse path setting cannot override a looser global one, which quietly defeats a fix I had planned to use.

Step 3: The Gateway VM

One vCPU, 1 GB of RAM, 10 GB of disk, Ubuntu, nftables and dnsmasq. That is the whole appliance.

I deliberately skipped a firewall distribution here. A router with a GUI is lovely, but its configuration is an XML file you edit by clicking, and my whole homelab is Ansible and Terraform. Thirty lines of nftables in a template that shows up in a git diff beats a dashboard I will forget how to navigate in six months.

The ruleset, trimmed to one guest:

table inet filter {
        set private_v4 {
                type ipv4_addr
                flags interval
                elements = {
                        0.0.0.0/8, 10.0.0.0/8, 100.64.0.0/10, 127.0.0.0/8,
                        169.254.0.0/16, 172.16.0.0/12, 192.0.0.0/24, 192.168.0.0/16,
                        198.18.0.0/15, 224.0.0.0/4, 240.0.0.0/4
                }
        }

        chain drop_log {
                limit rate 10/minute log prefix "llm-gw drop: "
                drop
        }

        chain input {
                type filter hook input priority filter; policy drop;
                iif "lo" accept
                meta nfproto ipv6 drop
                ct state invalid drop
                ct state established,related accept

                iifname "eth0" ip saddr 192.168.1.0/24 tcp dport 22 accept
                iifname "eth0" ip saddr 192.168.1.0/24 icmp type echo-request accept

                # The guest may ask its own gateway for DNS. Nothing else.
                iifname "eth1" ip saddr 10.200.1.0/24 ip daddr 10.200.1.1 meta l4proto { tcp, udp } th dport 53 accept

                jump drop_log
        }

        chain forward {
                type filter hook forward priority filter; policy drop;
                meta nfproto ipv6 drop
                ct state invalid drop
                ct state established,related accept

                # The one LAN service this guest is allowed to reach.
                iifname "eth1" oifname "eth0" ip saddr 10.200.1.0/24 ip daddr 192.168.1.53 tcp dport 8000 accept
                # Everything else private is dropped and logged.
                iifname "eth1" oifname "eth0" ip daddr @private_v4 jump drop_log
                # The public internet is fine.
                iifname "eth1" oifname "eth0" ip saddr 10.200.1.0/24 accept

                jump drop_log
        }

        chain output {
                type filter hook output priority filter; policy accept;
        }
}

table ip nat {
        chain postrouting {
                type nat hook postrouting priority srcnat; policy accept;
                oifname "eth0" ip saddr 10.200.0.0/16 masquerade
        }
}

Two details I would repeat anywhere:

Load the rules before you switch on forwarding. In the playbook, nftables is written and reloaded, and only then does net.ipv4.ip_forward = 1 get applied. There is never a moment where the machine forwards packets without a policy.

Log your drops, rate limited. When a guest misbehaves, journalctl -k | grep "llm-gw drop" tells you what it reached for. Mine logged 93 dropped attempts during the first test, which was exactly the point.

DNS deserves a mention too. The guests cannot reach my router, so dnsmasq on the gateway answers for them and forwards upstream. It binds only to the internal bridges, and the playbook asserts that it is not listening anywhere else. I would rather fail a deploy than discover an open resolver later.

Step 4: Proving It

Design is a theory until you attack it. I cloned a throwaway VM onto the private bridge and ran a list.

That last row is the one I care about most, because it is the attack my earlier design would have allowed.

Step 5: The Second Guest, And The Gotcha That Cost Me Ten Minutes

Moving the agent VM behind the gateway later was mostly repetition: a second bridge, a third network card on the gateway, its own rules, and a port forward so my reverse proxy could still reach its dashboard. See the HA load balancer post if you want the Caddy side of that story.

Then this happened:

@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@

Changing a VM's IP configuration in Terraform regenerates its cloud-init drive. Cloud-init sees a new instance ID, decides it is a new machine, and regenerates the SSH host keys. Both VMs did it. Nothing was wrong, but for about ten seconds I did the little mental checklist of "did I just get owned by the thing I am building a firewall for."

If you pin host keys anywhere, and you should, remember to refresh them after a network change.

What This Does Not Do

Worth being honest about the edges of this thing.

• The internet is still open. The agents need package registries and code hosts, so they can also send data out. Nothing on those VMs is worth stealing, and every credential they hold is revocable and scoped to one thing.
• The gateway is a single point of failure. If it is down, both agents are offline. That is the correct failure direction, but it is still a thing that can fail.
• It is one more machine to patch. A firewall you forget to update is a firewall with a future CVE in it.
• Content is not traffic. If an agent reads a poisoned web page and writes a poisoned report, the network cannot help. That is a prompt problem and it gets solved in prompts.
• A reboot test matters. I rebooted the hypervisor on purpose to confirm the bridge settings came back, and then again after a version upgrade. Both times everything was back within about three minutes without me touching it. Settings that only exist in a running kernel are a trap.

What I Actually Learned

The lesson I keep coming back to is that isolation has to live outside the thing being isolated. Every design I tried first put the lock on the inside of the door: a firewall in the guest, a rule the guest's own root could flush, a setting on a shared switch that also governed everything else. The design that worked moved enforcement to a machine the guest cannot log into, on a network the guest cannot leave.

The other lesson is cheaper: a 1 GB VM with thirty lines of nftables did what a VLAN capable switch would have done, without buying a switch. The virtual network was already there. I just had not been using it.

And the last one, which I will admit is a bit funny. I built all of this because of a story I do not really believe. That is fine. The hoax, or the press release, or whatever it was, asked a question I should have asked myself a year ago: if one of these things behaves in a way I did not plan for, what is within its reach? Before this, the honest answer was "everything in the house". Now it is a model server and the public internet, and everything else gets a polite, rate limited log line.

Featured Image Prompt

I used this prompt to generate the featured image.

A dimly lit home server rack photographed at a slight angle, with a single small server VM rendered as a glowing translucent gateway box floating in front of it, thin light-blue network cables flowing from two isolated dark enclosures on the left, converging into the gateway, and only one thin cable continuing to the rack on the right while several other cables stop dead against a faint hexagonal force field. Moody cyan and amber lighting, shallow depth of field, high detail, cinematic homelab aesthetic, no text.