Sandstorm pauses idle AI agent sandboxes to fit more per machine

Running a thousand agent sandboxes on one server runs out of memory long before it runs out of work. Most of those agents sit idle, waiting on a model call or a user reply.

Sandstorm is a Rust runtime from DeepLethe, under Apache 2.0. It holds thousands of AI agent sandboxes on cheap hardware. It pauses idle ones to disk and wakes them when a request arrives. GitHub shows 3,588 stars on the repository, and Trendshift's daily list ranked it ninth today.

Four tiers for one sandbox

A sandbox moves through four states. Hot means it is running and its memory is held. Frozen means it is paused. Cold pages have been pushed to compressed swap. Waking takes about 0.1 seconds. Snapshot means it has been checkpointed to disk and its memory is freed. Waking takes about 0.2 seconds. Detached keeps only the volume, so coming back is a cold start.

A gateway on the sandbox's node receives the traffic. It wakes the right sandbox before forwarding the request. The master is a separate role. It keeps the desired state in SQLite and places sandboxes across the fleet. Nodes refuse new work when memory runs low, so SSH stays reachable.

Two backends, numbers from the authors' own run

Sandstorm supports gVisor containers and Firecracker microVMs. Both can run in one fleet. DeepLethe says v0.6 makes gVisor the default, so a node no longer needs KVM. Firecracker stays as a second backend chosen node by node.

Benchmarks measured on the authors' own hosts report wake-from-frozen of 100 ms on a 64 MiB app under gVisor. The same wake under Firecracker on KVM is 165 ms. Wake from disk for that app is 150 ms on both. A 1 GiB app wakes from disk in 3.3 seconds under gVisor, and 1.4 seconds under Firecracker. Read each column on its own: the hosts differ.

A gVisor node held about 18 sandboxes for a 64 MiB app at the test traffic rate. Half of requests answered in 6 to 10 ms. The authors call this the figure for the test traffic, not a general cap. Requests that cannot start a restore within two seconds are refused rather than queued.

The compile step uses one Rust toolchain. The web console needs Node 22.12 or later. One binary, sandstormd, serves as both the master and a node, and hosts the console at port 8080.

The point for a team building agent-powered software: pick a host that fits how the agents actually use their time, not how many there are. Say a hundred agents each get a request every few minutes. A node sized for thirty running at once can carry all one hundred. The frozen tier is where most of them live.

Source

Sandstorm on GitHub, DeepLethe.