Letting an agent write and run Python is one of the most useful things you can give it. An analyst can ask it to reshape a spreadsheet export, compute rolling returns, chart a breakdown, or clean up a CSV that came out of some web portal. It’s also the tool I’d worry about most, because it runs code a language model wrote — on a server that probably holds credentials for other systems.
This post is about how I’d build that sandbox when it has to run in a self-hosted deployment: on a customer’s server, with hardware you don’t choose and nobody on hand to babysit it.
Make the sandbox a separate service with nothing worth stealing
The API process should never execute model-written code itself. Put execution behind a separate service — call it the sandbox broker — and have the API mint a short-lived signed grant for each request: who it’s for, when it expires, which capability it covers (run code, or write a file into the workspace), and which session it belongs to. The broker checks the signature, the expiry, and the capability before anything else.
Be honest about what that grant gives you. If the broker doesn’t keep a record of used nonces, grants are short-lived, not single-use, and the TTL is your replay window. That’s a reasonable trade for a stateless broker, as long as nobody claims the stronger guarantee.
What the guest doesn’t get matters as much. It shouldn’t mount the application’s data volume, any browser profiles, or the artifact store. It shouldn’t receive the Redis password or any encryption key. I’d also give the broker a hard-coded list of host paths it refuses to mount — /etc, /root, /home, the Docker socket, and the application’s own state directory — so a configuration mistake can’t quietly hand the guest the host.
Pick the runtime from what the host can actually do
You don’t control the customer’s hardware, so choose the strongest isolation available at startup and report which one you got:
def pick_runtime(host) -> Runtime:
if host.has_kvm:
return Runtime("microvm", reason="KVM available")
if host.has_runsc:
return Runtime("gvisor", reason="runsc registered with Docker")
if settings.allow_plain_docker:
return Runtime("docker", reason="operator opt-in, dev only")
return Runtime.unavailable("no isolating runtime on this host")A microVM when the host exposes /dev/kvm. Otherwise gVisor, which intercepts syscalls in a user-space kernel. Plain Docker only if an operator explicitly opts in, and labelled as such in the health check so nobody mistakes it for isolation. If none of those is available, the tool is unavailable and the agent says so.
Then the container flags. They’re a boring list, and every item is there for a reason:
client.containers.run(
"python:3.12-slim",
command=["sleep", "infinity"],
network_mode="none",
read_only=True,
tmpfs={"/tmp": "size=64m"},
cap_drop=["ALL"],
security_opt=["no-new-privileges"],
mem_limit="512m",
nano_cpus=1_000_000_000,
runtime="runsc",
)Code runs by exec-ing into that sleeping container. On timeout, kill the whole container, not just the Python process — a forked child or a stuck thread shouldn’t outlive the request. Keep the workspace outside the container so it survives the kill.
No network, and no exception path
It’s tempting to design a “network profile” the agent can request when it needs to fetch something. I’d leave it out. The guest gets network_mode: none, and there is no way to open it.
The agent still needs data, so the trusted side writes it in. When a tool returns something large — a price history, a fetched web page, a downloaded file — the harness saves it as a file in the session’s workspace and keeps a small index the code can read. The model doesn’t have to paste 40,000 rows into a code cell, and the sandbox never needs credentials to fetch anything.
Output deserves the same suspicion as input. Anything the code produces for the user should be in one deliverables directory, and every path handed back for download should be resolved and checked against it. A path outside that directory is an error, not something to skip quietly.
Capacity is a memory problem
On a single machine, what runs out first is memory — not CPU, and not the number of containers. So admission should be a formula, not a hope:
def free_slots(mem_available_mib, reserve_mib=1024, per_guest_mib=512):
return int((mem_available_mib / 1.2 - reserve_mib) // per_guest_mib)Two details matter more than the formula. Count guests that are still starting against capacity before their expensive startup finishes, or a burst of requests will all pass admission and then all start at once. And charge existing guests their full memory limit, not their current usage — a guest that’s idle now can allocate its whole limit a second later.
Add a global ceiling, one slot per user session, and an idle TTL. When a session goes idle, snapshot its workspace to a tarball and free the slot. The next request restores it.
Bugs worth designing against
A “local mode” that’s really a subprocess. Development setups often run untrusted code as a plain subprocess for convenience, with a comment saying it isn’t a security boundary. That comment won’t stop anyone shipping it. Make even the dev runtime a real container with no network and no capabilities. On Docker Desktop you may not be able to bind-mount a subdirectory of a named volume into a child container. Keeping the workspace inside the container and moving files with Docker’s archive API is slower, but it’s correct.
Fallbacks that crash before they fall back. Some microVM runtimes panic during initialisation on hosts without nested virtualisation, before your fallback logic gets a chance to run. Probe each runtime in isolation, and never let a runtime that failed its probe go ahead of one that passed.
The privileged broker. Depending on the runtime and the host’s AppArmor setup, the broker may need to run privileged and hold the Docker socket. Say so plainly: the broker is a high-trust component. The isolation boundary is the guest — the microVM or gVisor — not the broker. The broker’s job is to be small and to refuse things.
A write that returns 200, then a read that returns 404. This is the one I’d test for specifically. Suppose idle snapshots are made by running tar inside the guest and writing the archive into the workspace directory itself. Tar notices the file changing under it and exits with status 1 (“file changed as we read it”). If the pool swallows that exit code, it saves a stale copy and acknowledges the import anyway. The user uploads a file, gets a success, and can’t open it.
The fixes generalise: write archives somewhere outside the tree being archived; treat a non-zero tar exit as the failure it is; acknowledge an import only after reading back its size and hash; and make reads take the same lock as writes. While you’re there, copy deliverables explicitly rather than trusting a best-effort snapshot to include them. If a restore fails, keep the old workspace and say so, rather than quietly giving the user an empty one.
Evidence for computed numbers
One more idea I like. If the sandbox exposes trusted helper functions — a fee calculator, a position-sizing formula — have each call append a line to a log inside the workspace: function, version, a hash of the input, a hash of the output, and status. If the run is killed before the log is finalised, report the log as incomplete rather than pretending it’s whole.
That gives you evidence of which calculation produced a number. It isn’t authorization and it isn’t a tamper-proof attestation, and it’s worth saying both in the design doc. Being precise about what a mechanism doesn’t prove is a lot of what sandbox work turns out to be. The same theme comes up with browser agents, where the thing being protected is an identity rather than a filesystem.