--- title: "The Ten-Millisecond Agent" description: "How to run AI sandboxes at scale and (almost more importantly) how not to. I'll go into the forces that break agent infrastructure, and the platform we built from first principles to survive them: Millions of agents, ~10 ms cold starts, stateful scale-to-zero, and a strong hardware boundary around every one." doc_version: "1.0" last_updated: "2026-07-27" --- The last two years of agents have been staggering to watch and its almost certainly just the opening act. However 2026 continues, there will be dramatically more agents, doing dramatically more work, than there are today. I'm squarely in the camp of finding this exciting. An agent can compress work that used to take a person months into a handful of minutes. But every one of those agents has to run somewhere and that turns out to be a surprisingly hostile engineering problem. On top of the hostility, this problem secretly punishes anyone who reaches for the obvious tools. ## The agent boom racks up a pretty heavy bill Apps used to be a tidy loop of request and response (think a form, a button, a spinner, etc), but agents are not that. They remember, reason, and act. They run in the background, off events and signals. They call tools, pause for an API, ask a human for help, and pick the work back up later. Stateless, serverless compute has been the north star of distributed systems for a decade, but it doesn't fit this new reality. If you try to shoehorn an agent into a plain serverless or microservice stack you'll get exactly what you'd expect: Brittle, timeout-prone systems, and exploding cost. Before you write a line of agent logic, there is a set of precursor decisions that determine everything downstream: What do you run each agent in, and how do you run literally millions of them without going broke or getting breached? ## What agents demand from their infrastructure Agentic workloads ask for primitives that neither web backends nor traditional distributed systems were built to give. Strip it down and there are six forces at play — and the trouble is that they pull against each other. ## Why throwing money at it doesn't work The usual instinct when a workload gets big, is to add hardware. It's a reflex that has been deeply ingrained in us for at least a decade. But in the era of AI scale, it stops working incredibly quickly. At millions of intermittent agents, a stack that keeps a machine warm for *every agent* is lighting piles of money on fire. The root cause is in the startup path. If an environment takes seconds to boot, you can't afford to tear it down between spikes, so you keep it running, idle, and you keep paying. The only way out is to make starting so fast and cheap that keeping things off is the workable default. At the risk of sounding like agent myself by leading with the negation: Ten milliseconds is not a marginal improvement over a warm pool, it's a different regime. When starting is effectively free, you stop paying to keep things alive on the off chance they're needed. ## Asleep, but always ready: stateful scale-to-zero Agents spend most of their lives waiting on things like tools, on an API, on a human to approve the next step. Durable execution means those pauses can't cost progress. The agent has to survive them, and ideally cost nothing during down time. The platform transparently notices inactivity and scales an agent all the way down. It frees the CPU and releases the memory, thus eliminating the cost of idle time. The catch (everywhere else) is that "off" usually means "gone." Not here - when work resumes, the agent is back in ~10 ms and continues from exactly where it left off, with its full state intact, as if it had never gone to sleep. ## From a few hundred to a few hundred thousand Put cheap starts and true scale-to-zero together and the density economics change radically. A stack that keeps VMs warm fits maybe a few hundred on a server. When idle agents truly consume *nothing*, a single off-the-shelf box holds **hundreds of thousands of them** — each still wrapped in its own isolated virtual machine. ## A new workload deserves a new platform None of this is the result of tuning an existing cloud. At Unikraft we went back to first principles and redesigned every core component of the platform. This means designing controllers, proxies, and the virtualization stack itself for the thing they now have to do well: Run enormous numbers of small, intermittent, untrusted environments, at breakneck speed. These requirements translate into properties you won't find in any other cloud platform: - **~10 ms cold start** for a *full* agent environment, not a stripped function. - **Transparent scale-to-zero.** The platform detects inactivity and scales agents down in milliseconds to where they consume no resources at all. - **Stateful wake in ~10 ms.** Agents resume exactly where they left off, as if they'd never gone to sleep. - **Dockerfile-native.** Specify environments with a Dockerfile and turn them into templates the platform launches instantaneously. - **A true hardware boundary per agent.** Every single agent runs in a strongly isolated virtual machine without security compromises. - **Unprecedented density.** Think 100,000s of scaled-to-zero agents on a single box instead of a few hundred. This enables entirely new categories of products. Based on these fundamental properties, we built a platform that can run millions of agents at a fraction of the cost of any other approach. But to make these useful in production, we had to solve 2 more problems: ## Problem 1: Agent Clearance Isolating the compute is only half the security story though. An agent still has to reach the outside world to do things like calling an API, hitting a database, fetching a page — and that traffic is where the real danger lives. If you give an agent a raw API key you are handing a YOLO-deployed, prompt-injectable process your production credentials. So the solution is moving the secret out of the agent entirely while still allowing for the same functionality. We called this feature the "network shield." It is a dedicated, hardware-isolated microVM that sits on the path between every agent and the network. The agent talks to the shield, the shield holds the credentials, injects them into outbound requests, and enforces exactly where traffic is allowed to go. The agent can use a key without seeing it -- magic! The shield is built on the same primitive as everything else here - its a full Unikraft microVM. It cold-starts in milliseconds, scales to zero alongside the agent it guards, and adds a real hardware boundary. ## Problem 2: Extensibility & Plugins A platform can't anticipate every capability its users will need. So rather than bake them in, we expose a plugin API that lets you transparently extend the API of the microVMs themselves. You can essentially add new operations to an agent's environment without giving up any of the platform's guarantees. Every extended call still flows through the platform's controller and is authenticated there, and the microVM retains its core properties. Millisecond cold start, a hardware boundary, and scale-to-zero when idle. That basic mechanism is what we leverage to build the sandbox plugin: An extension that implements a first-class sandbox API on top of the microVM as a native, authenticated way to stand up secure, disposable agent sandboxes. Cold start, scale-to-zero, density, a hardware boundary, secrets the agent never touches, and an extension point to build on.That's the full spec, and it's one platform. > The new era of agents needs a new generation of cloud platform. We'll say it, unapologetically and opinionated: That platform is Unikraft. The isolation half of this story (why the virtual machine, and not a container or an isolate, is the only boundary that actually holds for hostile multi-tenant workloads) is an argument on its own. We made it in full in the [previous post](https://unikraft.com/blog/the-mighty-microvm). ## Sitemap - [Home](https://unikraft.com/) - [How It Works](https://unikraft.com/how-it-works) - [AI](https://unikraft.com/ai) - [Pricing](https://unikraft.com/pricing) - [Customer Stories](https://unikraft.com/customer-stories) - [Blog](https://unikraft.com/blog) - [Documentation](https://unikraft.com/docs) - [Glossary](https://unikraft.com/glossary.md) - [Contact](https://unikraft.com/contact)