Say No to Warm Pools

Somewhere in your infrastructure there is a cluster that is mostly asleep. It is fully provisioned, fully paid for, and almost entirely idle, just waiting for a traffic spike that might arrive this afternoon, or next Tuesday, or never. That cluster is called a warm pool, and it is one of the most expensive habits in modern infrastructure.

The logic is seductive. Cold starts are slow, dependencies take time to initialize, and when the spike comes you cannot afford to be caught provisioning. So you keep capacity hot, just in case. In effect, you pay peak prices twenty-four hours a day to avoid a few seconds of latency during the handful of minutes so your response times dont fall of a cliff.

It is a rational response to a real problem. It is also a workaround — and like most workarounds, it survives only because the thing it works around has never been fixed. Fix the cold start, and the warm pool has no reason to exist.

The always-on cluster

We have established that a warm pool is capacity you keep running ahead of demand. Three forces keep that pattern alive, and they are almost always the same ones:

• Cold-start penalties. Booting an OS, pulling an image, and warming a runtime can take many seconds. That is unacceptable in front of a user, so the boot is paid for in advance.
• Dependency initialization: Caches, connection pools, JIT warmup, model weights — the first request into a fresh instance is always by far the slowest.
• Compliance & network constraints: In regulated or locked-down environments, on-demand provisioning is simply too slow, so headroom is kept permanently online.

You wouldn’t pay this bill anywhere else

Real traffic does not look like a flat line. It looks much more like a heartbeat. It might be quiet overnight, go through a morning ramp or a lunchtime spike, with long troughs in between. Provisioning for the peak means paying for the peak during every trough, while the troughs are most of the day.

The gap between what you provision and what you truly use is the warm pool. According to Datadog, between 50% and 80% of cloud spend is wasted on idle resources - that means 2x to 5x over-provisioning.

Where warm pools bite

Idle cost is the obvious problem, but it’s not the only one. Teams running warm pools tend to fight the same four fires:

• Fragile scaling during spikes: When the spike finally exceeds the pool, you are back to provisioning under load, which brings you back to the exact same scenario the pool was meant to prevent, now happening at the worst possible moment.
• Drift from desired state: Long-lived pools get partially reconfigured, patched unevenly, and diverge from the image you think you are running.
• Ongoing operational overhead: Someone has to keep the pool warm enough to be useful but not so large it becomes too much of a cost item, in an endless cycle of tuning and babysitting.
• Over-provisioning and idle cost: As we discussed, the default failure mode. Size for the worst hour, pay for it every hour.

And then there is the compounding factor: A great many warm pools are Kubernetes clusters. That adds a second layer of latency on top of the first, because the Kubernetes control plane is not exactly famous for its reflexes. Scheduling, admission, image pull, readiness — by the time a new pod is genuinely serving, the spike may already be over. So you keep the pool even warmer to hide the orchestrator’s own lag. Warm pools stacked on slow control planes become a workaround for a workaround.

”Keep it warm, spikes just ahead.”

This pattern shows up wherever startup time touches the bottom line, especially in customer-facing services with spiky traffic, batch and data-processing jobs, internal platforms where a slow runner slows everyone down. It is also industry-agnostic and touches everything from finance, to retail, enterprise, and SaaS. The constraints differ: Regulated shops fight slow, security-gated provisioning, consumer products fight sudden spikes and user-visible latency, etc - but the reflex is the same everywhere: Keep it warm, spikes just ahead.

What good teams already do about it

Mature SRE teams already know warm pools are a smell, and they spend great effort on shrinking them. The playbook is usually the following:

• Make provisioning faster — image baking, immutable runners, pre-cached layers, so a cold start hurts less.
• Improve horizontal elasticity — faster autoscaling signals and quicker reaction, so capacity can chase demand instead of pre-empting it.
• Isolate workloads better — ephemeral runners per job, where feasible, so state never drifts and every start is clean.

Every one of these is an attempt to make the cold start cheap enough that you no longer need a pool of hot spares. They are exactly the right instincts. The problem is that, on a conventional stack, they only ever get you part of the way. The boot is faster, but it is still measured in seconds, so the pool shrinks a bit but never disappears.

Unikraft takes each of those instincts and runs it all the way to its conclusion.

The warm-pool problem, mapped to the Unikraft answer

The Unikraft answer: React, don’t reserve!

A warm pool is a result of the unfortunate realization that you cannot start fast enough when it matters. Unikraft eliminates that problem outright. Because our VMs cold-start in single-digit milliseconds, there is nothing to pre-warm, so the “cold” start is the fast path. That collapses the warm-pool calculus into three properties:

• Scale to zero (~ms): Suspend idle instances in milliseconds. Stop paying the second traffic stops.
• Scale out (10–20s): Bring one node or a whole fleet online on the spike — reactively, not pre-emptively.
• Native Kubernetes integration: Runs under the Kubernetes you already have — without the control-plane lag that forces the pool.

Kubernetes, transparently

Deploy your existing Kubernetes workloads onto Unikraft with no changes to your manifests. You get millisecond VM cold starts and automatic scale-to-zero underneath the orchestrator you already run, so the control plane never has to be kept warm.

The verdict

Stop provisioning for the spike. Start reacting to it.

Warm pools were always a workaround for one thing: A cold start too slow to trust. Remove the slow start (genuinely remove it, down to milliseconds) and the workaround loses all justification. No idle fleet, no drift, no 3 a.m. capacity-tuning, no control plane held warm to hide lag.

You do not need a pool of machines sitting in a hot tub waiting for traffic. You need infrastructure that shows up in milliseconds when the traffic does, and vanishes when it leaves. That is the whole pitch. Say no to warm pools.

Notes & references

• Cold-start and scale-out figures are representative for Unikraft guests and vary by configuration and workload.
• Cost/utilization and timeline diagrams are illustrative — they show the shape of warm-pool waste, not a specific measured deployment.
• For the isolation case behind the millisecond VM, see The Mighty (micro)Virtual Machine.
• Industry-typical overprovisioning figures are from Datadog’s 2024 Cloud Cost Report, which found that 50–80% of cloud spend is wasted on idle resources.