The Great Fragmentation of Cloud Compute

Ever stop and wonder why a single cloud provider offers multiple compute services? Are cloud engineers a uniquely masochistic subset of the population, the sort who delight in complicating their lives with baroque infrastructure setups?

Clearly I’m being facetious. It’s not as though anyone rang up the hyperscalers and said: “I’d love you to split your compute across several products, because that’s what my deployments require. In one, give me serverless. In the next, full VMs. In a third, Kubernetes. And in a fourth, plain containers.” Clearly nobody asked for this, and yet here we are.

The fragmentation

Four products for one job

Open any hyperscaler’s compute menu and you’ll meet the same split. Functions live in one service, full VMs in another, Kubernetes in a third, and plain containers in a fourth. I’ll name AWS below with Lambda, EC2, EKS, and ECS/Fargate, but Google and Azure are no better at all. Four separate products to do basically the same thing: Run your code.

Each of them has a superpower — but each also has a weakness.

The balancing act

The engineer’s inner monologue

It’s a tough job being in charge of these deployments. You don’t get to pick the right tool; you get to pick which poison you can live with for now. One can only imagine the inner monologue:

In fact, the landscape is so absurdly complex that AWS even has so-called “Decision Guides” to help engineers navigate the maze of options.

The magic wand

If we could just wave a wand…

Ideally, we’d wave a magic wand and take the best of every one of these worlds, fusing them into a single, gloriously boring compute service that just does the right thing:

1. Stateful scale-to-zero by default, in milliseconds.
2. Kubernetes support for those who need it. No warm pools, no performance penalty.
3. No restrictions on what you run. If it fits in a Dockerfile, the platform runs it.
4. No cold starts. Latency problems begone.
5. Strong isolation for every workload, with no performance degradation.
6. Instances as first-class state. Snapshot, template, fork, checkpoint and migrate any running workload.

All of it under one overarching mandate: Infrastructure problems (cold starts anyone?) should stay strictly with the infrastructure provider. The buck should stop there.

Sometimes we forget that this fragmentation was never fundamental. It’s an accident of how the last two decades of infrastructure happened to be built — not a law of (computer science) physics.

Rejoice! The wand is real

Building the wand

At Unikraft, we set out to prove that a single, unifying cloud platform is entirely possible. As a result there is now a platform in production today, running PostgreSQL for Prisma at over 100,000 strongly-isolated instances per machine, and headless browsers, agents and functions for a growing list of teams.

For those who need a little more detail on the wishlist above, here’s how the platform delivers on each of the six points:

Scale to zero, and back, in under 10 ms

Idle instances cost you nothing. A dormant VM resumes statefully from a snapshot faster than a traditional cloud finishes pulling its image. Idle cost: $0.

A virtual node in your existing cluster

Drop a Kraftlet into EKS, GKE, AKS or your own cluster. Schedule pods as usual and they run as strongly-isolated microVMs that wake in 10 ms and scale to zero. All the power of k8s with none of the warm pools.

If it fits in a Dockerfile, it runs

Your existing Dockerfile goes in and a specialized microVM image (typically at least 10× smaller) comes out. Same source, same registry workflow, no code changes. Postgres, Chromium, a full desktop, a Rust API. Whatever you can specify, the platform runs it.

First byte within a single round trip

The proxy buffers the request, the controller resumes the right instance from snapshot, the VMM brings the microVM live and the app replies, all inside one RTT. This way, you never pay for a cold start, and your users never experience one.

Strong isolation, no penalty

Your workload runs in a real VM with a hardware-enforced perimeter and a TCB two orders of magnitude smaller. And it’s fast and dense enough to fit 100,000+ of them on one off-the-shelf server, with 10–100× better unit economics.

Instances as first-class state

The same snapshot machinery that powers scale-to-zero also makes a running instance’s entire state a thing you can capture and move.

• You can freeze a fully warmed-up VM into a template and boot fresh copies from it instantly.
• Fork a live instance to branch its exact memory and disk.
• Checkpoint long-running work so it survives restarts.
• Migrate instances across hosts with no rebuild. Perfect for warm agent sandboxes, per-tenant databases, and CI that starts from a primed state instead of cold.

No more decision tree, no more picking which compromise to live with, no more gluing four services together with duct tape. And because you can deploy it on-prem, in your own cloud account (BYOC), or on dedicated hosts, you own the entire stack.

Turns out you can put the wand down and just have one, unifying compute service — the boring, wonderful kind that simply runs your code, with no “decision guides”. Cloud compute services, finally defragmented.