Running User-Built Web APIs in Milliseconds with Snapshot-Based MicroVMs

Introduction

Indices lets customers build their own fast, reliable APIs for any web-based workflow. To achieve this, specialised AI agents analyse the target site to determine the underlying request mechanisms, and output connectors which complete the workflow using raw web requests. These connectors are then hosted on Indices’ infrastructure as APIs, available 24/7 for customers to invoke on demand.

Using network requests instead of visual automation makes the connectors fast enough for real-time applications like voice agents, and reliable enough even for critical enterprise workloads.

On standard cloud VMs, delivering that means fighting multi-second cold starts with a permanently warm pool of instances. You’ll have to do extra orchestration, superfluous spend, and lots of unforseen constraints. Moving the run path to Unikraft’s snapshot-based microVMs let Indices cold-boot on the hot path in about six milliseconds, delete the warm-pool logic entirely, and cut latency roughly in half.

APIs that feel like APIs

When a customer builds a connector on Indices, they expect it to behave like any other API. You call it, and you pretty much get an instant answer. Internally, though, each call is doing a great deal of work. It’s resolving the connector’s code, configures proxies, sends sequences of raw web requests and extracts data from the responses, resolves MFA and TOTP codes, solves CAPTCHAs, etc.

“The external surface is an API,” Co-Founder Lorian explains, “but internally it’s built on web requests. To make that feel like an API, we have to work quite hard to minimize latency.” Their aim is for simple APIs to complete the whole round trip in under a second.

Where standard cloud VMs fell short

Indices first built this on a well-known sandbox provider and it worked, but the constraints didn’t fit the hot path. Cold starts were close to two seconds, which is far too slow to sit in front of a live API call. The workaround was a permanently warm pool of instances, which has its own problems of course:

• Idle spend. Instances billed around the clock whether or not traffic is being served.
• Messy orchestration. Keeping a warm pool alive means complicated, brittle logic around the run path.
• Truncated output. Only the first 8 KB of stdout and stderr came back from an instance, and Indices frequently needed more. Logs had to be pushed to object storage from inside the box and pulled back down separately.

“A cold boot on the hot path was a non-starter. So we maintained a warm pool. It worked, but required messy logic, and we were paying to keep those instances running the whole time.”

— Lorian Richmond · Co-Founder, Indices

Why Unikraft is the solution

Unikraft completely eliminated the tradeoff Indices was stuck with. Instead of pre-warming instances to cover for slow starts, our microVMs start fast enough to launch inside the request itself. The warm pool (and its cost + overhead) simply goes away.

Cold boot on the hot path

Boot times land around six milliseconds, so Indices now cold-boots a fresh instance directly on the run path. No warm pool, no pre-provisioning, dramatically simpler code.

“The boot times we get are about six milliseconds, so cold-booting on the hot path is completely fine now. It’s really simplified the logic as we have no warm pool to maintain.”

— Lorian Richmond · Co-Founder, Indices

Snapshotting away the setup

But the fast boot time is only half the win. Every instance used to spend time importing dependencies and initialising before it could do work. With snapshotting, Indices freezes a fully initialised instance mid-execution and resumes from there, so imports and init happen ahead of time. For most connectors that roughly halves the latency of a call.

“Because of snapshotting, we can do the imports and initialisation ahead of time. For most connectors that’s cut the latency in half.”

— Lorian Richmond · Co-Founder, Indices

Cross-tenant isolation

Indices’ connectors handle sensitive customer data and credentials, so cross-tenant isolation is a must. Unikraft gives every call its own microVM behind a full hypervisor boundary with no shared kernel. Simply put, fewer moving parts in the boundary means fewer ways for it to be wrong.

“I come from a security background, and I’ve learned over time that good security goes hand-in-hand with simplicity. Full hypervisor-level sandboxing gives me peace of mind in a way that complicated process-level isolation never did.”

— Lorian Richmond · Co-Founder, Indices

A primitive worth building on

The combination of isolated instances, frozen at a point in execution, resumable in milliseconds, scaling to zero when idle turned out to be a general-purpose building block. Indices has since migrated another part of its infrastructure onto Unikraft and expects more to follow.

“Being able to spin up a completely isolated instance with your code frozen mid-execution, in a couple of milliseconds is a very powerful primitive. We’ll likely be running more of our platform on Unikraft in future.”

— Lorian Richmond · Co-Founder, Indices

Faster, cheaper, and a lot less to maintain

Working with the team

Beyond the platform, Indices points to the people. Bug reports and requests get turned around super quickly by engineers who work directly on the product. We communicate on Slack and are deep in the weeds with the implementation

“They’re very responsive! Any bug or request, they’re ready to fix it fast. We’re working directly with field engineers who are very technical themselves. There’s very little abstraction between us and the people running the platform.”

— Lorian Richmond · Co-Founder, Indices

Build faster infrastructure