Panic at the Dinner Table: Which Fork Do I Use??

In the sandbox world, “fork” means copying a running VM. On Unikraft, that’s a branch operation. We have fork too: the real one, with parents, children and a return value that tells them apart. Here’s why we use two words.

We come from systems engineering. To us, fork isn’t a metaphor for “make a copy”. It’s a system call with more than fifty years of semantics behind it, and every Unix programmer has this burned into their brains.

So when we built a way to copy a live microVM, we didn’t call it fork. And when we later built something that actually behaves like fork, we did!

Let’s have a look at the man page then

fork(2)                  System Calls Manual                  fork(2)

NAME
     fork - create a child process

SYNOPSIS
     #include <unistd.h>

     pid_t fork(void);

DESCRIPTION
     fork() creates a new process by duplicating the calling
     process.  The new process is referred to as the child
     process.  The calling process is referred to as the
     parent process.

RETURN VALUE
     On success, the PID of the child process is returned in
     the parent, and 0 is returned in the child.

Three things make it fork:

1. The caller asks for it. Fork is initiated from the inside, by the process that’s about to be duplicated. This is different from the sandbox “fork”, that is requested from the outside, via an API, and retains none of the semantics of an actual fork after that initial operation.

2. Both sides resume from the same point. Same memory, same instruction, two copies.

3. Both sides know which one they are. The parent gets the child’s ID, the child gets zero. Everything built on fork, from pre-fork servers to supervisor trees, depends on that relationship.

The clueless copy

Most sandbox platforms use “fork” for something else: Take a running VM, snapshot it, start a second one from the snapshot. Definitely useful (not for nothing we built that functionality). But it’s triggered from outside, neither VM is told, and once the copy is running there’s no relationship between them. Two peers that happen to share a past.

That’s not fork(2), and so we named it a branch and made it a one flag difference on unikraft run. Memory and volumes are snapshotted copy-on-write, the source pauses for milliseconds, and the branch boots from exactly the same state. This performs rather well, even at large scale (e.g., 1000s of microVMs on a server): even in the extreme case of fork…errr…branching a 1TB Postgres database microVM, the operation took between 100-200 milliseconds.

$ unikraft run --metro fra -p 5432:5432/tls \
  --branch pg-prod
→ snapshot memory + volumes, copy-on-write
✓ branch running · pg-prod untouched

The mechanism is useful for many use cases, including launching sub-agents, making copies of prod databases to safely develop on them, or spinning up a preview environment. You can just throw the branch away when finished, no harm done.

fork(2), for virtual machines

Our fork is called by the guest, the way a process calls fork(2). Every instance has a small filesystem at /uk/libukp. Write to fork and the call blocks until the platform has created a child VM. Parent and child both resume from the same memory state. Then each reads the same file and gets its answer.

#!/bin/sh
# Fork this instance. Blocks until the platform is done.
echo 1 > /uk/libukp/fork

id=$(cat /uk/libukp/fork)

if [ "$id" = "00000000-0000-0000-0000-000000000000" ]; then
# child: same memory, new microVM
exec ./worker
else
# parent: id is the child's UUID
ls /uk/libukp/vm/$id/
fi

The parent gets the child’s UUID. The child gets a zero UUID. Each child appears under /uk/libukp/vm/<uuid>/, so the parent can track the instances it created. Same contract as fork(2), with a VM boundary where the process boundary used to be.

A name is also a contract

When you build on a primitive, its name tells you what you can rely on. If fork means fork, you can bring patterns you already know to VMs: Worker pools, pre-fork servers, a supervisor that fans out sub-agents and tracks every one. If fork just means “copy”, that’s a misleading name at best and leading you down a time-wasting rabbit hole at worst.

None of this makes branching the lesser feature. It’s what most teams need, and we do it in about ten milliseconds, at a density other platforms don’t reach. As system engineers (read: geeks) we felt it deserved its own name — not to tread all over the mighty fork primitive.

Summing up

A branch is a new VM made from a snapshot of another. A fork creates a child, and parent and child stay related.

We kept fork for what it means in an OS, and gave the other thing its own name. Both happily co-exist on our platform.

Branching docs · Forking docs