Network Shield and Relay
2 notes
- Network shield and relay are available to enterprise customers. Contact support to try them. support@unikraft.com.
- CLIDedicated CLI subcommands are coming soon. Until then, configure the shield and the relay through the API, for example with the unikraft api command. Create instance API.
A relay redirects all network traffic to and from a network interface toward another network interface. You can set up a relay VM and configure its interface as the relay for another VM's interface, so that all its traffic transparently passes through the relay.
The network shield
The network shield is an example of a relay: an instance inserted between a microVM and the public internet. It sits directly in the path, as a bump on the wire, so every packet the workload sends or receives goes through it first. Since many instances can specify the same microVM as their relay, many sandboxes can share the same network shield.
A microVM every packet passes through, which filters traffic and injects secrets
Third-party APIs, package registries
The shield provides filtering capabilities but also the possibility to inject credentials/secrets for agentic or other use cases. This is useful in sandbox environments, as it means the sandbox won't need unfiltered network access or access to secrets.
Each shield runs as its own microVM. This means each shield:
- Can be scaled to zero and woken up when it receives traffic.
- Is strongly isolated from the microVMs running the workloads, and also from other network shields.
- Is fully flexible and extensible in the capabilities it provides. The image it uses determines its behavior.
The official shield runs the netshield plugin, which serves an API for its policies.
A policy allows traffic by host (wildcards included), network, port, or protocol.
It can also intercept TLS, rate-limit requests and connections, and inject secrets into outgoing headers, so a sandbox sends {{GITHUB_TOKEN}} and never sees the real value.
The shield needs a certificate authority in CA_CERT and CA_KEY for TLS interception, which every sandbox behind it has to trust.
It starts closed, and a shield whose policy fails to apply stays closed, so replace it.
On Kubernetes, Kraftlet runs a shield as a sidecar container of a Pod.
Certificate authority
A policy with the https handler intercepts TLS.
The shield ends the TLS connection of the sandbox, so that it can read the request, limit requests, and add secrets.
It shows the sandbox a certificate for the host, signed with a certificate authority (CA) that you supply.
The http and passthrough handlers don't intercept TLS.
The shield doesn't start without the CA, even when no policy intercepts TLS.
Pass the certificate in CA_CERT and the private key in CA_KEY, both in PEM format with their line breaks.
Create a certificate authority
Create a CA for the shield alone:
Code
Give ca.key only to the shield, and keep it out of sandbox images.
Anyone with the key can create a certificate for any host, and every sandbox that trusts the CA accepts it.
For the same reason, don't use a CA that other systems trust.
Trust the CA in a sandbox
Every sandbox behind the shield has to trust ca.crt.
Otherwise, its connections through an https policy fail with a certificate error.
On Debian and Ubuntu, update-ca-certificates adds every file under /usr/local/share/ca-certificates/, in subdirectories too, whose name ends in .crt to the system trust store.
You can put the certificate there without rebuilding the image, with an inline ROM that you add to the sandbox when you create it.
The relay example puts it at /usr/local/share/ca-certificates/shield/ca.crt in the official Debian image.
Then, in the sandbox, install the ca-certificates package, which runs update-ca-certificates.
In an image that already has the package, run update-ca-certificates yourself.
In an image of your own, copy the certificate there when you build it:
Code
Tools that use the system trust store, such as curl, wget, and Go programs, then trust the shield. Some runtimes keep their own list of certificate authorities and ignore the system trust store. Point them at the certificate as well:
| Runtime | Setting |
|---|---|
| Node.js | NODE_EXTRA_CA_CERTS, set to the path of ca.crt |
Python requests | REQUESTS_CA_BUNDLE=/etc/ssl/certs/ca-certificates.crt, the system bundle that update-ca-certificates writes |
| Java | keytool -importcert -cacerts -alias network-shield -file ca.crt -storepass changeit -noprompt |
Relay
To use a relay, reference another interface by name inside a relay object.
The following example puts a sandbox behind the official network shield, which takes two interfaces:
- The first interface is on the platform's network, and the shield serves its API there.
- The second interface has a name, and sandboxes relay through it.
The
sandbox_ifacesetting of thenetshieldplugin names iteth1.
The shield runs a proxy, a DNS resolver, and its API, so it needs more than the default 128 MB of memory, and the example gives it 1024 MB.
The shield reads its certificate authority from ca.crt and ca.key.
jq puts both files in the request with their line breaks, and @- makes unikraft api read the request from standard input.
The sandbox runs the official Debian image, with the sandbox plugin so that you can run commands in it.
An inline ROM puts ca.crt at /usr/local/share/ca-certificates/shield/ca.crt, so that the sandbox can trust the CA.
The sandbox gets 512 MB of memory, which apt-get needs.
The shield starts closed, so the sandbox can't reach anything yet.
To let traffic out, send policies to the netshield plugin of the shield.
These policies allow the following traffic:
allow-debian: plain HTTP to the Debian package mirror, forapt-get.allow-github: HTTPS to GitHub, which the shield intercepts.sandbox-plugin: the platform's connections to the sandbox plugin, whichunikraft instance execuses. An inbound policy names the sandbox by its private name,<name>.internal, so send it after you create the sandbox.
Code
Install ca-certificates and curl in the sandbox.
The package runs update-ca-certificates, which adds the shield's CA from the ROM to the system trust store:
Code
curl then trusts the certificate that the shield shows for GitHub:
Code
Relay builds on custom network configuration: the relay object is a field of a network_interfaces entry, so the same permission and limitations apply.
The relay object accepts the following fields:
| Field | Required | Description |
|---|---|---|
name | Yes | The name of the interface to relay traffic through, which must belong to another instance in your account. |
relay_dns | No | Whether the relay forwards DNS requests, true by default, with false leaving them to the default DNS server. |
Relay status
The instance status shows the relay of each interface that uses one, and the instance that the relay interface belongs to.
unikraft instance get doesn't show relays, so read the status from the API:
Code
Response (excerpt)
Relay notes
-
All network traffic passes over the relay. Many VMs can share a single relay.
-
Both ends of a relay have to be interfaces that Unikraft Cloud allocates, meaning entries where you set no
ip,mac, ortap_nameof your own. -
You don't need to enable VM-to-VM communication to use this feature (which you don't want in a sandbox environment).
-
DNS requests can be either passed to the relay or the default DNS server. This is to simplify setups where security isn't the focus as you don't need to run a DNS server in the relay VM.
-
The guest acting as a relay VM should have net.ipv4.ip_forward=1 and rp_filter=0 set.
Error handling
| Error message | Cause |
|---|---|
Unknown member 'network_interfaces' | Your account lacks the net_manager permission required for this feature. |
Invalid relay '<interface>'. Please make sure the relay exists and does not create a circular dependency | The interface you named doesn't exist, already relays through another interface, or is itself a custom interface. |
Network interface without an IP address cannot use a relay | You configured a relay on an interface without an ip, and the relay datapath keys on the client interface's address. |
Limitations
- The feature requires the
net_managerpermission (see the note at the top of this page). - Creating a template from an instance drops its network configuration, since binding a template to a specific TAP device or IP defeats the purpose of cloning many instances from it.
- The create call sets the configuration once, and no PATCH support exists yet.
- Both ends of a relay have to be interfaces that Unikraft Cloud allocates, meaning entries where you set no
ip,mac, ortap_nameof your own. - Neither direction of a relay chain works. An interface that already relays can't serve as a relay, and one serving as a relay for others can't take a relay itself.
- Every custom network configuration limitation applies here too, since relay is part of that feature.
Learn more
- Custom network configuration: the interface, gateway, and nameserver settings that relay builds on.
- Instances: how instances work, from their lifecycle to their configuration.
- Sandboxes: run untrusted code in a hardware-isolated microVM.
- Kubernetes: a shield per Pod on a Kraftlet node.
- Unikraft Cloud's REST API reference, in particular the section on instances.