Autoscale
Limited Access
At the moment, autoscale is not enabled by default (you might get an "Autoscale not enabled for your account" error). If you would like to enable it, please reach out to the Unikraft Cloud Discord or send an email to support@unikraft.com.
Autoscaling is load balancing where the number of instances used to handle your traffic automatically adapts to match the current traffic load. On Unikraft Cloud, scale-out (the process of adding instances to cope with increased load) happens in milliseconds. You can transparently and effortlessly handle load increase including traffic peaks. No more headaches due to slow autoscale like keeping hot instances around to deal with peaks, coming up with complex predictive algorithms, or other painful workarounds. You can set autoscale on and let Unikraft Cloud handle your traffic increases and peaks.
The basics
As with load balancing, autoscaling in Unikraft Cloud takes care of a service. Services allow you to load balance traffic for an Internet-facing service like a web server by creating many instances within the same service.
While you can add or remove instances to a service to scale your service, doing this manually makes it hard to react to changes in traffic load. Keeping many instances running to cope with intermittent bursts would be wasteful and expensive. This is where autoscale comes into play.
With autoscale enabled, Unikraft Cloud takes care of the heavy lifting for you by continuously monitoring the load of your service and automatically creating or deleting instances as needed.
Autoscale, as well as load balancing in general, currently supports only Internet-facing services.
Setting up autoscale
Dedicated unikraft CLI subcommands for autoscale are coming soon.
In the meantime, drive autoscale through the API, which you can invoke with curl or the unikraft api command.
First, create an instance, in this example using NGINX:
The output shows the instance address and other details:
Autoscale requires an instance template to create new instances during scale-out.
If you enable autoscale through the API, provide the template under the top-level template field.
The next step is to convert the instance you deployed into a template that autoscale clones during scale-out:
unikraft
The instance's state changes to template, and its name (nginx-67zbu) is the <template-name> you reference below.
Converting the instance into a template consumes it, so the service that the deploy step created for it (nameless-fog-0tvh1uov) no longer exists.
Create a new persistent service for autoscale to manage:
unikraft
The command prints the new service's details, including its uuid (for example, 996d006f-a491-44ae-a1dc-7a6186971351).
Use that uuid—shown as <service-uuid> below—in the autoscale calls.
All that's left to do now to set up autoscale is to configure the autoscale settings (including the template to clone) and to define a scaling policy. Unikraft Cloud then takes care of cloning this template whenever load increases.
You can create the template with on-demand template creation instead of managing the template yourself. The instance-create call then prepares and reuses the underlying template automatically.
Achieve this with the following commands:
Note the following:
- The first command sets the template to clone, and configures it to scale up to a maximum of 8 instances and a minimum of 1; the command also sets the warm up and cool down time to 1 second each, so it doesn't constantly fluctuate up and down.
- The second command sets the scale-out policy based on CPU utilization (in millicores): between 60% and 80% utilization, the system increases instances by 50%. From 80% onward, the number of instances doubles.
- The third command sets the scale-in policy: below 50% utilization, the system reduces the number of instances by half (note the
-sign for scale-in).
The platform waits between decisions so the service doesn't constantly scale in and out.
The warm-up time is how long it waits after scaling out before it can scale again, and the cool-down time is how long it waits after scaling in.
Set them with the warmup_time_ms and cooldown_time_ms fields.
Refer to the API autoscale reference for more details.
Keep in mind that a few restrictions apply to how you define scale-in/scale-out steps.
To check it's working, list the autoscale properties of the service:
You should see output like:
To list an individual policy:
You should see output like:
Policy types
Four autoscale policy types are available. A service can have more than one policy active at the same time.
Step policy
The step policy scales instances based on metric thresholds.
You define up to 4 steps, each specifying a lower bound, upper bound, and the scaling change to apply when the metric falls in that range.
You should order steps by lower bound with no gaps between them and no overlaps:
The following metrics can drive a step policy:
| Metric | Description |
|---|---|
cpu | CPU utilization in millicores |
inflight_reqs | Number of requests the platform is processing across all instances |
reqs_per_sec | Request throughput in requests per second |
Step policies support three scaling change types:
| Type | Description |
|---|---|
change | Change the instance count by the specified value (positive to scale out, negative to scale in) |
exact | Set the instance count to exactly the specified value |
percent | Change the instance count by the specified percentage of the current count |
On-demand policy
The on-demand policy creates a new instance immediately when an incoming request finds no available instances.
This prevents request queuing but introduces cold start delays.
Create policy
The create policy provisions a new VM when an instance exceeds the num_requests threshold.
Setting replace to true deletes the original VM after the new one starts.
Idle policy
The idle policy scales in (removes instances) when the service has been idle—receiving no requests—for a configurable period.
Learn more
- The CLI reference.
- Unikraft Cloud's REST API reference, and in particular the section on autoscale.