Copy page

Features

Autoscale

Autoscaling is load balancing where the number of instances used to handle your traffic automatically adapts to match the current traffic load. On Unikraft Cloud, scale-out (the process of adding instances to cope with increased load) happens in milliseconds. You can transparently and effortlessly handle load increase including traffic peaks. No more headaches due to slow autoscale like keeping hot instances around to deal with peaks, coming up with complex predictive algorithms, or other painful workarounds. You can set autoscale on and let Unikraft Cloud handle your traffic increases and peaks.

The basics

As with load balancing, autoscaling in Unikraft Cloud takes care of a service. Services allow you to load balance traffic for an Internet-facing service like a web server by creating many instances within the same service.

While you can add or remove instances to a service to scale your service, doing this manually makes it hard to react to changes in traffic load. Keeping many instances running to cope with intermittent bursts would be wasteful and expensive. This is where autoscale comes into play.

With autoscale enabled, Unikraft Cloud takes care of the heavy lifting for you by continuously monitoring the load of your service and automatically creating or deleting instances as needed.

Setting up autoscale

First, create an instance, in this example using NGINX:

[●] Deployed successfully!
 │
 ├────────── name: nginx-4d7u3
 ├────────── uuid: 8fda2a70-6a32-4b5e-8900-4395b33d02d7
 ├───────── state: running
 ├─────────── url: https://small-leaf-rafirkw7.fra.unikraft.app
 ├───────── image: nginx@sha256:389bfa6be6455c92b61cfe429b50491373731dbdd8bd8dc79c08f985d6114758
 ├───── boot time: 20.36 ms
 ├──────── memory: 256 MiB
 ├─────── service: small-leaf-rafirkw7
 ├── private fqdn: nginx-4d7u3.internal
 ├──── private ip: 172.16.6.5
 └────────── args: /usr/bin/nginx -c /etc/nginx/nginx.conf

This single deploy or run flow does 3 things:

1. Creates an instance of NGINX which will serve as the autoscale master instance.
2. Creates a service via the -p flag (named small-leaf-rafirkw7).
3. Attaches the instance to the service (the -p flag also does this automatically).

All that's left to do now to set up autoscale is to set an autoscale configuration policy and to set the instance as master. Unikraft Cloud then takes care of cloning this master instance whenever load increases. To achieve this, use the legacy CLI scale command:

Note the following:

• The first command sets the master to the created instance, and configures it to scale up to a maximum of 8 instances and a minimum of 1; the command also sets the warm up and cool down time to 1 second each, so it doesn't constantly fluctuate up and down.
• The second command sets the scale-out policy based on CPU utilization (in millicores): between 60% and 80% utilization, the system increases instances by 50%. From 80% onward, the number of instances doubles.
• The third command sets the scale-in policy: below 50% utilization, the system reduces the number of instances by half (note the - sign for scale-in).

Testing it

To check it's working, you can use the legacy CLI scale get command to list the autoscale properties of the service:

You should see output like:

          uuid: 5ca059ec-a24a-41f2-8413-f09bc58730ca
          name: small-leaf-rafirkw7
       enabled: true
      min size: 0
      max size: 8
   warmup (ms): 1000
 cooldown (ms): 1000
        master: f840ac12-f485-4f02-9f33-6a0a7de46f1f
      policies: scale-out-policy;scale-in-policy

To list an individual policy, use the legacy CLI scale get command as follows:

You should see output like:

adjustment_type: percent
enabled: true
metric: cpu
name: scale-out-policy
status: success
steps:
- adjustment: 50
  lower_bound: 600
  upper_bound: 800
- adjustment: 100
  lower_bound: 800
type: step

You can further check that the master instance is on standby (scaled to zero), assuming your service hasn't received any traffic yet. You can get the UUID of your master instance from the legacy CLI scale get command above.

You should see output like:

          uuid: f840ac12-f485-4f02-9f33-6a0a7de46f1f
          name: nginx-9mbf2
          fqdn: restless-resonance-0oo7m7s8.fra.unikraft.app
    private ip: 172.16.6.4
         state: standby
    created at: 30 minutes ago
         image: nginx@sha256:d4325c1f1a472c511723148adc380d491029f4c98a2367fbeff628c6456d4180
        memory: 256 MiB
          args: /usr/bin/nginx -c /etc/nginx/nginx.conf
           env:
       volumes:
      service : 5ca059ec-a24a-41f2-8413-f09bc58730ca
     boot time: 19465us

Note the value of the state field. Now to make sure the service is up, curl the service address:

curl https://small-leaf-rafirkw7.fra.unikraft.app

You should get an immediate response, even though the instance was on standby. You can use a watch command to see if you catch the instance changing state from standby to running:

Policy types

Four autoscale policy types are available. A service can have more than one policy active at the same time.

Step policy

The step policy scales instances based on metric thresholds. You define up to 4 steps, each specifying a lower bound, upper bound, and the scaling change to apply when the metric falls in that range.

You should order steps by lower bound with no gaps between them and no overlaps.

kraft cloud scale add <service-name> \
  --name my-step-policy \
  --type step \
  --metric cpu \
  --adjustment-type change \
  --step 600:800/2 \
  --step 800:/4

Metrics

The following metrics can drive a step policy:

Metric	Description
cpu	CPU utilization in millicores
inflight_reqs	Number of requests the platform is processing across all instances
reqs_per_sec	Request throughput in requests per second

Scale change types

Step policies support three scaling change types:

Type	Description
change	Change the instance count by the specified value (positive to scale out, negative to scale in)
exact	Set the instance count to exactly the specified value
percent	Change the instance count by the specified percentage of the current count

On-demand policy

The on-demand policy creates a new instance immediately when an incoming request finds no available instances. This prevents request queuing but introduces cold start delays.

POST /services
{
  ...
  "policies": [
    {
      "name": "scale-out",
      "type": "on_demand"
    }
  ]
  ...
}

Create policy

The create policy provisions a new VM when an instance exceeds the num_requests threshold. Setting replace to true deletes the original VM after the new one starts.

POST /services
{
  ...
  "policies": [
    {
      "name": "create",
      "type": "create",
      "replace": true,
      "num_requests": 100
    }
  ]
  ...
}

Idle policy

The idle policy scales in (removes instances) when the service has been idle—receiving no requests—for a configurable period.

POST /services
{
  ...
  "policies": [
    {
      "name": "scale-in",
      "type": "idle",
      "idle_time_ms": 1000
    }
  ]
  ...
}

Learn more

• The CLI reference and the legacy CLI reference.
• Unikraft Cloud's REST API reference, and in particular the section on autoscale.