Copy page

Features

Autoscale

Autoscaling is load balancing where the number of instances used to handle your traffic automatically adapts to match the current traffic load. On Unikraft Cloud, scale-out (the process of adding instances to cope with increased load) happens in milliseconds. You can transparently and effortlessly handle load increase including traffic peaks. No more headaches due to slow autoscale like keeping hot instances around to deal with peaks, coming up with complex predictive algorithms, or other painful workarounds. You can set autoscale on and let Unikraft Cloud handle your traffic increases and peaks.

The basics

As with load balancing, autoscaling in Unikraft Cloud takes care of a service. Services allow you to load balance traffic for an Internet-facing service like a web server by creating many instances within the same service.

While you can add or remove instances to a service to scale your service, doing this manually makes it hard to react to changes in traffic load. Keeping many instances running to cope with intermittent bursts would be wasteful and expensive. This is where autoscale comes into play.

With autoscale enabled, Unikraft Cloud takes care of the heavy lifting for you by continuously monitoring the load of your service and automatically creating or deleting instances as needed.

Setting up autoscale

First, create an instance, in this example using NGINX:

Code
git clone https://github.com/unikraft-cloud/examples
cd examples/nginx/
unikraft build . --output <my-org>/nginx:latest
unikraft run --metro fra \
  -m 256M \
  -p 443:8080/http+tls \
  --image <my-org>/nginx:latest

The output shows the instance address and other details:

Code
metro:        fra
name:         nginx-67zbu
uuid:         8a8bc1b9-0af6-420e-a426-190dc2da9eaa
state:        starting
image:        <my-org>/nginx
resources:
  memory:     256MiB
  vcpus:      1
service:
  uuid:       a942b9b5-ad17-3ffe-dcd2-ef4331f9087a
  name:       nameless-fog-0tvh1uov
  domains:
  - fqdn:     nameless-fog-0tvh1uov.fra.unikraft.app
networks:
- uuid:       62d9bbf0-aec8-61f6-7bdb-86edf63dd068
  private-ip: 10.0.3.3
  mac:        12:b0:c6:23:ed:15
timestamps:
  created:    just now

The next step is to convert the instance you deployed into a template that autoscale clones during scale-out:

Code
unikraft instance template create nginx-67zbu

The instance's state changes to template, and its name (nginx-67zbu) is the <template-name> you reference below.

Converting the instance into a template consumes it, so the service that the deploy step created for it (nameless-fog-0tvh1uov) no longer exists. Create a new persistent service for autoscale to manage:

Code
unikraft service create --metro fra \
  --name nginx-autoscale-service \
  --domain nginx-autoscale \
  --service 443:8080/http+tls

The command prints the new service's details, including its uuid (for example, 996d006f-a491-44ae-a1dc-7a6186971351). Use that uuid—shown as <service-uuid> below—in the autoscale calls.

All that's left to do now to set up autoscale is to configure the autoscale settings (including the template to clone) and to define a scaling policy. Unikraft Cloud then takes care of cloning this template whenever load increases.

Achieve this with the following commands:

Code
unikraft api /v1/services/<service-uuid>/autoscale -X POST --metro fra \
  '{
    "min_size": 1,
    "max_size": 8,
    "warmup_time_ms": 1000,
    "cooldown_time_ms": 1000,
    "template": {
      "name": "<template-name>"
    }
  }'

unikraft api /v1/services/<service-uuid>/autoscale/policies -X POST --metro fra \
  '{
    "name": "scale-out-policy",
    "type": "step",
    "metric": "cpu",
    "adjustment_type": "percent",
    "steps": [
      { "lower_bound": 600, "upper_bound": 800, "adjustment": 50 },
      { "lower_bound": 800, "adjustment": 100 }
    ]
  }'

unikraft api /v1/services/<service-uuid>/autoscale/policies -X POST --metro fra \
  '{
    "name": "scale-in-policy",
    "type": "step",
    "metric": "cpu",
    "adjustment_type": "percent",
    "steps": [
      { "upper_bound": 500, "adjustment": -50 }
    ]
  }'

Note the following:

• The first command sets the template to clone, and configures it to scale up to a maximum of 8 instances and a minimum of 1; the command also sets the warm up and cool down time to 1 second each, so it doesn't constantly fluctuate up and down.
• The second command sets the scale-out policy based on CPU utilization (in millicores): between 60% and 80% utilization, the system increases instances by 50%. From 80% onward, the number of instances doubles.
• The third command sets the scale-in policy: below 50% utilization, the system reduces the number of instances by half (note the - sign for scale-in).

High watermark

An autoscale group can be configured with a high watermark - as an early warning that it's getting close to its maximum size. Set it through the high_watermark field, as a fraction of max_size between 0 and 1. The default is 0, which turns the signal off.

Once a service crosses its watermark, the platform can report this on the responses it sends back. A client that spreads load across metros uses this to steer new work elsewhere before the service fills up. Your metro operator switches that response header on.

To check it's working, list the autoscale properties of the service:

Code
unikraft api /v1/services/<service-uuid>/autoscale -X GET --metro fra

You should see output like:

Code
{
  "status": "success",
  "data": {
    "service_groups": [
      {
        "status": "success",
        "uuid": "5ca059ec-a24a-41f2-8413-f09bc58730ca",
        "name": "nameless-fog-0tvh1uov",
        "enabled": true,
        "min_size": 1,
        "max_size": 8,
        "warmup_time_ms": 1000,
        "cooldown_time_ms": 1000,
        "template": {
          "name": "nginx-67zbu",
          "uuid": "8a8bc1b9-0af6-420e-a426-190dc2da9eaa"
        },
        "policies": [
          {
            "name": "scale-out-policy",
            "type": "step",
            "enabled": true,
            "metric": "cpu",
            "adjustment_type": "percent",
            "steps": [
              {
                "adjustment": 50,
                "lower_bound": 600,
                "upper_bound": 800
              },
              {
                "adjustment": 100,
                "lower_bound": 800
              }
            ]
          },
          {
            "name": "scale-in-policy",
            "type": "step",
            "enabled": true,
            "metric": "cpu",
            "adjustment_type": "percent",
            "steps": [
              {
                "adjustment": -50,
                "upper_bound": 500
              }
            ]
          }
        ]
      }
    ]
  }
}

To list an individual policy:

Code
unikraft api /v1/services/<service-uuid>/autoscale/policies/scale-out-policy -X GET --metro fra

You should see output like:

Code
{
  "status": "success",
  "data": {
    "policies": [
      {
        "status": "success",
        "policy": {
          "name": "scale-out-policy",
          "type": "step",
          "enabled": true,
          "metric": "cpu",
          "adjustment_type": "percent",
          "steps": [
            {
              "adjustment": 50,
              "lower_bound": 600,
              "upper_bound": 800
            },
            {
              "adjustment": 100,
              "lower_bound": 800
            }
          ]
        }
      }
    ]
  }
}

Policy types

Four autoscale policy types are available. A service can have more than one policy active at the same time.

Step policy

The step policy scales instances based on metric thresholds. You define up to 4 steps, each specifying a lower bound, upper bound, and the scaling change to apply when the metric falls in that range.

You should order steps by lower bound with no gaps between them and no overlaps:

Code
unikraft api /v1/services/<service-uuid>/autoscale/policies -X POST --metro fra \
  '{
    "name": "my-step-policy",
    "type": "step",
    "metric": "cpu",
    "adjustment_type": "change",
    "steps": [
      { "lower_bound": 600, "upper_bound": 800, "adjustment": 2 },
      { "lower_bound": 800, "adjustment": 4 }
    ]
  }'

The following metrics can drive a step policy:

Step policies support three scaling change types:

On-demand policy

The on-demand policy creates a new instance immediately when an incoming request finds no available instances. This prevents request queuing but introduces cold start delays.

Create policy

The create policy provisions a new VM when an instance exceeds the num_requests threshold. Setting replace to true deletes the original VM after the new one starts.

Idle policy

The idle policy scales in (removes instances) when the service has been idle—receiving no requests—for a configurable period.

Learn more

• The CLI reference.
• Unikraft Cloud's REST API reference, and in particular the section on autoscale.