Copy page

Tutorials

Rootfs Compression

Compressing the root file system represents using an algorithm like gzip, zstd, etc. to reduce the size of the rootfs on disk. This has the added benefit of reducing the time it takes to push the image to the Unikraft Cloud platform, as you transfer less data. It also means that the size on disk on the host side reduces, which can be beneficial when dealing with many images or large images. This size can drop depending on the contents of the rootfs and the compression algorithm used. For example, text files often compress by up to 90% and more, whilst already compressed files like images or videos don't compress well at all.

Getting started

The Unikraft Platform supports both compressed and uncompressed rootfses, but it's highly recommended to use uncompressed ones if size allows. Decompressing is a step done by the kernel at boot time (or runtime, with EROFS), resulting in slower access times across the board. That's why the first attempt at using the platform should always be with uncompressed rootfses.

Both formats, CPIO and EROFS, support compression. CPIO does full file system compression (like running zip on the file). EROFS does per-block compression meaning that the system decompresses only the accessed blocks on-the-fly. Both of them support most compression algorithms, but the recommended ones are those that have fast decompression speeds like lz4hc.

As presented in the rootfs formats tutorial and in the FAQ, the performance characteristics of the two formats differ. When using compression, the system amplifies these differences further. For CPIO, the system must decompress the entire archive first before using it. This means that boot times increase further, and memory usage during boot is also higher negating the benefit of using compression in the first place. EROFS is better, as the system decompresses only the accessed blocks on-the-fly, meaning that boot times are less affected, and memory usage is also kept lower. This has the added downside that Direct Access (DAX) isn't possible when using compression, as the host side can't map a compressed image into guest memory. This caveat results in having to load the entire compressed rootfs into guest memory, increasing memory usage by at least the size of the compressed rootfs. Still, if many files aren't accessed in boot, the cold boot time might decrease compared to an uncompressed EROFS rootfs.

Packaging a compressed rootfs

The unikraft CLI doesn't support compressing rootfses. This choice is deliberate, as the performance downsides aren't worth it most of the time. The legacy kraft CLI implements CPIO compression directly.

Both CLIs support deploying pre-compressed EROFS rootfses.

As a study case, this tutorial uses this node image, which has a big rootfs size. A full guide on using this image is available here.

CPIO compression

To package a compressed CPIO rootfs using the legacy CLI, you can use the --compress flag when running the kraft pkg or kraft cloud deploy command. This uses gzip compression to compress the initrd file before packaging it into a OCI image and uploading it to the Unikraft Cloud platform.

EROFS compression

To package a compressed EROFS rootfs, you need to use the mkfs.erofs tool before packaging it with the CLI. To do this you will also have to first export the image built from the Dockerfile into a tar archive:

Code
docker build -o type=tar,dest=rootfs.tar -t node-playwright-chromium -f Dockerfile .

You can then use the resulting tar archive to create a compressed EROFS rootfs using the following command. Note that you can change the compression algorithm and level by modifying the -z option. In this example the level is 9 (maximum) and the algorithm is lz4hc (high compression lz4):

Code
mkfs.erofs --all-root -b 4096 -d2 -E noinline_data -z lz4hc,9 rootfs.erofs --tar=rootfs rootfs.tar

Finally, you can package and deploy the compressed EROFS rootfs using the CLI:

Performance impact

You can see below the same node image packaged and deployed four times, with and without compression, and with both CPIO and EROFS formats. You can view the results in the table below, which show the image size, boot times, and minimum memory requirements for each case.

To check the image size, you can list your images:

rootfs Type	Compression	Image Size	Boot Time	Instance Memory Required
CPIO	No	916 MiB	1152.08 ms	2700 MiB
CPIO	Yes (gzip)	400 MiB	3479.96 ms	2200 MiB
EROFS	No	924 MiB	155.39 ms	1000 MiB
EROFS	Yes (lz4hc)	542 MiB	122.10 ms	600 MiB

As expected from the previous assumptions, the compressed CPIO rootfs boots slower than the uncompressed one. This happens due to the need to decompress the entire archive before use. EROFS performs better in both compressed and uncompressed forms, compared to CPIO. The compressed EROFS rootfs boots even faster than the uncompressed one. This is likely due to the reduced amount of data that the system reads from disk, resulting in faster access times despite the decompression overhead. This only happens in the cold boot scenario. In warm boots, the uncompressed EROFS rootfs tops the charts as the system needs no decompression at all, and wakeups happen faster.

When it comes to memory usage, you can see that the CPIO images require a lot more memory to boot. This is because the system needs to decompress the entire archive in memory before using it, resulting in higher memory usage during boot. Then, Unikraft Cloud reclaims that memory after boot, but the initial memory overhead is still higher than the uncompressed case. With EROFS, you don't see this limitation.

Conclusion

As a rule of thumb, use uncompressed rootfses whenever possible and try to always use EROFS over CPIO for better performance. If limited by size constraints, try using compressed EROFS rootfses with a fast decompression algorithm like lz4hc.

Learn more

• The rootfs formats tutorial for understanding the differences between CPIO and EROFS.
• The mkfs.erofs documentation for more information on creating compressed EROFS file systems.
• The CLI reference and the legacy CLI reference.
• The kraft pkg command reference for packaging images.
• The Kraftfile reference for specifying rootfs sources and formats.