Skip to main content
Hosts boot Firecracker microVMs, so they need KVM. Every node a host runs on must expose /dev/kvm. The server needs none of this and runs on any node.

Node requirements

Bare-metal nodes have KVM. Cloud VMs have it only when the instance type and hypervisor offer nested virtualization and it is enabled. Check from a shell on the node:

Label and taint the nodes

Label the KVM nodes so hosts schedule there, and taint them so nothing else does:
The chart’s defaults match:
For a managed node pool, set the label and taint on the pool, so every node it adds has them. With host autoscaling, the node autoscaler must launch nodes with them too. Anti-affinity keeps one host per node, and each host is named after its node. host.affinity is merged with that rule, so you can add zone or instance-type preferences without losing it.

How the host isolates microVMs

The host container runs as root with every capability. That privilege is what isolates the microVMs. Each one runs under Firecracker’s jailer:
  • chrooted into its own directory under the host’s data volume,
  • in its own cgroup, with cpu.max capped at its vCPUs,
  • as its own unprivileged uid, from a range of 65536 starting at UID_BASE (10000000).
An escaped guest lands in a process that sees only its own kernel and root filesystem and cannot signal or read any other microVM. See Isolation.

Resources and the memory budget

Guest memory and vCPUs are charged to the host pod, not to the host process. Size host.resources for the microVMs a host runs at once. Two settings bound what a host commits: A host with a budget stops its oldest warm instances to make room for a new one, and fails the start when only busy instances are left. Without a budget, nothing stops the microVMs from committing more memory than the pod’s limit, and the pod is OOM killed when they do. Set host.resources.limits.memory to cover host.memoryBudgetMiB plus the host process on top of it:
When memoryBudgetMiB is set and host.resources names no memory request or limit, the chart uses the budget as the pod’s memory request. The data volume is an emptyDir holding the kernel, the root filesystem cache and the jails. It must be one filesystem, since jails hard-link the kernel and root filesystems. Cap it with host.data.sizeLimit. Losing it only means rebuilding on the next invocation.

Images and registries

Hosts pull function and runtime images themselves, with no container daemon. ECR, Artifact Registry and GCR authenticate with the host ServiceAccount’s cloud identity. Other registries are pulled anonymously unless a Docker config on the host carries credentials. host.image.pullSecrets applies only to the host image itself.