/dev/kvm. The server needs none of this and runs on any node.
Node requirements
Bare-metal nodes have KVM. Cloud VMs have it only when the instance type and hypervisor offer nested virtualization and it is enabled. Check from a shell on the node:
Label and taint the nodes
Label the KVM nodes so hosts schedule there, and taint them so nothing else does:host.affinity is merged with that rule, so you can add zone or instance-type preferences without losing it.
How the host isolates microVMs
The host container runs as root with every capability. That privilege is what isolates the microVMs. Each one runs under Firecracker’s jailer:- chrooted into its own directory under the host’s data volume,
- in its own cgroup, with
cpu.maxcapped at its vCPUs, - as its own unprivileged uid, from a range of 65536 starting at
UID_BASE(10000000).
Resources and the memory budget
Guest memory and vCPUs are charged to the host pod, not to the host process. Sizehost.resources for the microVMs a host runs at once.
Two settings bound what a host commits:
A host with a budget stops its oldest warm instances to make room for a new one, and fails the start when only busy instances are left. Without a budget, nothing stops the microVMs from committing more memory than the pod’s limit, and the pod is OOM killed when they do.
Set
host.resources.limits.memory to cover host.memoryBudgetMiB plus the host process on top of it:
memoryBudgetMiB is set and host.resources names no memory request or limit, the chart uses the budget as the pod’s memory request.
The data volume is an emptyDir holding the kernel, the root filesystem cache and the jails. It must be one filesystem, since jails hard-link the kernel and root filesystems. Cap it with host.data.sizeLimit. Losing it only means rebuilding on the next invocation.
Images and registries
Hosts pull function and runtime images themselves, with no container daemon. ECR, Artifact Registry and GCR authenticate with the host ServiceAccount’s cloud identity. Other registries are pulled anonymously unless a Docker config on the host carries credentials.host.image.pullSecrets applies only to the host image itself.