Virtual Machine¶
Virtual machines are supported via the KVM hypervisor using the QEMU emulator and the Libvirt virtualisation API.
How it Works¶
When a lab is created, Sherpa performs the following for each VM node:
- Create the boot disk in the storage pool: a QCOW2 overlay on the base image by default, or a full copy when
boot_disk.clone_method = "copy" - Resize the boot disk if
boot_disk.sizeis specified in the manifest - Generate a libvirt domain XML definition from the node's parameters
- Apply zero-touch provisioning configuration (cloud-init ISO, ZTP scripts, etc.)
- Define and start the domain via the Libvirt API
Each VM runs as a persistent libvirt domain under KVM/QEMU.
Resource Allocation¶
Every node model has default resource parameters that can be overridden in the manifest. Typical defaults:
| Resource | Range | Examples |
|---|---|---|
| CPU | 1–4 vCPUs | Cisco ASAv: 1, Arista vEOS: 2, Cisco Cat9000v: 4 |
| RAM | 2048–20480 MiB | Cisco ASAv: 2048, Cisco CSR1000v: 3072, Cisco XRv9000: 20480 |
| Machine Type | pc, q35 variants |
Most use pc, some require specific q35 versions |
| BIOS | SeaBIOS or UEFI | Most use SeaBIOS; XRv9000, Nexus 9300v use UEFI |
CPU mode defaults to host-model, which auto-detects the host CPU capabilities. Nested virtualisation is supported via the VMX CPU flag when required.
Performance Tuning¶
A node's cpu, numa, boot_disk and interfaces settings
can tune it further:
- give it the host CPU exactly (host-passthrough) and a CPU topology;
- pin its vCPUs, and QEMU's own threads, to host CPUs;
- hold its memory on one NUMA node, or prefer one NUMA node;
- set its disk cache and I/O mode, and give its disks an I/O thread;
- add virtio-net queues to its interfaces.
Sherpa refuses a node whose pinned CPUs overlap another node's, in any lab on the server.
A node held strictly on one NUMA node never uses another node's memory, so the server also
refuses it when that NUMA node does not have room for it. A node that only prefers a NUMA node
(numa.memory_mode = "preferred") uses other nodes' memory when its node is full, and is only
checked against the whole server.
Memory¶
A virtual machine takes the memory it is configured with as the guest touches it, not when it
starts. The server therefore checks memory before it starts virtual machines and unikernels:
their configured memory, plus that of every running virtual machine and unikernel and of the
nodes other requests are starting, must fit in the server's memory less a host reserve. See
[host_memory] for the reserve, the overcommit
percentage (overcommit_percent) and the message a refusal gives.
Guest memory can also be backed by 2 MiB huge pages through the host's transparent hugepages.
QEMU already asks for them, so this needs no manifest setting; the host's
/sys/kernel/mm/transparent_hugepage/enabled must be madvise or always. The
installer sets madvise unless it
is already always.
Disk Management¶
Base images are imported into /opt/sherpa/images/. When a lab starts, each VM's boot disk is created in the Sherpa storage pool (sherpa-pool, at /opt/sherpa/libvirt/images/) as a QCOW2 overlay on the base image, or as a full copy when boot_disk.clone_method = "copy". See Boot Disk Overlays. The original base image is never modified. ZTP configuration disks, such as cloud-init ISOs, are created in the same pool.
Supported disk formats: qcow2, raw, iso, img.
Disk drivers vary by model:
| Driver | Usage |
|---|---|
virtio |
Default for most models |
sata |
Used by some models that lack virtio support |
ide |
Legacy models |
Boot disk resizing increases the volume capacity via the Libvirt storage API. The new size must be larger than the current allocation.
Networking¶
VM network interfaces are attached as libvirt network devices. The interface driver is configurable per model:
| Driver | Usage |
|---|---|
virtio |
Default for most models |
e1000 |
Models that require Intel NIC emulation |
vmxnet3 |
VMware-compatible models |
Each VM is connected to the management network on its first interface. Additional interfaces are attached for data plane links as defined in the manifest. Each end of a point-to-point link is a tap device joined to its peer by an eBPF redirect program.
Console Access¶
Each VM provides two console access methods:
Serial Console¶
The serial console of each VM is a UNIX socket that libvirt binds under /opt/sherpa/run/console/, in a folder only the sherpa account, the kvm group and root can reach. Connect with sherpa console <node> from any machine with the CLI. The connection goes through sherpad and only the lab's owner can open it. See Serial Console and sherpa console.
VNC¶
Each VM has a VNC graphics adapter. QEMU listens for it on a UNIX socket under /opt/sherpa/run/console/, next to the serial console. Connect a VNC viewer with sherpa vnc <node> from any machine with the CLI. The connection goes through sherpad and only the lab's owner can open it. See VNC and sherpa vnc.
Lifecycle¶
| Operation | Description |
|---|---|
| Create | Domain XML is defined and the VM is started |
| Stop | VM is shut down via Libvirt |
| Resume | Stopped VM is restarted |
| Redeploy | VM is destroyed, disks are deleted, and the node is recreated with fresh configuration |
| Destroy | Domain is undefined and all associated disks are removed |
Power behaviour:
- On poweroff — VM is destroyed
- On reboot — VM is restarted
- On crash — VM is destroyed
Additional Features¶
- Watchdog only when asked — Sherpa adds a hardware watchdog only when the node sets
vm.watchdog. (q35VMs always have the chipset's iTCO watchdog, which libvirt adds; it does nothing unless the guest starts it.) Cisco IOS images arm one early in boot and feed it only after a long init phase, so under nested virtualization aresetwatchdog resets them in a loop - QEMU Guest Agent — virtio channel for host-guest communication
- NVRAM — UEFI VMs store NVRAM at
/var/lib/libvirt/qemu/nvram/ - Windows support — Hyper-V enlightenments (relaxed, vapic, spinlocks, vpindex, runtime, synic, stimer, frequencies, tlbflush, ipi) and the Hyper-V clock are enabled for Windows VMs