Skip to content

Manual

Preparation

Enable Virtualization Extensions on Host

Install Supporting packages

sudo apt install \
  cpu-checker \
  ssh \
  mtools \
  e2tools \
  gzip \
  unzip \
  qemu-utils \
  dosfstools \
  e2fsprogs

Check Virtualization Capabilities

kvm-ok

Docker

The Docker Engine API is used for to interface with the container runtime.

Install

Install Docker Engine by following the official docs

Post Install

Add your username to the Docker group by following the official docs

QEMU/KVM/Libvirt

The Libvirt API is used to interface with Libvirt which manages VMs via QEMU/KVM.

Install

sudo apt install \
  qemu-kvm \
  libvirt-daemon-system \
  libvirt-clients \
  libvirt-dev \
  bridge-utils \
  virtinst \
  libosinfo-bin \
  libguestfs-tools \
  ovmf

Post Install

Add your username to the libvirt and kvm groups

sudo usermod -aG libvirt,kvm $USER

Set the default libvirt URI.

echo 'export LIBVIRT_DEFAULT_URI=qemu:///system' > ~/.zshrc

Transparent Hugepages

Let QEMU back VM memory with 2 MiB pages. Skip this if /sys/kernel/mm/transparent_hugepage/enabled already shows [always].

echo madvise | sudo tee /sys/kernel/mm/transparent_hugepage/enabled
echo 'w /sys/kernel/mm/transparent_hugepage/enabled - - - - madvise' | \
  sudo tee /etc/tmpfiles.d/sherpa-thp.conf

Networking

Bridge Interface

network:
  version: 2
  ethernets:
    eth1-iface:
      match:
        macaddress: 6c:0b:84:aa:68:4e
      set-name: eth1
      dhcp4: false
      dhcp6: false
  bridges:
    br-sherpa0:
      interfaces: [eth1-iface]
      parameters:
        stp: false
        forward-delay: 0

Sherpa

Sherpa Server

Users/Groups

Add sherpa user and group

sudo adduser --system \
  --group \
  --shell /usr/sbin/nologin \
  --no-create-home \
  sherpa

Add your username to the group. Members of the sherpa group can read the lab folders and the log, and can copy node images into /opt/sherpa/images. They cannot change anything sherpad runs or reads its secrets from.

sudo usermod -aG sherpa $USER

Create the sherpa directory, the folder of the binaries and the folder of the storage pool, owned by root. systemd runs sherpad from there, so neither sherpa nor its group may change them

sudo install -d -o root -g root -m 755 /opt/sherpa /opt/sherpa/bin /opt/sherpa/libvirt

Create the lab directory, readable only by the sherpa user and group

sudo install -d -o sherpa -g sherpa -m 2750 /opt/sherpa/labs

Create the database directory, readable only by the sherpa user and group, as the database holds password hashes

sudo install -d -o sherpa -g sherpa -m 750 /opt/sherpa/db

Create the secrets and certificate directories, which only the sherpa user can open. sherpad keeps its login token secret and its database key in the first, and its TLS certificate and key in the second

sudo install -d -o sherpa -g sherpa -m 700 /opt/sherpa/.secret /opt/sherpa/.certs

Create the folders sherpad writes its images, container and binary files, PID file and console sockets to. sherpad cannot create a folder in /opt/sherpa itself, as that is root's

sudo install -d -o sherpa -g sherpa -m 775 /opt/sherpa/images
sudo install -d -o sherpa -g sherpa -m 755 /opt/sherpa/containers /opt/sherpa/bins /opt/sherpa/run

Create the config, environment and log directories, open to the sherpa user and group only

sudo install -d -o sherpa -g sherpa -m 750 /opt/sherpa/config /opt/sherpa/env /opt/sherpa/logs

sherpad init, run as sherpa, writes /opt/sherpa/config/sherpa.toml with mode 600. If you write an environment file, /opt/sherpa/env/sherpa.env, make it readable by sherpa only, as it can hold secrets

sudo install -o sherpa -g sherpa -m 600 /dev/null /opt/sherpa/env/sherpa.env

See file permissions for every mode.

AppArmor

Update AppArmor policy to allow qemu to access config files.

sudo install -d -o sherpa -g kvm -m 2750 /opt/sherpa/libvirt/images
sudo sh -c 'echo "  # For ignition files" >> /etc/apparmor.d/abstractions/libvirt-qemu'
sudo sh -c 'echo "  /opt/sherpa/libvirt/images/** r," >> /etc/apparmor.d/abstractions/libvirt-qemu'
sudo systemctl restart libvirtd.service

Install the Binaries

Download sherpad and the sherpa CLI for the release you want from the releases page, and install them in /opt/sherpa/bin, owned by root, with links in /usr/local/bin. Set VERSION to the release's tag, such as v0.7.0.

VERSION=<release tag>
TARGET=x86_64-unknown-linux-gnu
cd "$(mktemp -d)"
for binary in sherpad sherpa; do
  curl -fsSLO "https://github.com/bwks/sherpa/releases/download/${VERSION}/${binary}-${TARGET}.tar.gz"
  tar xzf "${binary}-${TARGET}.tar.gz"
  sudo install -o root -g root -m 755 "${binary}" "/opt/sherpa/bin/${binary}"
  sudo ln -sf "/opt/sherpa/bin/${binary}" "/usr/local/bin/${binary}"
done
sherpad --version

Pull the Sherpa Router Image

Each lab's router, which serves DHCP, DNS and the nodes' boot files, runs from this image.

sudo docker image pull ghcr.io/bwks/sherpa-router:latest

Install the Service

sherpad runs under systemd as sherpa, with the network capabilities it needs.

sudo tee /etc/systemd/system/sherpad.service > /dev/null << 'UNIT'
[Unit]
Description=Sherpa Daemon - VM, Container, and Unikernel Management Server
Documentation=https://github.com/bwks/sherpa
After=network-online.target docker.service libvirtd.service
Wants=network-online.target
Requires=docker.service libvirtd.service
StartLimitBurst=5
StartLimitIntervalSec=60

[Service]
Type=simple
User=sherpa
Group=sherpa
WorkingDirectory=/opt/sherpa
AmbientCapabilities=CAP_NET_ADMIN CAP_NET_RAW CAP_BPF CAP_PERFMON
CapabilityBoundingSet=CAP_NET_ADMIN CAP_NET_RAW CAP_BPF CAP_PERFMON
ExecStart=/opt/sherpa/bin/sherpad start --foreground
Restart=on-failure
RestartSec=5
EnvironmentFile=-/opt/sherpa/env/sherpa.env
RuntimeDirectory=sherpa
RuntimeDirectoryMode=0700
RuntimeDirectoryPreserve=yes
NoNewPrivileges=no
PrivateTmp=no
LimitNOFILE=65536
TasksMax=4096

[Install]
WantedBy=multi-user.target
UNIT
sudo systemctl daemon-reload
sudo systemctl enable sherpad.service

PrivateTmp stays off so sherpad can read image files you import from /tmp.

Create Blank Disks

Some nodes boot from a copy of a blank disk in /opt/sherpa/images/blank_disk/. Once the Sherpa binaries are in /opt/sherpa/bin, run the update script for the version you installed: it finds nothing to install, and creates the blank disks that are missing, with their folder. It never changes a disk that is there, so it is safe to run again.

curl -fsSLO https://raw.githubusercontent.com/bwks/sherpa/refs/heads/main/scripts/sherpa_update.sh
sudo bash sherpa_update.sh --version <installed version>

Initialize and Start

Initialize the server as sherpa, with the address clients reach it on. It writes /opt/sherpa/config/sherpa.toml, creates the database, and asks for the admin password and for your own user: its name, password and SSH public key. sherpa cannot read your home folder, so paste the key itself when asked. See file permissions.

sudo -u sherpa -H sherpad init --server-ipv4 <server address>
sudo systemctl start sherpad

Lab management subnets come from 172.31.0.0/16 and fd00:b00b::/48. If either is in use on your network, add --management-prefix-ipv4 <prefix> or --management-prefix-ipv6 <prefix>.

Give your users the server's key pin, the SHA-256 of its public key, which clients ask them to check on their first connection; see Server Key Pin:

sudo -u sherpa -H sherpad status

Sherpa Client

Install the sherpa CLI as described in Client. The step above already installed it on the server. Before the first lab, each user sets a device password and registers an SSH key.