Pertisk

Documentation

Pertisk KOS Docs

Immutable, API-only Kubernetes node OS with a management plane for Proxmox, ESXi, and Nutanix AHV.

Project overviewGitHubLive demo

Overview

Pertisk KOS is an immutable, API-only Kubernetes node OS written in Rust. Nodes boot a locked-down Linux image whose only job is to run Kubernetes — managed through a typed API, not SSH.

  • Node OS — pertiskd as PID 1, gRPC management (pertiskctl), containerd + kubelet, serial console dashboard
  • Management plane — pertisk-mgmt (API + React UI) creates and operates HA clusters on Proxmox, standalone ESXi, and Nutanix AHV
pertiskctl / mgmt UI ──gRPC mTLS──► pertiskd (PID 1) ──► containerd + kubelet
pertisk-mgmt ──HTTPS──► Proxmox API / ESXi SOAP / Nutanix Prism REST

Quick start — management UI

export MGMT_ADMIN_USER=admin
export MGMT_ADMIN_PASSWORD=admin
export MGMT_SECRET_KEY=$(openssl rand -hex 32)

make mgmt
./out/bin/pertisk-mgmt --listen 0.0.0.0:8080 --db ./data/mgmt.db
# open http://127.0.0.1:8080

Dev with UI hot reload:

MGMT_ADMIN_PASSWORD=admin cargo run -p pertisk-mgmt -- --listen 127.0.0.1:8080
cd web/mgmt-ui && npm run dev   # :5173 proxies /api → :8080

Lab host deploy (images + mgmt RPM):

./deploy-h255.sh
# or: ARCH=amd64 ./deploy-h255.sh

Quick start — lab clusters

make cloud ARCH=amd64
make pertiskctl

# Proxmox
export PROXMOX_SSH=root@<pve>
./scripts/proxmox-lab-up.sh \
  --controlplanes 3 --workers 3 \
  --vip <free-ip> --cni cilium

# ESXi
# ./scripts/vsphere-lab-up.sh …

# Nutanix AHV (Prism Element)
export NUTANIX_URL=https://<prism>:9440
export NUTANIX_USER=admin
export NUTANIX_PASSWORD='…'
export NUTANIX_STORAGE=SelfServiceContainer
export NUTANIX_NETWORK=vlan.0
export NUTANIX_INSECURE=1
LAB_SUBNET=10.1.1.0/24 ./scripts/nutanix-lab-up.sh --skip-build --cp-vmid 210 --workers 1

Node OS

  • Same cloud image for controlplane and worker (role comes from machine config)
  • pertiskd PID 1: GPT / STATE / EPHEMERAL disks, DHCP or static net, containerd, kubelet, signed A/B updates
  • Serial console dashboard (Talos-style status TUI on Proxmox / ESXi / AHV Serial)
  • Multi-arch amd64 / arm64 (initramfs + cloud qcow2/raw)
  • A/B OS updates with Ed25519-signed bundles (pertisk-update / pertisk-sign)
  • UKI / Secure Boot lab path (make uki, make enroll-ovmf)
  • Guest extensions: nfs-client, qemu-guest-agent
  • Observability: gRPC mTLS :50000, Prometheus :50001, plus pertiskctl logs / attest / quote / etcd / containers / interfaces / disks

Cluster lifecycle

  • pertiskctl gen config — controlplane + worker YAML (HA multi-CP, dual-stack CIDRs)
  • Bootstrap first CP (PKI + static pods: etcd, apiserver, controller-manager, scheduler)
  • Join workers and additional control planes (stacked etcd)
  • HA: controlplanes > 1 → stacked etcd + kube-vip VIP
  • Rolling Kubernetes upgrade (drain → bump version → Ready → uncordon)
  • Mgmt UI: create, add nodes, bulk reboot/delete, hardware resize, kubeconfig download

Networking

Mode / CNI Notes
IPv4 / IPv6 / dual-stack Pod + service CIDRs; optional VIP6
Built-in cluster.cni: bridge Unique podCidr — single-node / lab
Cilium (lab default) kubeProxyReplacement; guest needs shared bpffs
Calico / Flannel Via lab-up or examples/cni/ with cni: none
kube-vip Static pod on CPs

Management UI

Single-port API + UI (pertisk-mgmt):

  • Auth: local, Auth0, or both — roles admin | operator | viewer
  • Dashboard: cluster counts, reachability, CPU / memory / disk gauges
  • Providers: Proxmox (API token), vSphere ESXi (SOAP /sdk), Nutanix AHV (Prism Element REST)
  • Cluster detail: Overview, Nodes, K8s workloads, Shell, Config, Upgrade, Jobs
  • Node detail: inventory, live health, metrics charts, log tail
  • Machines, Templates, Audit, adopt/join tokens
  • Terraform provider for the same API (kind = proxmox | vsphere | nutanix)

Providers

Provider Status
Proxmox VE Supported (API token; optional SSH for arm64 create)
VMware ESXi (standalone) Supported (qcow2→VMDK) — not vCenter
Nutanix AHV (Prism Element) Supported (qcow2 URL import + UEFI VM)
QEMU / bare metal EFI Supported
AWS / GCP / Azure Outlined only (paused)

Nutanix AHV notes

  • Prism Element on port 9440; storage container + AHV managed network / VLAN
  • UI: Providers → Add → Kind: Nutanix (AHV) → Test (login, hosts, storage, network) → Save
  • Upload: mgmt HTTP pull (:18765) → Prism image_import_spec → UEFI AHV VM
  • Use Serial Console (VGA freezes at EFI stub — expected); virtio disk bus by default
  • Mgmt should share L2 with guests for MAC→IP (LAB_SUBNET)

Build images

make help
make build VERSION=0.2.0 ARCH=amd64 EMBED_BOOT=1 EMBED_RUNTIME=1
make cloud VERSION=0.2.0 ARCH=amd64     # → out/pertisk-cloud-*.qcow2
make uki ARCH=amd64
make pertiskctl
make mgmt / make mgmt-rpm

Observability

./out/bin/pertiskctl -e 127.0.0.1:50000 logs dmesg -n 50
./out/bin/pertiskctl -e 127.0.0.1:50000 logs pertiskd
./out/bin/pertiskctl -e 127.0.0.1:50000 containers
./out/bin/pertiskctl -e 127.0.0.1:50000 interfaces
./out/bin/pertiskctl -e 127.0.0.1:50000 disks
./out/bin/pertiskctl -e 127.0.0.1:50000 attest
./out/bin/pertiskctl -e 127.0.0.1:50000 quote --verify
./out/bin/pertiskctl -e 127.0.0.1:50000 etcd snapshot