Bill Gernert
Platform & Orchestration

Proxmox VE

The five-node high-availability cluster that every VM in the lab is provisioned onto - the metal underneath the whole platform.

What it is & why I use it

Proxmox VE is an open-source virtualization platform that combines KVM virtual machines and LXC containers with clustering, high availability, and software-defined storage under one API. I chose it as the base layer for AutomationLab because it gives me a real, API-driven hypervisor cluster - the same primitives a production platform team works with - without a per-socket licensing model getting in the way of experimentation.

The point of the lab is to treat infrastructure as something you declare and automate, not something you click together. Proxmox exposes everything through a REST API, which is exactly what lets Terraform stand VMs up from code instead of from a wizard.

How I use it in the lab

  • Five nodes, clustered. The cluster runs as a single management plane with a shared quorum, so I administer all nodes as one system rather than five separate hosts.
  • High availability. Selected VMs are enrolled in HA groups; if a node fails, the cluster restarts those guests elsewhere automatically. Enrollment is the final, deliberate step of provisioning - not the default - so I control exactly what is allowed to migrate.
  • Terraform-driven. Each VM gets its own Terraform workspace using the bpg/proxmox provider. A pipeline plans and applies against the cluster API; there is no hand-built VM anywhere in the lab.
  • Dynamic load balancing. The cluster rebalances running guests across nodes based on load, so no single node becomes a hotspot as the lab grows.
  • Clean decommission. The provisioning flow reverses cleanly - HA de-registration, then destroy - so tearing a VM down is as automated as building it.

Architecture

Diagram coming Cluster topology: five HA nodes, the shared quorum and storage layer, and where the Terraform provider plugs into the cluster API. Rendered as a scrubbed diagram - no addresses or node hostnames.

In action

Screenshot coming Cluster summary view with all nodes healthy and an HA-managed guest mid-migration. Captured with hostnames and addresses cropped or redacted per the publishing checklist.

Lessons learned & gotchas

Gotcha Quorum is not optional. In a small cluster it is tempting to run an even node count, but you want a reliable majority for the quorum to survive a single-node loss without the whole cluster going read-only. Plan the vote layout before you need it, not during an outage.
  • HA is a promise you have to mean. Enrolling a guest in HA means the cluster will happily restart it on another node - including after a transient blip. Anything with node-local state has to be designed for that before you enroll it, or you will be surprised at the worst time.
  • The API is the contract. Once every VM comes from Terraform against the API, drift shows up as a plan diff instead of a mystery. Clicking a change in the UI just means the next apply wants to undo it - so I stopped doing that.

Impact

5
HA nodes, one plane
100%
VMs from code
0
hand-built guests

Proxmox turned "the lab" from a pile of machines into a single programmable substrate. Everything above it - GitOps, monitoring, identity - assumes it can ask for a VM and get one, reliably, from code. That assumption is what makes the rest of the platform possible.

Code & further reading

Repo (link coming) Proxmox VE

Repository links point to scrubbed, public-safe versions of the lab's Terraform and pipeline code. Internal addresses, hostnames, and secrets are never published.