A practical guide to clustering heterogeneous homelab nodes the right way

---

If you’ve been running multiple Proxmox nodes as independent standalone hosts and decided to bring them together into a single cluster, you probably hit the same wall I did: most of the clustering documentation assumes you have enterprise-grade shared storage, fencing devices, and a dedicated cluster network. In a homelab, you have none of that — and if you proceed naively, you end up with a cluster that’s more fragile than your original standalone setup.

This post documents the challenges I faced when clustering three heterogeneous nodes and how I solved each one.

---

The Setup

Three nodes, all running Proxmox VE 9, each with its own local storage:

• A lightweight mini PC running 24/7, designated as cluster master
• A heavier compute node with more RAM and storage, used for demanding workloads
• A third mini PC running a specific NVR workload

None of the nodes share a storage pool. Each has its own NVMe SSDs, HDDs, and LVM-thin pools. No SAN, no Ceph, no NFS for VM disk storage — just local disks.

The goal was simple: a single Proxmox UI to manage everything, with each node remaining fully independent and capable of booting its VMs regardless of whether the other nodes were reachable.

---

Challenge 1: Hostname and DNS Consistency

Before even thinking about pvecm create, Proxmox requires that all nodes can resolve each other by FQDN. Corosync and the cluster filesystem (pmxcfs) depend on it.

In my case, two nodes had their search domain set to local.homelab.net while the third was still on the old homelab.net. The result: hostname --fqdn returned different domains across nodes, which would cause cluster communication issues down the line.

The fix: Standardize all nodes to the same search domain before touching anything cluster-related. On Proxmox, don’t edit /etc/resolv.conf directly — it can be overwritten. Use the PVE API instead:

pvesh set /nodes/<nodename>/dns --search local.homelab.net

Verify with:

hostname --fqdn

All three nodes should return <nodename>.local.homelab.net before proceeding.

---

Challenge 2: The Quorum Problem for Standalone Nodes

This is the big one, and it catches most homelab admins off guard.

Proxmox clusters use Corosync for node heartbeating and quorum to determine cluster health. The default behavior: if a node loses quorum (can’t reach enough peers), Proxmox freezes VM operations on that node. It won’t start VMs, it won’t stop them gracefully — everything just hangs.

Quorum fencing exists for a good reason in enterprise environments: if two nodes can both write to the same shared disk simultaneously, you get catastrophic data corruption. Quorum prevents this by shutting down the “minority” side.

But here’s the thing — if you have no shared storage, there is no shared disk to corrupt. Each node only touches its own local disks. Quorum fencing in this scenario provides zero protection and causes real pain: if your cluster master goes offline for maintenance, your other two nodes refuse to start VMs until it comes back.

The fix: Set no_quorum_policy: ignore in Corosync configuration. This tells the cluster to keep running VM operations even when quorum is lost.

After creating the cluster, edit /etc/pve/corosync.conf and add it to the quorum section:

quorum {
  provider: corosync_votequorum
  expected_votes: 3
  no_quorum_policy: ignore
}

With this in place, each node operates independently even if its peers are unreachable. You get the unified management UI when everything is up, and you get resilience when things are down.

---

Challenge 3: Ghost Disks — The Silent Disaster

When nodes join a cluster, Proxmox replicates storage.cfg cluster-wide. This means every node suddenly “sees” the storage pools defined on all other nodes — including local LVM-thin pools that physically only exist on one machine.

The result: a node will happily display another node’s local storage as “available,” and if you accidentally provision a VM disk there, it will fail silently or corrupt. Even worse, Proxmox’s UI won’t clearly warn you that you’re trying to use storage that doesn’t exist locally.

This is the ghost disk problem.

The fix: Use the nodes= directive in storage.cfg to restrict each storage pool to only the node it physically lives on.

Example storage.cfg:

lvmthin: fast-nvme
    thinpool fast-nvme
    vgname fast-nvme
    content rootdir,images
    nodes node1

lvmthin: secondary-ssd
    thinpool secondary-ssd
    vgname secondary-ssd
    content rootdir,images
    nodes node2

lvmthin: nvr-storage
    thinpool nvr-storage
    vgname nvr-storage
    content rootdir,images
    nodes node3

The nodes= line means that storage only appears in the UI when you’re looking at the correct node. No cross-contamination, no ghost disks.

Important: storage.cfg is cluster-wide and lives in /etc/pve/. Any node can modify it, but changes replicate everywhere. Edit it once after all nodes have joined — not before, because a node join overwrites the local storage.cfg with the master’s copy.

---

Challenge 4: VMs Vanish From the UI After Joining

Storage was fixed, the cluster was up, all three nodes were showing green. Then I noticed something alarming: the VMs and containers on two of the three nodes had completely disappeared from the Proxmox web UI — not stopped, not errored, just gone. No entries at all under those nodes.

To be clear about what “disappeared” means here: the VMs were still running. Every service was reachable, every SSH session connected, every workload humming along normally. The problem was purely at the management layer — Proxmox itself had lost track that those VMs existed. You couldn’t start, stop, snapshot, or manage them through the UI or API. They were invisible to the cluster, but alive on the metal.

What happened: When a standalone node joins a cluster, Proxmox transitions its local filesystem to pmxcfs — the distributed cluster filesystem. As part of this transition, VM and container configuration files need to be migrated from the old standalone path into the new cluster-aware path at /etc/pve/nodes/<nodename>/qemu-server/. In my case, that migration silently failed on two of the three nodes. The config files weren’t in the old path, weren’t in the new path — they were nowhere on the live filesystem.

Checking both locations confirmed the worst:

ls /etc/pve/nodes/<nodename>/qemu-server/   # empty
ls /etc/pve/qemu-server/                    # empty

Where the configs actually were: Before completing the join, Proxmox automatically creates a compressed SQLite backup of the node’s cluster database at /var/lib/pve-cluster/backup/. The configs were in there — they just never made it out into the live filesystem.

The recovery process: extract the backup into a temporary SQLite database, query it for the config files, and write them directly into the correct cluster path.

# Load the backup into a temporary database
zcat /var/lib/pve-cluster/backup/config-<timestamp>.sql.gz | sqlite3 /tmp/node-restore.db

# Inspect the schema — it uses 'name', not 'path'
sqlite3 /tmp/node-restore.db "SELECT name FROM tree WHERE name LIKE '%.conf';"

# Restore each VM config to its correct cluster path
for vmid in 100 102 103 105; do
  sqlite3 /tmp/node-restore.db \
    "SELECT data FROM tree WHERE name='${vmid}.conf';" \
    > /etc/pve/nodes/<nodename>/qemu-server/${vmid}.conf
done

# For LXC containers, the path differs
for ctid in 300 301; do
  sqlite3 /tmp/node-restore.db \
    "SELECT data FROM tree WHERE name='${ctid}.conf';" \
    > /etc/pve/nodes/<nodename>/lxc/${ctid}.conf
done

After writing the configs, the VMs reappeared in the UI immediately — no restart required. pmxcfs picks up new files in real time.

The lesson: If VMs disappear from the UI after a node joins the cluster, don’t panic and don’t touch the running workloads. The data and the disks are fine. The configs are almost certainly in the SQLite backup. Check /var/lib/pve-cluster/backup/ first.

---

Challenge 5: Why I Deliberately Skipped HA

Proxmox’s High Availability feature is prominently visible in the UI, and it’s tempting to think “I have a cluster now, I should enable HA on my important VMs.” Resist this.

HA in Proxmox works by detecting that a node has gone offline and automatically restarting its VMs on a surviving node. This requires two things that a local-storage homelab doesn’t have: shared storage (so the surviving node can actually access the VM’s disk) and a fencing mechanism (a way to guarantee the original node is truly dead before another node starts the same VM, preventing two nodes from writing to the same disk simultaneously).

Without both of these, enabling HA causes more problems than it solves. If a node goes offline, the cluster will repeatedly attempt to migrate and restart the VM on another node — and repeatedly fail, because the disk isn’t there. The cluster enters a retry loop, the VM ends up in an undefined state, and you’re left untangling it manually.

The deliberate choice here is to simply not use HA at all. The cluster serves a different purpose in this setup: unified management, a single web UI, and consolidated monitoring. Each node is responsible for its own VMs. If a node goes down, its VMs go down with it — intentionally and cleanly, with no cluster intervention. That’s fine for a homelab. You know where your VMs live, you know how to bring them back, and you don’t need the cluster to second-guess you.

---

The Order of Operations

Getting the sequence right matters. Here’s what worked:

1. Fix hostnames and DNS on all nodes first
2. Back up each node’s storage.cfg before touching anything
3. Rename/fix any storage pools that have naming conflicts between nodes
4. Create the cluster on the master node (pvecm create)
5. Immediately set no_quorum_policy: ignore in corosync.conf
6. Join remaining nodes one at a time (pvecm add --force)
7. After all nodes have joined, edit storage.cfg once to add nodes= restrictions to every local storage pool
8. Verify in the UI that each node only shows its own storage
9. If VMs are missing from the UI, recover configs from /var/lib/pve-cluster/backup/ — don’t touch the running workloads

---

End Result

After working through all of this, the outcome is exactly what a homelab cluster should be: a single pane of glass for managing all your nodes, with each one remaining fully autonomous. Any node can go down for maintenance, upgrades, or power savings without affecting the others. The UI shows everything in one place when nodes are reachable, and gracefully marks them offline when they’re not.

No shared storage required. No HA complexity. No quorum anxiety.

If your homelab nodes are heterogeneous machines with local-only storage, this approach is the right one — just make sure you address the gotchas before you start, not after.

---

Running Proxmox VE 9.x across all nodes. Commands and behavior may vary slightly on older versions.