A practical guide to clustering heterogeneous homelab nodes the right way<\/em><\/p>\n\n\n\n If you’ve been running multiple Proxmox nodes as independent standalone hosts and decided to bring them together into a single cluster, you probably hit the same wall I did: most of the clustering documentation assumes you have enterprise-grade shared storage, fencing devices, and a dedicated cluster network. In a homelab, you have none of that \u2014 and if you proceed naively, you end up with a cluster that’s more fragile than your original standalone setup.<\/p>\n\n\n\n This post documents the challenges I faced when clustering three heterogeneous nodes and how I solved each one.<\/p>\n\n\n\n Three nodes, all running Proxmox VE 9, each with its own local storage:<\/p>\n\n\n\n None of the nodes share a storage pool. Each has its own NVMe SSDs, HDDs, and LVM-thin pools. No SAN, no Ceph, no NFS for VM disk storage \u2014 just local disks.<\/p>\n\n\n\n The goal was simple: a single Proxmox UI to manage everything, with each node remaining fully independent and capable of booting its VMs regardless of whether the other nodes were reachable.<\/p>\n\n\n\n Before even thinking about In my case, two nodes had their search domain set to The fix:<\/strong> Standardize all nodes to the same search domain before touching anything cluster-related. On Proxmox, don’t edit Verify with:<\/p>\n\n\n\n All three nodes should return This is the big one, and it catches most homelab admins off guard.<\/p>\n\n\n\n Proxmox clusters use Corosync<\/strong> for node heartbeating and quorum<\/strong> to determine cluster health. The default behavior: if a node loses quorum (can’t reach enough peers), Proxmox freezes VM operations on that node<\/strong>. It won’t start VMs, it won’t stop them gracefully \u2014 everything just hangs.<\/p>\n\n\n\n Quorum fencing exists for a good reason in enterprise environments: if two nodes can both write to the same shared disk simultaneously, you get catastrophic data corruption. Quorum prevents this by shutting down the “minority” side.<\/p>\n\n\n\n But here’s the thing \u2014 if you have no shared storage, there is no shared disk to corrupt. Each node only touches its own local disks. Quorum fencing in this scenario provides zero protection and causes real pain: if your cluster master goes offline for maintenance, your other two nodes refuse to start VMs until it comes back.<\/p>\n\n\n\n The fix:<\/strong> Set After creating the cluster, edit With this in place, each node operates independently even if its peers are unreachable. You get the unified management UI when everything is up, and you get resilience when things are down.<\/p>\n\n\n\n When nodes join a cluster, Proxmox replicates The result: a node will happily display another node’s local storage as “available,” and if you accidentally provision a VM disk there, it will fail silently or corrupt. Even worse, Proxmox’s UI won’t clearly warn you that you’re trying to use storage that doesn’t exist locally.<\/p>\n\n\n\n This is the ghost disk problem.<\/p>\n\n\n\n The fix:<\/strong> Use the Example The Important:<\/strong> Storage was fixed, the cluster was up, all three nodes were showing green. Then I noticed something alarming: the VMs and containers on two of the three nodes had completely disappeared from the Proxmox web UI \u2014 not stopped, not errored, just gone<\/em>. No entries at all under those nodes.<\/p>\n\n\n\n To be clear about what “disappeared” means here: the VMs were still running. Every service was reachable, every SSH session connected, every workload humming along normally. The problem was purely at the management layer \u2014 Proxmox itself had lost track that those VMs existed. You couldn’t start, stop, snapshot, or manage them through the UI or API. They were invisible to the cluster, but alive on the metal.<\/p>\n\n\n\n What happened:<\/strong> When a standalone node joins a cluster, Proxmox transitions its local filesystem to Checking both locations confirmed the worst:<\/p>\n\n\n\n Where the configs actually were:<\/strong> Before completing the join, Proxmox automatically creates a compressed SQLite backup of the node’s cluster database at The recovery process: extract the backup into a temporary SQLite database, query it for the config files, and write them directly into the correct cluster path.<\/p>\n\n\n\n After writing the configs, the VMs reappeared in the UI immediately \u2014 no restart required. The lesson:<\/strong> If VMs disappear from the UI after a node joins the cluster, don’t panic and don’t touch the running workloads. The data and the disks are fine. The configs are almost certainly in the SQLite backup. Check Proxmox’s High Availability feature is prominently visible in the UI, and it’s tempting to think “I have a cluster now, I should enable HA on my important VMs.” Resist this.<\/p>\n\n\n\n HA in Proxmox works by detecting that a node has gone offline and automatically restarting its VMs on a surviving node. This requires two things that a local-storage homelab doesn’t have: shared storage (so the surviving node can actually access the VM’s disk) and a fencing mechanism (a way to guarantee the original node is truly dead before another node starts the same VM, preventing two nodes from writing to the same disk simultaneously).<\/p>\n\n\n\n Without both of these, enabling HA causes more problems than it solves. If a node goes offline, the cluster will repeatedly attempt to migrate and restart the VM on another node \u2014 and repeatedly fail, because the disk isn’t there. The cluster enters a retry loop, the VM ends up in an undefined state, and you’re left untangling it manually.<\/p>\n\n\n\n The deliberate choice here is to simply not use HA at all. The cluster serves a different purpose in this setup: unified management, a single web UI, and consolidated monitoring. Each node is responsible for its own VMs. If a node goes down, its VMs go down with it \u2014 intentionally and cleanly, with no cluster intervention. That’s fine for a homelab. You know where your VMs live, you know how to bring them back, and you don’t need the cluster to second-guess you.<\/p>\n\n\n\n Getting the sequence right matters. Here’s what worked:<\/p>\n\n\n\n After working through all of this, the outcome is exactly what a homelab cluster should be: a single pane of glass for managing all your nodes, with each one remaining fully autonomous. Any node can go down for maintenance, upgrades, or power savings without affecting the others. The UI shows everything in one place when nodes are reachable, and gracefully marks them offline when they’re not.<\/p>\n\n\n\n No shared storage required. No HA complexity. No quorum anxiety.<\/p>\n\n\n\n If your homelab nodes are heterogeneous machines with local-only storage, this approach is the right one \u2014 just make sure you address the gotchas before you start, not after.<\/p>\n\n\n\n Running Proxmox VE 9.x across all nodes. Commands and behavior may vary slightly on older versions.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":" A practical guide to clustering heterogeneous homelab nodes the right way If you’ve been running multiple Proxmox nodes as independent standalone hosts and decided to bring them together into a single cluster, you probably hit the same wall I did: most of the clustering documentation assumes you have enterprise-grade shared storage, fencing devices, and a … Continue reading “Building a Proxmox Home Cluster Without Shared Storage, HA, or Quorum Worries”<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[25],"tags":[16,22],"_links":{"self":[{"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=\/wp\/v2\/posts\/122"}],"collection":[{"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=122"}],"version-history":[{"count":1,"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=\/wp\/v2\/posts\/122\/revisions"}],"predecessor-version":[{"id":123,"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=\/wp\/v2\/posts\/122\/revisions\/123"}],"wp:attachment":[{"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=122"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=122"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blog.lfps64.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=122"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
\n\n\n\n
\n\n\n\nThe Setup<\/h2>\n\n\n\n
\n
\n\n\n\nChallenge 1: Hostname and DNS Consistency<\/h2>\n\n\n\n
pvecm create<\/code>, Proxmox requires that all nodes can resolve each other by FQDN. Corosync and the cluster filesystem (pmxcfs<\/code>) depend on it.<\/p>\n\n\n\nlocal.homelab.net<\/code> while the third was still on the old homelab.net<\/code>. The result: hostname --fqdn<\/code> returned different domains across nodes, which would cause cluster communication issues down the line.<\/p>\n\n\n\n\/etc\/resolv.conf<\/code> directly \u2014 it can be overwritten. Use the PVE API instead:<\/p>\n\n\n\npvesh set \/nodes\/<nodename>\/dns --search local.homelab.net\n<\/code><\/pre>\n\n\n\nhostname --fqdn\n<\/code><\/pre>\n\n\n\n<nodename>.local.homelab.net<\/code> before proceeding.<\/p>\n\n\n\n
\n\n\n\nChallenge 2: The Quorum Problem for Standalone Nodes<\/h2>\n\n\n\n
no_quorum_policy: ignore<\/code> in Corosync configuration. This tells the cluster to keep running VM operations even when quorum is lost.<\/p>\n\n\n\n\/etc\/pve\/corosync.conf<\/code> and add it to the quorum section:<\/p>\n\n\n\nquorum {\n provider: corosync_votequorum\n expected_votes: 3\n no_quorum_policy: ignore\n}\n<\/code><\/pre>\n\n\n\n
\n\n\n\nChallenge 3: Ghost Disks \u2014 The Silent Disaster<\/h2>\n\n\n\n
storage.cfg<\/code> cluster-wide. This means every node suddenly “sees” the storage pools defined on all other nodes \u2014 including local LVM-thin pools that physically only exist on one machine.<\/p>\n\n\n\nnodes=<\/code> directive in storage.cfg<\/code> to restrict each storage pool to only the node it physically lives on.<\/p>\n\n\n\nstorage.cfg<\/code>:<\/p>\n\n\n\nlvmthin: fast-nvme\n thinpool fast-nvme\n vgname fast-nvme\n content rootdir,images\n nodes node1\n\nlvmthin: secondary-ssd\n thinpool secondary-ssd\n vgname secondary-ssd\n content rootdir,images\n nodes node2\n\nlvmthin: nvr-storage\n thinpool nvr-storage\n vgname nvr-storage\n content rootdir,images\n nodes node3\n<\/code><\/pre>\n\n\n\nnodes=<\/code> line means that storage only appears in the UI when you’re looking at the correct node. No cross-contamination, no ghost disks.<\/p>\n\n\n\nstorage.cfg<\/code> is cluster-wide and lives in \/etc\/pve\/<\/code>. Any node can modify it, but changes replicate everywhere. Edit it once after all nodes have joined \u2014 not before, because a node join overwrites the local storage.cfg<\/code> with the master’s copy.<\/p>\n\n\n\n
\n\n\n\nChallenge 4: VMs Vanish From the UI After Joining<\/h2>\n\n\n\n
pmxcfs<\/code> \u2014 the distributed cluster filesystem. As part of this transition, VM and container configuration files need to be migrated from the old standalone path into the new cluster-aware path at \/etc\/pve\/nodes\/<nodename>\/qemu-server\/<\/code>. In my case, that migration silently failed on two of the three nodes. The config files weren’t in the old path, weren’t in the new path \u2014 they were nowhere on the live filesystem.<\/p>\n\n\n\nls \/etc\/pve\/nodes\/<nodename>\/qemu-server\/ # empty\nls \/etc\/pve\/qemu-server\/ # empty\n<\/code><\/pre>\n\n\n\n\/var\/lib\/pve-cluster\/backup\/<\/code>. The configs were in there \u2014 they just never made it out into the live filesystem.<\/p>\n\n\n\n# Load the backup into a temporary database\nzcat \/var\/lib\/pve-cluster\/backup\/config-<timestamp>.sql.gz | sqlite3 \/tmp\/node-restore.db\n\n# Inspect the schema \u2014 it uses 'name', not 'path'\nsqlite3 \/tmp\/node-restore.db \"SELECT name FROM tree WHERE name LIKE '%.conf';\"\n\n# Restore each VM config to its correct cluster path\nfor vmid in 100 102 103 105; do\n sqlite3 \/tmp\/node-restore.db \\\n \"SELECT data FROM tree WHERE name='${vmid}.conf';\" \\\n > \/etc\/pve\/nodes\/<nodename>\/qemu-server\/${vmid}.conf\ndone\n\n# For LXC containers, the path differs\nfor ctid in 300 301; do\n sqlite3 \/tmp\/node-restore.db \\\n \"SELECT data FROM tree WHERE name='${ctid}.conf';\" \\\n > \/etc\/pve\/nodes\/<nodename>\/lxc\/${ctid}.conf\ndone\n<\/code><\/pre>\n\n\n\npmxcfs<\/code> picks up new files in real time.<\/p>\n\n\n\n\/var\/lib\/pve-cluster\/backup\/<\/code> first.<\/p>\n\n\n\n
\n\n\n\nChallenge 5: Why I Deliberately Skipped HA<\/h2>\n\n\n\n
\n\n\n\nThe Order of Operations<\/h2>\n\n\n\n
\n
storage.cfg<\/code> before touching anything<\/li>\n\n\n\npvecm create<\/code>)<\/li>\n\n\n\nno_quorum_policy: ignore<\/code> in corosync.conf<\/code><\/li>\n\n\n\npvecm add --force<\/code>)<\/li>\n\n\n\nstorage.cfg<\/code> once to add nodes=<\/code> restrictions to every local storage pool<\/li>\n\n\n\n\/var\/lib\/pve-cluster\/backup\/<\/code> \u2014 don’t touch the running workloads<\/li>\n<\/ol>\n\n\n\n
\n\n\n\nEnd Result<\/h2>\n\n\n\n
\n\n\n\n