Sessions ran from about 01:20 through to around 18:00 today, in bursts, and the throughline across all of them is the same shape twice over. The lab finished becoming a real three-node cluster — pve03 joined, quorum went to 2-of-3, and by the end of the day the whole thing was standing on its own without n5ubuntu holding it up. But two separate things that had looked fine for a while turned out to have been lying the whole time: a NIC that made a perfectly healthy host look dead to the rest of the cluster, and a backup hook that had been reporting success for failed backups since 2026-07-28. Neither lie was subtle once looked at directly. Both had gone unnoticed because nothing forced anyone to look.
Three nodes, one cluster
pve03 joined n5hq as node ID 3 today — pvecm add x.x.0.20 --use_ssh 1 --link0 x.x.0.25 — and quorum became 2-of-3, config version 3, expected votes 3. Every guest already running on the cluster stayed up through the join; nothing had to be stopped or migrated to make room for the third vote. That's the milestone the day gets its shape from: the cluster stopped being two nodes with a plan for a third and became three nodes for real.
The network side got built the same day rather than deferred. pve03 got its own active-backup bond — a 2.5G USB NIC as primary with the onboard 1GbE as failover, mirroring the design already running on pve01 — with a config backup taken first at /root/interfaces.bak-2026-08-10 so the previous state wasn't lost if the bond needed to be unwound. A separate apt lockout on pve03, caused by an enterprise repository it had no subscription for, got fixed in the same pass rather than left to bite the next update run.
The node that wasn't dead
At 11:29 pve02 disappeared from the network and took the n5ubuntu VM 200 with it. corosync on pve01 logged the sequence cleanly: link: host: 2 link: 0 is down at 11:29:18, token loss a second later at 11:29:19, and Members left: 2 at 11:29:24. Read from the outside, that's every signature of a dead host — no link, no token, node gone from the membership list within six seconds.
It wasn't dead. The root cause was an e1000e NIC transmit-unit hang — the host itself stayed healthy the whole time, and only its ability to transmit had wedged. Diagnosing that meant not trusting the "unreachable" signal at face value, because on this exact class of symptom a confident wrong call has been expensive before: an earlier incident where a hung NIC and a broken CIFS mount both looked exactly like failing hardware cost an unnecessary drive pull. The fix, once the actual cause was clear, was straightforward — a second 2.5G USB NIC fitted and bonded active-backup on pve02, the same pattern pve03 got, with failover tested afterward and the lab staying reachable at 0% loss through the test. The general lesson got written down rather than left as something only remembered until the next incident: "unreachable" is not the same as "crashed," and high iowait is not the same as a failing disk — read the kernel log before blaming hardware.
The new bonded NIC's MAC had to go onto the UniFi admin-gate whitelist before pve02 could talk to the rest of the network properly, and that got caught pre-emptively rather than found the hard way. A CLIENT_IP_CONFLICT did still turn up from the onboard MAC after the bonding change, recorded and resolved the same day.
A hook that had been lying since July
The bigger find, and the one worth being uncomfortable about, was that the vzdump-kuma-hook — the script whose entire job is to tell Uptime Kuma whether a backup succeeded — had been reporting success for failed backups since 2026-07-28. Nearly two weeks of green. The cause was specific and simple once found: the hook pushed up on the job-end event unconditionally, and job-end fires whether the backup succeeded or failed. job-abort, the event that would have told the truth, never fires at all in this Proxmox version's hook chain. So every failed vzdump run for almost two weeks had reported exactly the same status as a successful one.
The fix was a rewritten hook installed fresh on all three nodes — a genuine replacement on pve01, and new installs on pve02 and pve03, both of which had no backup-status hook at all before today. Each node got its own Kuma push token, and maxretries was set to 0 on every backup monitor so a retry couldn't mask a failure by quietly succeeding on attempt two. None of that was taken on faith: the new hook was verified against both a real vzdump run and a deliberately forced-failure run, on all three nodes, so the failure path was actually exercised rather than assumed to work because the success path did. The monitors now watching backup status are kuma1 44 for pve01, 51 for pve02, 50 for pve03, and kuma2 14 for the pve02 host plus 15 for the pve03 host. Separately, the ugnas01-backup storage target got un-pinned from nodes pve01 to cluster-wide so pve02 and pve03 could actually write backups to it at all, which they could not do before.
Where Claude actually lives now
The whole Claude workspace moved today, off n5ubuntu and onto its own container — n5claude, CT 109, x.x.0.9, an unprivileged Debian 13 LXC on pve01. The reason is on record in the vault's own words: it "was moved here 2026-08-10 so that Claude survives the n5ubuntu → pve03 migration instead of dying with it." n5ubuntu is being powered off permanently as part of that migration, and a workspace that only existed on that machine's disk would have gone with it.
A new backlog file, runsheets/BACKLOG-n5claude-followups.md, went in the same day, and it records the move's actual loose ends rather than presenting the migration as clean: credentials are now copied across five separate locations instead of one; a UniFi controller credential was found world-readable at mode 664 until today; a seedbox password was sitting embedded in a mode-644 .bashrc; the gh CLI is missing entirely on the new host; and there's a stray SSH keypair, ~/n and ~/n.pub, whose purpose nobody has identified yet. None of those are fixed — they're written down as open, which is the point of the file.
Five automation consumers that used to talk to Claude on n5ubuntu got redirected to n5claude the same day. A new SSH proxy script, ~/bin/claude-remote.sh, was created to carry that traffic, and n5-update-judge.sh and n5-doc-stack.sh were both repointed at the new host. That got checked rather than assumed working: all three judge modes were run end to end — apt in 22 seconds, container in 16, pve in 12 — and all three came back schema-valid JSON. Explicitly not moved: the two Discord bot services, claude-ops.service and claude-supervisor.service. They're still on n5ubuntu, and they're now the last thing pinning Claude to a machine that's scheduled to go dark.
Tailscale changes hands
The primary Tailscale subnet router moved from n5ubuntu to pve01 today as well, for the same reason as the workspace move — n5ubuntu is being powered off permanently once the pve03 migration finishes, and anything still depending on it needed a new home first. Tailscale 1.102.2 went onto pve01, reachable on the tailnet at x.x.27.67.
A gate re-argued, and settled
One open decision got closed rather than left hanging: the G0.6 RAM gate, which had been sitting unratified, got a decision from me today. The threshold moved from a flat "≥28 GiB" to something more specific — host RAM ≥23 GiB and a 16 GiB VM allocation actually verified under load, rather than a number picked in the abstract. No RAM got bought as a result; VM 200 stays at 16384 MB. It's a small item next to the rest of the day, but it's the kind of decision that would otherwise sit open indefinitely waiting for a "someday" that never arrives, and closing it took less effort than the ongoing cost of leaving it open.
---
So: three nodes for real, one node that only looked dead, and one backup hook that had been lying for the better part of two weeks before anyone checked it against a forced failure instead of trusting the green. Both lies had the same shape — a check that reported success because nothing had ever tested the failure path, on a host that had every other reason to look fine from the outside. And the honest loose end for the day is a plain one: every runsheet edit that came out of all of this — BACKLOG.md, RUNSHEETS.md, the app and platform docs for monitoring, n8n, uptime-kuma, n5ubuntu, pve01, pve02, ugnas01, plus the new files for the n5claude followups, pve02's PBS notes, pve03, and the bond self-heal system doc — is still sitting uncommitted in the runsheets repo as I write this. The work happened. The commit didn't, not yet.