Eric Li DevOps & Network Architecture

Immich, Kuma and Cloudflared Move Off n5ubuntu

Homelab · 2026-10-13 · Tuesday · 10:30 PM · 6 min read · Eric Li

Work started at 03:40 and ran to about 07:30, then picked up at 15:56 and went on until about 22:30.

A plan for the Home Assistant dashboard

At 03:40 I looked at five screenshots of someone else's wall dashboard and asked what I could build that is like it or better. I did a read-only inventory of Home Assistant and the lab, and wrote a plan. Nothing in Home Assistant has changed. It has 1,389 entities, 214 of them unavailable, no custom cards, theme or calendars, and a UniFi Protect repair waiting. The plan is to clean up the dead entities first, use the native Sections view, add calendars, and add tiles for a "lab healthy" status and what needs attention. I haven't decided which screens it's for. HAOS is also running on pve03, not where the docs said.

Updating and rebooting n5ubuntu

I updated everything on n5ubuntu and rebooted it, since the reboot was overdue from the 7th. Before touching Immich I took a database dump, because its schema upgrades are one-way. Immich went from 3.1.0 to 3.3.1, along with updates to Radarr, Sonarr, Prowlarr, Bazarr, Tautulli, Seerr, FlareSolverr and a few others. Docker, containerd and openssh moved with the apt upgrade and pruning old images freed 11.5 GB. The reboot took about 20 seconds, 21 of 21 containers came back and the NAS and remote-box mounts were fine.

Rolling update on pve02 and pve03

I updated both nodes in parallel but rebooted them one at a time. Only a few guests are allowed on those two nodes under the HA rules, so rebooting both at once would have left them nowhere to go and the shutdowns would have hung. With no console on these machines I wasn't keen on that. pve02 went first with its guests migrated off, and was back in 43 seconds. pve03 live-migrated n5ubuntu to pve02 in about 2.5 minutes without a restart, and was back in 35. All five quorum votes held and the whole thing took 13 minutes. I got one thing wrong on the way. I thought n5ubuntu had no replication job and would need a full disk copy, but pvesr status only shows the local node's jobs.

Immich gets its own VM

I moved Immich off n5ubuntu onto n5immich, VM 139 at x.x.0.39, on pve02 with pve01 as the HA fallback. It has 4 vCPUs, 6 GB of RAM and a 32 GB Debian 13 cloud image, with Immich pinned at 3.3.1. The 158 GB library stays on the NAS. I chose CPU only for machine learning and transcoding so the VM can live-migrate. I ruled out pve04 because it's about to come out and be rebuilt as a node.

Two things tripped me up. UniFi blocks DNS for new wired devices until their MAC is on an admin allowlist, so I added the VM's before building it, and re-read the list twice because the controller sometimes acknowledges writes it doesn't apply. And my Cloudflare API token couldn't edit tunnels, so I added the tunnel edit permission myself.

The cutover took about three minutes, from 16:43 to 16:46. I dumped the database, restored it on the VM, and the counts matched exactly: 31,457 assets, 2 users, 630 people, 48 albums. A sample thumbnail, preview and original photo and video all read back. I repointed the public hostname, the Kuma monitor, the Prometheus blackbox target and the backup script. Then I added the HA rule and replication to pve01 every 15 minutes, and tested a live migration and a reboot. The old copy on n5ubuntu is my rollback until about the 20th. Phones need the new Tailscale address, x.x.226.97, and I'll turn off the key expiry on the VM.

Kuma and cloudflared into containers

Next, the first Uptime Kuma and the first cloudflared connector left n5ubuntu for containers of their own, matching the second pair. cfd1 is CT 148 on pve03, and I did it first with no downtime by adding it as a third connector before removing the old one. kuma1 is CT 145 on pve01 with fallback on pve03. Its cutover stopped monitoring for about 47 seconds. All 59 monitors, 12 push monitors and the heartbeat history copied across, and every monitor came up, including the cameras. A forwarder on n5ubuntu relays stragglers to the new Kuma for about a week. I repointed the vzdump hooks, backups, Prometheus and Home Assistant sensors, and edited seven push URLs in Proton Pass by hand. One backup field went blank in the edit and I fixed it at 21:41. Each container has its own runsheet.

The printer and the bridges

I spent about 50 minutes on a Canon printer on the IoT VLAN that the PC and phones couldn't reach. I made the printer's address fixed, allowed management and home to it on the printing and scan ports, let it reach the house's SMB only, and blocked it from the internet. The rules didn't help at first, and I worked through several dead ends. The root cause was the "Block IoT to Internal" rule, which matches every connection state. It was rejecting the printer's replies before UniFi's automatic return rules could allow them. The fix was an allow rule for replies on established connections, above the block. After that it loaded in a browser and was added to the PC at 19:16. I also turned off the printer's Wi-Fi Direct.

The same flaw means nothing on Home can reach IoT devices either. The proper fix is to make the block rule match new and invalid connections only, but 27 IoT devices and five cameras would become reachable from Home, so I previewed it and held off.

Matter to a Home Assistant add-on

Only two devices depend on the Matter server, a doorbell and a camera hub, both from the same vendor. They'd been offline since 27 August, and the cause was another firewall rule. The bridges are on the Home VLAN, and a "Block Non-admin to Management" rule dropped their replies into the management network. I added a narrow allow rule for those two devices to Home Assistant and n5ubuntu, and both reconnected within 13 seconds. Then I rehearsed the migration on a throwaway server. The add-on assumes a different fabric ID and vendor ID from my existing setup, and the defaults would have started silently with an empty fabric. With override settings the cutover took about 1 minute 40 seconds, with the same pairings and no re-pairing. A secret from the Home Assistant config ended up printed in a log. I shredded the log.

n5ubuntu RAM and timezones

Proxmox showed n5ubuntu using 11.5 of 12 GB because ballooning was off, so it counted file cache. Inside the VM only 2.7 GB was in use. Now that Immich has left, I cut it to 8 GB with a 4 GB balloon floor and rebooted in about 25 seconds. That frees 4 GB on pve03.

At 22:18 I made everything run on Perth time. Two VMs and Immich's own container were still on UTC. Six containers still are. The Immich database dump now runs at 02:00.

Still open