The Journey
Notes from the journey — infrastructure, learning, and building in public.
-
No Root, No Decoder, No Problem: Reading a HEIC Photo and an 84-Page PDF on a Container With Nothing Installed
A new client engagement arrived as an iPhone photo set and a vendor PDF, and this container could read neither: no HEIC decoder, no PDF renderer, no root to install one. Two dependency-free tools fixed that.
-
A Day With No Journal, Reconstructed From Git and a Backfill That Fixes the Same Problem
Twenty-four commits, four unrelated threads, and no journal entry at all for the day itself: written back from git and transcripts, which is exactly the failure mode the day's own backfill was fixing.
-
The Abort-On-Failure Hook Had One Failure Mode It Couldn't Catch
A commit hook, a design folder and a redesign preview all lived in exactly one place. Today each one got a second copy, and the hook turned out to need it most.
-
An Animation You Cannot Pause Is One You Cannot Review
206 messages, 17 build-and-judge rounds across three pieces, and zero commits, because nothing cleared three independent critics.
-
Inspired By Means Rebuilt In, Not Repainted As
A morning retint kept the old page and swapped its colours. The review said none of the old design, and the rebuild started from an empty stylesheet.
-
The Reboot That Fixed It Once Before and Didn't the Second Time
Five separate-looking faults turned out to be one storage pool gone read-only, and the fix that looked like a fix had already failed once before.
-
The Disk That Read 82% Free and Was 99.9% Full
Every instrument in the lab reported terabytes of free space while the account's quota sat at 99.9% full, one of four systems that reported healthy for days while doing nothing.
-
A Line Passing Through a Switch Read as a Connection
Moved a VM off a failing NAS, drew the lab's real topology, and mirrored 584 pages of docs — the diagram had a bug worth keeping.
-
The Validator Was Checking the Same Wrong Path as the Bug
A budget cap sat one indentation level outside the block that enforces it, and the check written to catch that exact mistake read the same wrong path.
-
A Restart That Reported Success and Changed Nothing
A cost-tracking proxy logged 21 real requests at zero spend, and the cause was a restart that returned clean without touching the container.
-
The Check Meant to Avoid Touching a Secret Printed It Anyway
A shell idiom for testing whether a variable was set leaked a live token twice today, after doing the same thing two weeks earlier.
-
The File That Was Actually Being Read Was Three Layers Down
A 403 traced through three wrong files before the real one, and a daily 90-second outage turned out to have the wrong cause for a month.
-
The Rule That Could Only See One User in Thirty-One
A retention rule read a per-account field as if it were server-wide, missing thirty of thirty-one users' activity until the getter got traced line by line.
-
Every Number I Started With Was Measuring Something Else
The monitoring stack moved to its own VM with a four-minute-fifty-two-second gap, n8n moved off the main server into its own container, two hypervisors were patched and rebooted, and a Raspberry Pi got found and named. Every one of those jobs started from a number that turned out to be measuring the wrong thing — memory that wasn't full, storage that wasn't that size, an address that wasn't free, and a directory listing that wasn't orphaned.
-
The Hardlink That Never Once Linked Anything
A hardlink setting had been silently copying whole files all along, and the review behind it found two more things that only looked safe.
-
A Regex Ate Half a Selector and CSS Didn't Complain
A full day rebuilding the site's identity — animated lockups in the nav on both sites, every rounded corner squared to 3px, the footer rebuilt, and the black keyed out of new artwork so it can sit on anything. Also the day I broke the homepage with a regex that deleted the end of a compound selector, which CSS accepted without a word.
-
Four Things Were Quietly Doing Nothing and All of Them Looked Fine
A Stream Deck plugin fork reached v1.8.0 with dial support and a vendored dependency, and along the way turned up an upstream bug where an optional chain silently swallowed the call behind the plugin's headline feature. Then a release bot published a version nobody asked for, a backup job was found dead since a move it was supposed to have survived, and the drift checker that exists to catch stale claims was found never to have looked at half the tree.
-
Everything Redundant Turned Out to Share Something
VM 200 shrank in place and live-migrated to pve03 with zero downtime, PBS went live on its own VM, NFS shared storage got built and benchmarked, and restores finally got proven against real archives instead of assumed. Then a four-wave read-only analysis of the whole lab found the same flaw in three separate places: the second copy of a thing depends on the first.
-
The Host Wasn't Dead and the Backups Weren't Real — Both Passed Every Check Anyway
pve03 joined the cluster today as node three of three, quorum now 2-of-3, while pve02 vanished from the network at 11:29 and looked exactly like a dead host — it was a hung NIC transmit queue, not a dead box. In between I found the backup monitor had been reporting success for failed backups since 2026-07-28, and moved Claude's own workspace off n5ubuntu onto its own container ahead of that machine being powered off for good.
-
Three Times This Morning, Something Told Me It Worked and Hadn't
This is a morning, not a day. Before eight I closed a critical security hole in the board tool's evidence command, sat at a keyboard walking the main host through a root-privileged setup, and pushed a website fix — and three separate times in between, something told me it was fine when it wasn't: a stale test job that quietly reverted a fix, a push mirror that had been empty since the repo was created, and a deploy that never ran because the repo had gone private.
-
The Checks Kept Passing While the Thing Under Them Was Broken, Four Times Today
I tidied the vault root at four in the morning and found that every one of the 59 references that would have broken were bare paths — zero of them were the Obsidian links I'd assumed were the risk. The rest of the day went into locking down an autonomous worker's network access, five rounds of review, ending with a design whose central claim turned out to have never been true and a decision to stop rather than deploy it. Both threads taught the same lesson: a green check on a script that warns-and-continues, or a reviewer reasoning from a document instead of running the thing, tells you nothing.
-
A Repo Containing the CLI but Not the Deployment Is Half of It
Six quiet days ended with a session-resume that handed back a week-old thread from an unrelated project, which was the clearest sign of the gap I got all night. From there the board project went from a passed Phase 0 check to a production container, a CLI, monitoring, and a repo that two reviewers picked apart before anything ran — one of them for shipping half a plan. A second, public version of the same tool was still mid-build at the point I stopped.
-
The Night I Learned a Model Can't Be Trusted to Report on Itself
A week away with no reliable remote access meant one all-night session had to force every decision that would otherwise sit blocked for seven days. Three separate tests found the same failure in three different clothes: a model cannot be trusted to report on its own state. A sandbox died three times in production and got retired for a plain firewall rule, the night's slowest test still ended with no answer, and a subtitle-fetching job that I'd declared solved turned out to be reading an index that was six percent finished.
-
The Only Backup Was Still Open in the Chat Window
A curated reinstall I handed off with 'you take over from here' finished its uninstall step underneath its own preflight backup, and four hand-condensed rule files were gone with no copy anywhere — recoverable only because they were still sitting in the live session's context from being read at startup. In the same day I closed a real remote-code-execution hole in the SSH automation and nearly reopened it with the fix meant to close it, watched a four-voice council correct its own arithmetic live while killing a runsheet-trim plan, and found two contradicting network audits that turned out to both be right, 42 minutes apart. All of it ran under a Claude platform outage that sat over the morning.
-
I Built a Downloads Sorter in Six Hours, and Every Bug in It Looked Like Working Code
The first backup this server has ever had reported success and contained 1.3 GB of live database files. That set the tone for the day. The rest of it went into a Windows Downloads sorter that produced a run of bugs which all executed cleanly and did the wrong thing, an 18-check test suite that failed once for exactly the reason it existed, and a repository that had to be deleted and recreated rather than force-pushed.
-
A Second Server in the Morning, an Open Front Door by Midnight
The plan was to stop keeping everything on one machine. A small OptiPlex became a second Proxmox node, Home Assistant moved into a real VM, and along the way I took my own network down by plugging in a cable. Then in the evening I turned eight agents loose on the whole lab to criticise it, and they found an approval button reachable from the internet whose only lock was a number you could guess.
-
The Wi-Fi Was Pinned to One Channel by a Network That Was Switched Off
Four access points, all sitting on 2.4 GHz channel 6, and every attempt to change that failed with 'This action could not be completed'. The cause turned out to be a disabled SSID still enforcing a channel lock across the whole site. Fixing it led to a second find: the same power setting produced 17 dBm on one 5 GHz channel and 29 dBm on another. That evening, planning a second Proxmox node turned up one IP address doing two different jobs across eight firewall rules.
-
I Diagnosed the Same Outage Twice and Got Two Different Answers
A 3:35am PDF-tool deployment turned up two silent failures: an OCR mount pointing at a directory the image doesn't contain, and a login seeder that only works against an empty database. Then two separate sessions, three hours apart, looked at the same camera dropout and reached different conclusions, neither aware the other had happened. Three of the day's confident findings were later revised, and the corrections are at the bottom.
-
Twenty-Three Cents to Read a Photo, and Almost None of It Was the Photo
I asked a simple question about a side project: what does it actually cost to have Claude read one photo of a handwritten work order? Three real measurements said about 23 cents each. The autopsy was worse than the number: roughly 117,000 input tokens billed for about 3,200 tokens of actual work, because every call was re-sending the entire agent harness. The rewrite took ten minutes. Whether it saves what I think it saves, I still can't prove.
-
I Asked If My Containers Were Up to Date, and Ended the Day Running My Own Git Forge
The day started with a reboot and six words: are all my docker containers updated? Nine hours later the server answers that question by itself every night, and asks me on Discord when an update looks risky. It also, unplanned, ended up hosting my own git forge. The best part was a one-character bug that quietly turned every yes into a no.
-
I Certified the Network at 2.5 Gigabit, Then Spent Three Hours Losing to a USB Dongle
I wanted real numbers for my own network, not another internet speed test. The morning went well: the wired path certified at 2.38 gigabits, and both devices that looked slow turned out to be behaving exactly as physics says they should. Then a new handheld arrived with a 2.5G USB ethernet dongle that would transmit and never receive, and I produced four confident diagnoses in three hours. All four were wrong.
-
Discord In, Linux ISOs Out
The home server archives Linux ISOs: some ship once like an LTS release, others push a new point release every week for years. Requesting one was already a Discord message, but the last metre was still me, dragging finished downloads into the folders the library software actually watches. This week the pipeline closed: a small sorter container now shelves every ISO by genre and language, season-sized release archives stopped failing once we found the dead mirror behind the timeouts, and the request app got a cleaner name. Then, in the evening: the server proved it could reboot with nobody home, and the Discord ops bridge became a public repo anyone can install.
-
I Gave the AI a Browser, and We Shipped the Redesign
The redesign had been sitting on a preview port waiting for my eyes, because I was the only one in the loop who could see pixels. That changed today: headless Chromium went onto the server, and suddenly every change came back with screenshots attached. What followed was the most productive day this site has ever had — a rebuilt footer, a scroll-driven purple Earth with ethernet cables patching into world capitals, my actual rack photo on the homelab page, live Docker stats on three pages, and the whole thing pushed to production before bed.
-
Two Rooms That Turn Themselves Off, Once I Worked Out What They Were Called
A Sunday that didn't start until four in the afternoon and then ran three jobs with nothing in common. Two rooms in the house learned to turn their own lights off, with the delay picked out of the sensor's own history instead of a forum post. A planning question about changing internet providers answered itself in two minutes, then turned up a piece of leftover configuration I had no memory of creating. And a late-night design pass on the portfolio got told it looked like AI slop.
-
I Sat Down to Add One Button and Rebuilt Half My Smart Home
The plan was small: wire a few Home Assistant controls onto my Stream Deck. Opening Home Assistant to do it exposed the real state of things — five broken integrations, and a chunk of the house that lived in Apple Home but had never made it into HA at all. So the button waited. What followed was a day of standing up a Matter server, untangling why my devices were spread across three VLANs that Matter can't see across, migrating the whole Meross fleet to local control, and clearing every broken integration off the board. Classic me: I went in for a light switch and came out having rebuilt the foundation.
-
I Built a Status Page, and It Caught My NAS Crashing Before I Finished Setting It Up
The plan was a quiet evening standing up Uptime Kuma to watch the homelab. On its very first round of checks it flagged Immich as down — a real outage, not a config mistake. That kicked off a four-hour hunt that ran through stale CIFS mounts and a misbehaving Samba service before SSH into the NAS finally gave a straight answer: the disks were perfect, and smbd was segfaulting on a corrupted database held in RAM. The monitoring's first job turned out to be catching its own storage backend falling over.
-
The Outage Was Already Over, and the Fake Flash Drive Wasn't Fake
I moved the WAN uplink on my gateway to a different port and every tunnelled hostname started returning Cloudflare error 1033. It took five minutes to close, and it had already fixed itself for four of them. What I was actually looking at was my own browser cache. Later the same morning a 256 GB flash drive that refused to format looked briefly like a counterfeit, then wedged Windows' disk service, and I still don't know how that one ended.
-
Twenty-Five Containers Updated, and the Cleanup That Ate My Rollback
I asked for one thing: update all my Docker containers. The tunnel came back refusing the only command it had ever been given, the request bot came back with a nine-byte program whose entire contents were the words Not Found, and the last known-good image I could have rolled back to had been deleted by my own cleanup eight minutes earlier. Every job that night turned up something the job itself had caused or had been hiding.
-
Moving Immich to the NAS — and the Drive That Tried to Take My Photos With It
A simple storage migration turned into a genuine scare when the external SSD holding my entire photo library dropped offline mid-copy and the filesystem went into emergency shutdown. SMART was clean though — it was the USB connection all along. One cable swap later: a clean 107 MB/s copy, verified file-for-file, and Immich now living on the NAS.
-
Taking the Email Off the Page, and Knowing When to Undo a Logo
Pulled my real email out of the contact form and routed it through a masked endpoint, added tag filtering to the blog so posts can be browsed by topic, then spent the evening building a custom logo to replace the placeholder diamond — tried it in the nav and footer, decided it wasn't right, and reverted it. Finished by pushing the day's work live.
-
The Copy Goes Live, and the Lab Page Turns Personal
Shipped the copy and card rework that had been sitting on the local server, matched the portfolio subpages to it, then pulled the Lab out of the public site and rebuilt it as a personal launcher that only links the services already safe to be public. Finished by catching the blog up and writing a small script to generate posts from journals.
-
Sharper Words, Smaller Cards: A Copy Pass on the Portfolio
Went back to the portfolio that shipped yesterday and fixed the part I'd skimmed: the writing. Stripped the filler out of the copy, rewrote it to talk to an employer, shrank the project cards down to a title and a Details button, and made the dead buttons actually work — all kept local for review, nothing pushed yet.
-
Three Designs, a Domain Cutover, and the Linux ISO Stack Comes Home
The portfolio redesign shipped to n5hq.me after three full rebuilds — then the bigger job: pulling the whole Linux ISO automation stack off the rented seedbox and onto the home server, with the 22TB of storage staying put and the file moves still happening at the source.
-
Storage Fixes, Vault Cleanup, and the Reboot That Bit Back
Renamed the external drive mount, set up Samba shares, fixed the T7 Shield's UAS instability — then the reboot that fix required quietly took down Home Assistant and the website dev server. Recovery story included.
-
Network Fixes, Tailscale, and a NIC Upgrade
Diagnosed the Aqara doorbell and Canon printer issues via UniFi MCP, got Tailscale running across all devices, installed Claude CLI on the MacBook, and swapped the server NIC to 2.5GbE.
-
Automating the Blog & Cleaning Up the Portfolio
Wired up GitHub Actions to auto-publish blog posts, stripped placeholder content, and cleaned up the portfolio tiles to show only what's real.
-
Building Momentum: Homelab Services + Website Launch
Deployed the monitoring stack (Prometheus, Grafana, Loki), made Immich publicly accessible, and completed the N5HQ website with portfolio and contact sections.
-
Home Assistant Goes Public
Got Home Assistant accessible remotely via Cloudflare Zero Trust tunnel — after about 1.5 hours of chasing a silent 502.