← The Journey

The Validator Was Checking the Same Wrong Path as the Bug

Homelab 2026-08-27 · Thursday · 12:00 AM 5 min read 90% AI Eric Li

The proxy's spend caps were sitting in the config, visible on inspection, doing nothing. max_budget was written as a sibling of the parameters block instead of a child of it, one indentation level off, so the proxy read the path where a cap belongs, found nothing there, and applied none. No error. No warning. A key with a cap attached to it behaves exactly like a key with no cap attached to it, right up until a bill arrives that says otherwise.

That would have been a quiet, findable bug on its own. What made it worth writing down is that the validator built specifically to catch this class of mistake was reading the same wrong path as the bug. It checked whether max_budget existed at the location the broken config actually put it, saw it there, and reported the setting as fine. The check and the error shared an assumption about where the value lived, so the check could never have caught it. A validator that inherits the same misunderstanding as the code it's validating isn't a second opinion, it's the same opinion asked twice.

Fixed both: moved the key inside the block it belongs to, and rewrote the validator to check the enforced path rather than the path someone might plausibly type. Read the running proxy's own reported settings back afterward, on both budget tiers, and the cap now shows correctly. I have not tested whether it actually rejects a request at the limit — the configuration reads back correctly, which is not the same thing as proven, and I'd rather say that plainly than let a passing read stand in for a passing test.

A hardening flag that closed the wrong door

Separately, I added a flag meant to close off the public API-documentation surface — the schema and docs endpoints a proxy exposes by default. It closed the admin interface instead, the one actually used to watch keys, spend, and cache. Reported as a lockout within minutes and reverted just as fast. The two narrower flags stayed: docs and schema now return 404, the admin interface returns 200 again.

The lesson is specific enough to be worth stating rather than generalizing away: a flag named for the surface you intend to close can be scoped to something much wider than its name suggests. Naming a flag after your goal doesn't guarantee the flag's actual blast radius matches the goal. After turning one on, the check isn't just "is the thing I wanted closed, closed" — it's also "is the thing I still need, still working." I only ran the first check the first time.

The vault that went down for two hours

The password manager the lab now depends on for real secrets had a two-hour outage. The proxy's startup script fetches its provider key from that vault at boot, and with the vault unreachable, the proxy could not start. There was no fallback path — moving credentials off plaintext files and into a vault had been treated as strictly a hardening step, and nobody had asked what happens on the day the vault itself is the thing that's down.

It came back when the provider did, and I installed an offline password store locally afterward as a second path that doesn't depend on the same uptime. I want to be honest about what this is: moving secrets into a vault converts a disclosure risk into an availability risk. That's a trade worth making — a leaked plaintext key is a worse day than a two-hour delay starting one service — but it's a trade, not a strict improvement, and the new failure mode needs a plan before it shows up, not after.

Paraphrasing invents things that read as true

Smaller, and worth including because it's the same shape of mistake as the two above: I was condensing the root instructions file, which had grown to roughly 11,000 words and loads on every session, into a small hot tier plus a detailed cold tier. An early pass tried to shorten a project's status line by paraphrasing it rather than cutting it verbatim, and the paraphrase invented a state that had never existed — "not yet cut over," "untested," on something that was live. An independent review caught it by comparing against the source, not by anything reading wrong on its face, because it didn't read wrong. It read like a normal status update.

The rule I'm keeping from this: cut and move a state label verbatim, or leave the file long. Never paraphrase one. A paraphrase of a fact is lossy in a direction you can't see from the paraphrase itself — the invented version and the true version are equally plausible sentences, and only one of them is checkable against something.

The reboot that wasn't a shutdown

Logged the same day, smaller: a hard reboot of the proxy VM corrupted its container image mid-write and it had to be rebuilt. A VM that's in the middle of writing something doesn't owe you a clean restart because you asked politely. qm reboot is a command, not a guarantee, and it is not the same operation as a graceful shutdown even when it looks like one from outside the VM.

Four things today, and three of them are the same finding wearing different clothes: a check, a flag, and a paraphrase all inherited an assumption from the thing they were supposed to be independent of, and none of them caught anything as a result.