The Harness (Agents Running the Homelab)

Most of my day-to-day infrastructure work is done by an AI agent now. Not in the press-release sense, in the boring sense: documentation sync, watchdogs and crons, deploys, cost reports, and the site you’re reading this on are all maintained by Claude Code running as a lead engineer with a crew of specialised agents underneath it. This is the write-up of how that’s wired.

First, the hardware it mostly runs on. It’s a single big box in the basement, and I’ll spare you the full bill of materials, but it’s more GPU than a sensible person keeps at home. Here it is, captured as a gaussian splat you can drag around:

Open the splat full screen if the embed’s fighting you. Yes, that’s a plywood bench. It’s structural.

The shape of it

One orchestrator, many spokes. The orchestrator is a Claude Code session that acts as lead engineer: it reads the task, decides who does the work, reviews what comes back, and commits. Underneath it is a roster of specialised subagents, each with its own prompt, tool allowlist, and model pin: a coder, a reviewer, a debugger, a janitor for cleanup jobs, a doctor that diagnoses the session itself when things get stuck, and a git agent that writes commit messages so nobody else has to.

The one hard topology rule is hub-and-spoke: spokes never spawn other spokes. Every delegation flows through the lead engineer, results flow back, and an agent that thinks another agent should be involved can only recommend it. This sounds bureaucratic until you’ve watched two agents delegate to each other in a loop while the token meter spins.

The strangest agent in the roster is @wwmd, which stands for exactly what you think it does: What Would Martin Do. It’s a read-only decision persona that gets called when a pipeline hits a judgment call at 3am, and it grounds its answers in a maintained file of my actual beliefs and decision heuristics rather than vibes. Writing down your own decision-making well enough that a model can impersonate it is a genuinely uncomfortable exercise, and I recommend everyone try it.

The one I lean on for anything with real stakes is @overseer, a multi-model advisory panel. The same question goes to Gemini, OpenAI and Grok in parallel, plus a web search, and a frontier model synthesises the answers into a verdict, including where the panel disagreed. One model confidently wrong is a familiar failure mode. Three models from three vendors confidently wrong in the same direction is rare enough to bet on. Architecture reviews, risky migrations, and “is this plan stupid” checks all go through it, and it’s talked me out of at least one genuinely bad idea a month since it existed.

The whole thing also has a persona directive: the orchestrator is told to be blunt, direct, and Australian. This has no engineering value whatsoever. It just means the infrastructure calls my bad ideas stupid before executing them, which keeps me honest.

Keeping the money under control

The failure mode of agent systems isn’t the wrong answers you’d expect, it’s the cost- so the rule here is a waterfall: free local models first, paid cloud only when the task actually needs them. Local inference goes through a model-router that every LLM call in the house is forced through, which makes it the single chokepoint where every token gets counted. Consumers ask for a capability tier (coder-s, coder-m, reason-m, extract-m, embed, parse), not a vendor, so backends can be swapped underneath without touching a single consumer. The routing is the clever bit: the router knows which backends are up, what each alias actually costs, and when a task has outgrown the local tier, so escalation to a cloud model is a routing decision made per request, not a config change somebody has to remember to revert.

What justifies the hardware isn’t the chat traffic, it’s the knowledge-extraction workloads. Turning a project engagement’s document corpus into structured entities (the knowledge graph pipeline has its own write-up) means millions of tokens per run, and at cloud API prices those runs would be a monthly bill with a comma in it. On owned GPUs the marginal cost of a run is electricity, which changes what you’re willing to attempt: re-running a whole corpus because the ontology improved stops being a budget conversation.

The discipline runs on hooks rather than good intentions. Every agent definition has to carry an explicit model pin, and a check blocks any that don’t. That rule exists because one agent once shipped without a pin, silently inherited the most expensive model in the house, and ran 109 jobs on it before anyone noticed. There’s also a two-strike rule for debugging: if the orchestrator runs two diagnostic commands without resolving an issue, it has to hand off to the cheap debugging agent instead of burning premium tokens on hypothesis roulette. And a weekly scorecard reads the transcripts and pings me over Telegram if the delegation discipline is slipping.

The iron

The GPU rig is one big Proxmox host with an EPYC-class CPU and a stack of datacentre GPUs passed through to VMs, enough of them to pool around 160GB of VRAM into a single VM, with a separate CUDA guest for the small always-resident models. The local workhorse is a fleet of mid-size open-weight models (the ~26B class) under llama.cpp, one per GPU, with an HAProxy gateway routing the capability aliases across them. The models put themselves to sleep after sitting idle and wake on demand when a request comes in, because GPUs spinning to serve nobody is the kind of thing you only let happen once. The really big reasoning tiers fall through to cloud APIs, and everything routine stays on the LAN.

I measure the power rather than guess at it. There’s a Refoss EM16P energy meter clamped onto the panel with a labelled circuit per feed, and it streams into the Power tab of the home dashboard. Idle, the whole rig draws about 180W. When an extraction run lights up every GPU at once it can spike to 3200W, which the air conditioning then gets to hear about:

That graph is the house’s cooling circuit reacting to a Boston summer day with a compute rig in the basement. The dashboard knows which circuit is which, so “what did that overnight batch job actually cost” is a query, not a guess.

Newer machinery

The harness is never finished. Half of it exists because something went wrong in an instructive way, and lately a few things have gone wrong instructively.

Running several agent sessions at once used to mean they’d fight over the same git working tree and quietly corrupt each other’s edits. Now each session gets its own throwaway git worktree, and a reaper sweeps up the ones left behind when a session dies mid-thought. Deeply unglamorous concurrency hygiene, but it’s the difference between “run three of these at once” and “don’t.” The same instinct runs through a shared documentation tree that lives on every machine at once: changes get committed, rebased on top of whatever the other boxes did, and pushed, so the reference material is the same everywhere and no single machine is quietly the master copy.

Review got cheaper and more paranoid. Where @overseer is the deliberate panel you convene for a big call, there’s now an automatic one that runs on every turn whether you asked for it or not: each diff gets a fast second opinion from a different vendor’s model, a tripwire that pipes up when a change smells wrong. To keep that from costing real money, a small local model triages the claims first and only escalates the ones worth a paid judge’s attention. And because a watchdog that has quietly gone blind is worse than no watchdog at all, a periodic canary re-runs known-bad cases past the reviewer to confirm it still catches them. A watchdog watching the watchdog.

The experiment I’m most interested in right now is splitting the labour by cost, and it genuinely is an experiment, not something I’d bet the house on. The frontier model does the part actually worth paying for: the planning, the judgment, working out what “done” looks like. A small local model does the mechanical execution, the two talking over a channel between them. The whole thing runs inside a clone of the production box, so when it does something stupid it does it somewhere that can’t take the real system down with it. Whether it turns out to be the future or a dead end I honestly don’t know yet, which is exactly why it’s a clone and not prod.

What it actually does all day

Unglamorous things, mostly. It keeps documentation in sync with what’s actually deployed and audits the drift. It deploys services to the LXC fleet and mirrors every live edit back to git, because config that only exists on a host is a time bomb. It writes its own cost reports. And it rebuilt this website off Wix and now maintains it: the page you’re reading was drafted, built, and committed through the harness, with me as the review gate.

It’s also crept into running the house. The same dashboard that watches GPU state and power draw now watches the cats, two of them, both wearing AirTags, their rough whereabouts plotted on a map so I can tell at a glance whether anyone’s gone wandering. (No, I’m not printing coordinates. They’re cats, not fugitives.) A household to-do, shopping and notes system sits behind it too, backed by Notion so the phone app in the kitchen is a first-class client and not some bespoke thing only I can drive. Digests and alerts go out over Telegram. None of this is load-bearing infrastructure- it’s just what happens when you’ve got a general-purpose harness lying around and a domestic problem in front of you.

Decisions that matter get written down as architecture decision records, and the agents read those before acting, so a call made in March doesn’t get quietly relitigated by a model in July. Open work lives in GitHub issues with claim leases, so two sessions can’t silently do the same job twice, and every commit carries a trailer naming the initiative it belongs to. None of this is novel, and that’s sort of the point- the practices that keep a human engineering team from tripping over itself, issue trackers, decision records, code review, post-incident write-ups, turn out to be exactly what keeps a crew of agents honest too. The agents didn’t need new discipline invented for them. They needed the old discipline actually enforced, which hooks are better at than managers ever were.

That, honestly, is the load-bearing trick of the whole thing: the harness is only as good as the written-down context it runs on. The models are interchangeable, but the context isn’t.

Where it goes next, I’ve no idea. The whole field reshapes itself every month, and half the harness exists because something went wrong in an instructive way. Still curious what it looks like when the plywood bench is the normal way software gets run, and not the weird one.