The Fourth Rule of The Foundry

“The Fourth Rule of Project Mayhem is that you cannot lie.” ~ Chuck Palahniuk, Fight Club

In part one I said the first rule of The Foundry is that you talk about The Foundry: the agents I built report what they did, admit what they broke, and leave a paper trail. I still believe that. What I hadn’t thought hard enough about was the gap between reporting what you did and reporting what you didn’t do, and that gap is where my investigator agent had been lurking. Its reports were tidy, confident, and well organized, the kind of onboarding summary I’d have been happy to get from a human engineer, and on a bad day they were missing about a quarter of what was in the code, which is also a lot like a human engineer, just a very junior one. Nobody lied, exactly. It just never mentioned the files it hadn’t opened.

I noticed the missing pieces only because I was reviewing my team’s work, something that I think is critical in agentic development—at least at first. To really make these things work well, you need to have experience in the field. Otherwise, you don’t know what to ask, and you don’t know whether the answers you’re getting are accurate and make sense, or just aggressively wrong.

Those gaps are a big part of why the team has grown since August, so this post is also an update. There are ten agents now instead of nine, plus four skills that my own Claude Code session runs, and the whole thing installs as a Claude Code plugin. Rather than walk through every member in a wall of words, here’s the roster, with what each one does and why I set it up that way:

Rule four

Reviewing the work told me something was off; a test told me how much, and it’s one I should have run a lot sooner. I built a 191-file practice repository, planted 48 specific facts in it (things a good investigator ought to find and report) along with a few decoys, and ran the old investigator against it three times. It found 35 facts, then 35 again, then all 48. It made nothing up in any run, which already puts it ahead of plenty of status reports I’ve read. The third run proved it could find everything, so the problem was diligence: handed a big pile of files and told to read them all, it skimmed. This is a known issue with LLMs, and something I write about in my Prompting Claude series.

The fix was to stop asking one agent to read everything. In version 2.0, investigate hands off every file to its own file-reader, ten at a time, and splits anything over about 1,000 lines into overlapping sections. File-reader holds the Read tool and nothing else, so it can’t edit anything, wander off, or start agents of its own; it reads its one file and reports what’s in it, with line numbers. Investigate then checks the readers’ results against its own list of files, sends a reader for every gap, and ends its report with a Coverage section that names every file nobody read. On a repository over 500 files, it stops and asks me before it starts. Against the same practice repo, three runs, the new version found all 48 facts every time and made up none.

What it costs

Now the part that doesn’t help my case. The old investigator cost about 47 cents a run on that test, and the new one costs about $2.31, roughly five times as much. With every agent on Sonnet, the mid-tier model, it ran about $5.57. Moving everything to Haiku, the cheapest model, was cheaper still, but one run lost its entire report, and every run was slower, so the readers run on Haiku and investigate stays on Sonnet. I made that call, and I’d make it again: five times the price of a skimmed report is still cheap next to a plan built on files nobody opened. Your math may differ, especially on a big repository, since the bill grows with the number of files.

There’s one more wrinkle, and it’s my favorite. The test harness’s first verdict said the new version failed, because the scorer counted decoys as made-up facts even when a report had marked them dead. I fixed the scorer, rescored every stored report, and the new version passed. Then the Haiku version failed the same way, on decoys a report had marked dead under a bold label, and I fixed the scorer a second time. So I’ve now lived the robots-grading-robots problem, and the robot that needed the most correcting was the one doing the grading. Quis custodiet ipsos custodes, Juvenal asked; he was talking about guards and wives, not test harnesses, but the question works here, and sometimes I just need an excuse to trot out random literature quotes.

How I actually use it

I use the team for just about everything code-based that I do now: investigating older codebases of mine, finding software errors (again, mostly in my old code), and building new software, with me as the director and the team doing the work. I’m still in the review-it-myself stage, to a certain extent, but my confidence is growing with each task.

The shape is simple even when the work isn’t. My main Claude Code session leads and usually runs a more capable model (Opus or Fable): it briefs each agent on what it needs, reads what comes back, and owns every commit and push. A new feature starts with brainstorm, which asks me numbered questions and builds nothing until I approve the design. The architect turns that design into a plan, run-plan builds it task by task with test-engineer trying to break each one, and ship walks the result out the door, asking once with every step listed. I run every production command myself, and ship is written so that it never runs one. It’s slower than asking one model to just do the thing, and I won’t pretend otherwise; for a small, fully specified change, the pipeline steps aside and the work just gets done.

The fifth rule

Part one’s rule still holds, since a team that reports what it did is worth more than one that doesn’t. A report is only as good as what it leaves out, though, and I didn’t know what mine left out until I measured it. The Foundry is still free, still MIT licensed, and still at github.com/SomeClown/the-foundry . If you install it, try it on something you know cold before you hand it something you don’t, because Palahniuk’s narrator found out the hard way where the next rule leads:

“The Fifth Rule of Project Mayhem is you have to trust Tyler.” ~ Chuck Palahniuk, Fight Club

I’d rather measure Tyler. So far he’s earning his keep, at about five times the old rate.