Write the standards, not the steps

Early this year an autonomous research agent had to work in a domain it was never built for. Different records, different rules, different questions. The honest expectation was a couple of weeks of rework.

It took two days. Copy the system prompt across, delete the two tool registrations that only made sense in the old domain, point it at a different knowledge base, done. Nobody rewrote the reasoning.

That story usually gets told as “the architecture generalised,” which is true and tells you nothing. The commits say something narrower and much more useful. Almost the entire prompt described how an analyst is supposed to behave. Almost none of it described the domain. Behaviour ports. Domain knowledge doesn’t. The ratio between the two is the whole story.

What it was

An autonomous research agent. Give it an open-ended question — are these two records the same thing, why is this group failing its checks, how far does this problem actually extend — and it comes back with an evidence-backed report someone with domain expertise can act on. Not a chatbot. The unit of work was an investigation that used to eat an analyst’s afternoon, sometimes their whole day.

One decision behind it took real pushback at the time, and it’s the one that mattered. The frontier models were getting better every few weeks. Spending the prompt teaching the model how to investigate felt like a losing bet. So it mostly doesn’t. It says who the model is and what it’s accountable for, and leaves the method alone.

That was the intent, at least. The first cut hedged on it badly, which is worth coming back to.

The file opens on a job description. You are an investigative analyst. You do not blindly trust the records you are given. You do not blindly trust the automated systems either — and when your evidence contradicts one of them, you are expected to override it and say so out loud. Thoroughness over speed. Keep going while each step is still adding evidence. Stop when it stops, not when you hit a limit somebody else picked.

Then the standards. How to weigh a piece of evidence before leaning on it. How to calibrate confidence. What to do when the evidence refuses to resolve. How to report a null result instead of manufacturing a finding to justify the run. An estimation audit: before writing any number, label it counted, verified or extrapolated — and never dress an extrapolation up as a count. An adversarial self-check: if the main conclusion is wrong, what’s the likeliest way, and was that signal already sitting there, dismissed?

Read that list again and notice what’s absent. Not one line of it is about the domain. Those are the standards of a careful analyst, and they hold whatever the records happen to describe. Crossing into a new domain, every one of them came along for free.

The proportion is the finding

The domain-specific part turned out to be a single section — how identity and duplication worked among that particular set of records. Everything else was accountability, judgement and standards. Call it nine parts standards to one part domain.

Which is exactly why the port wasn’t a rewrite. Swap the knowledge base. Delete the two tools that belonged to the old world. Rename the leftovers.

One thing helped mechanically, though it isn’t the lesson. The prompt never names its tools. It describes tool categories by purpose — something to ground you in the rules before you query, something to map distributions across a large dataset, something to inspect one record in depth, something to check the outside world — then says tools are registered dynamically, go discover what you have and read each description. That started as laziness. Tools were being added and renamed constantly, and a hand-maintained list in prose would have been stale within the week. The happy accident was that deleting two tools broke nothing in the text. Convenient. The prompt would have ported without it.

A prompt that teaches method ages with the model. A prompt that sets standards outlives it.

The scaffolding that came out

The first version was not this clean. It had plan, execute and verify as three separate, sequential stages — the decoupled-planning family, which ReWOO (Reasoning WithOut Observation, 2023) formalised as planner, worker, solver: write the plan up front, run the tools against it, compose the answer at the end, and never let an observation reach back into the planning. The opposite pole is ReAct, where reasoning and acting interleave and every observation shapes the next thought.

Staging felt responsible at the time. It’s legible, you can point at which box failed, and there is real comfort in that when you’re about to hand serious work to something non-deterministic.

It was also a ceiling. A model made to walk plan, then execute, then verify will walk them in that order even when the evidence in front of it argues for returning to the plan, or for verifying one thing before executing the next. The stages were doing what a nervous author wanted, not what the investigation needed.

So they came out. What replaced them is closer to ReAct with standards bolted on: the model interleaves however it likes, and the prompt constrains what counts as acceptable work rather than the order the work happens in.

Worth noticing: those stages ported to the new domain perfectly well. They were domain-neutral. Portability and quality are two different questions, and a prompt can be portable and still be holding the model back.

Evaluation, or the lack of it

The first version’s whole output contract was one page of markdown, the way a good analyst hands over a one-pager. That was it. And because the deliverable was a document rather than a label, there was nothing to score it against. There’s no ground truth for “was this a good investigation” — two competent analysts write different one-pagers about the same problem and both can be right.

So the method was crude. Every run asked whoever requested it whether the output was useful. Roughly 98% came back approved.

That number is genuinely hard to read. Maybe the agent was excellent. Maybe the bar was soft. The most defensible reading is that it measured the format — a one-pager is legible enough that someone who knows the problem can tell inside ninety seconds whether you understood it. Approval on that says a lot about relevance and readability. It says very little about correctness.

And it held up exactly as long as the agent stayed advisory. The day people started acting directly on the output, a thumbs-up per run stopped being enough, and closing that gap has been most of the work since. Starting there still looks right, though. Building a formal eval harness first would have meant a quarter spent measuring something nobody had confirmed they wanted.

The transferable bit

Write down the standards, not the steps.

The standards are the part you know and the model doesn’t. What counts as evidence here. When to distrust your own data. What you are never allowed to guess. What to do when the honest answer is “I can’t tell.” Those survive the next domain, the next team, and the next model release.

The steps are the part the model can work out for itself. Every one you pin down is a decision taken away from something that may well reason better than you about that specific case.

Nine-tenths of that prompt was standards. That’s why it moved in two days.

One last thing: none of this would have worked on a weaker model. That first version shipped on Claude Opus 4.6, and the entire bet — describe the standards, leave the method alone — only pays off if the thing on the other side can actually be trusted with the method. It could. Thanks are due for that.