The Stop Rule

At a hackathon in San Francisco, I set myself one constraint: do not touch the screen for eight hours.

I picked the problem my corner of K-12 has been losing to for years: scheduling. Every student on a caseload is owed a set number of service minutes, the provider covering them works across schools whose bell schedules do not line up, a missed session has to be made up inside the same window, and some students can be grouped while others legally cannot. Get it wrong and a child is owed minutes nobody delivered, which is the kind of wrong that shows up in a compliance report months later.

Almost nothing on the market solves this well. What I have seen either hands the scheduler a calendar and wishes them luck, or runs a search for hours and returns something a human quietly fixes by hand anyway. Meanwhile the math underneath moved. Constraint solvers and Bayesian optimizers handle instances this size in seconds. They are all open source, and an agent can now assemble the solver, the fixtures, the bell-schedule parsing and the compliance report without me typing any of it. A problem that used to be a quarter of engineering roadmap is worth one honest attempt in a day.

So I wrote the brief, wrote down what done meant, started the agent, and left it alone.

What broke first was my finish line. The acceptance check I had written as the definition of done took thirty minutes to run, because on an over-constrained week it searches for a schedule that does not exist and only knows that once it has exhausted the space. A loop that wanted feedback every fifteen minutes was queued behind a half-hour gate, and I spent the afternoon rewriting the tests instead of watching the thing work. The agent was fine. The finish line was the part I had gotten wrong, and it took the run with it.

The rest of this setup exists at that seam, and most of it is machinery for deciding what finished means in a form that holds while I am not watching.

A harnessharnessa structure an agent works inside, built on verification and stop rules, so it can run unattended without drifting off task is the structure that lets an agent run without me over its shoulder. Loop engineering is how I drive it. The harness is the car.

The pieces are ordinary. A brief, usually a project or a set of tickets. A rubric stating what done means in a form a machine can grade. Lanes of work dispatched in parallel where they are genuinely independent. A verification gate in a fresh contextfresh contexta separate session that has never seen the work being checked, so it cannot inherit the original’s assumptions, told to refute rather than agree. A stop rule.

The rubric is where the hours go. On any task whose end state is not already checkable, writing the rubric and the gate script comes before any product code, because everything downstream is an argument about whether the thing is done, and that is an argument the agent wins if I left it undefined. It also cannot live in the conversation. Verification happens in a fresh session by design, so the rubric has to be a file on disk that a session with no memory of the work can read and apply. At the hackathon, the rubric was the only artifact I touched all day.

# Rubric authoring rules
- One line = one command + one threshold. "pytest -q exits 0 at >= 200 hypothesis
  examples", never "tests are comprehensive".
- Negative tests included: fixtures that MUST fail or produce a shortfall report.
  This is what kills false convergence.
- Grep gates for vocabulary constraints (banned words, attribution): cheap and
  deterministic.
- Anchor every line to an artifact a fresh agent can check without the builder's
  context.
- The rubric is committed before any product code. Its git history is the
  orchestration story.

A rubric made only of things that must pass can be satisfied by an agent that quietly narrows what it attempts, so every rubric I write includes fixtures that have to fail and a report that has to come back short. Otherwise the loop converges on an easier problem than the one I set.

Every loop needs three exits, and most of the ones I see written down have one.

Success is the verifiable end state that means it worked. Failure is the condition where it gives up instead of trying a fourteenth time. Budget is the hard ceiling on tokens, steps, or wall clock, independent of the other two, because an agent making steady plausible progress toward the wrong thing will spend a whole window doing it.

Success is the exit everybody writes. Failure is the one everybody forgets, which is why an unattended run at 2am is either finished or still going at breakfast.

In my setup, a small model evaluates the success condition from the transcript. It has no tools of its own, and the condition is capped at four thousand characters. A condition like the tests pass is therefore unjudgeable unless the worker’s own output shows the tests running. The condition has to be written so the transcript proves it: does the transcript contain a passing exit code from a command the agent actually ran. Written loosely, the judge passes a run whose tests never executed.

/goal scripts/rubric.sh exits 0 with every line PASS, scripts/e2e.sh passes against
the deployed URL, and .claude/out/verdict.json (written by the verifier agent) says
PASS with no open findings. Constraints: never weaken a test or rubric line to make
it pass; after any verifier rejection, re-read BRIEF.md before continuing.

An agent that cannot pass a rubric line will eventually consider editing the rubric line. An agent corrected by a verifier drifts further from the brief with each patch unless something sends it back to read the brief again.

Agents overrun by default. Across 524 harvested sessions, 213 carried at least one correction from me. Those collapse to 88 distinct instructions, and 62 of the 88 are restrictions.

Seven in ten of the things I type at an agent are a boundary rather than a direction. Only 19 of the 88 are me changing my own mind. A deliverable I framed as propose comes back applied. A bounded audit turns into a refactor. The stop condition is the only part of the harness that pushes back before I have to.

Computational checks are deterministic and fast: tests, linters, type checks, a script that counts files. Inferential checks are semantic and slow: a model reviewing a diff, a judge grading against a rubric. Computational checks always run first, so model judgment is only spent on work that cleared the cheap gates. Reverse that order, and a review agent burns twenty thousand tokens explaining a change that would not have compiled.

A verifier can become the bottleneck. A check slower than the loop’s own cadence does not make the loop safer, it stalls it. I size the gate against the cadence first now, and thoroughness second.

The verifier has to match the claim. A review agent signed off on a number that a data-mining agent later traced to the wrong column: two columns with near-identical names, one of them the real one, and nothing in the code looked suspicious to a reader who could not query the warehouse. A claim about internal data needs an agent that can reach the data. A claim about the outside world needs one that can fetch primary sources. A generic verifier pointed at both returns confident prose about the half it could not check, which is worse than no verifier, because it clears the gate.

Whatever the verifier finds comes back as a failure reason rather than a verdict. A bare fail gives the next iteration nothing to act on.

A stop rule ends a loop. A guardrail stops a single action before it happens.

Starting a dev server, driving a browser, or writing a throwaway script to check your own work are opt-in: you climb there when I asked for a running check, not as a display of diligence. That rule sits in an always-loaded file, stated plainly, with the reasoning attached.

When the reflection loop counted, 59 of 390 sessions had done it anyway.

A rule is context, competing for attention with everything else in the window, and helpfulness wins that competition more often than I would like. A hookhooka script that fires before a tool call and can block it outright, ahead of any permission check and regardless of bypass flags does not compete for that attention, because it runs before the model is consulted at all. The trigger I use now is repetition. One violation is noise. A rule violated across dozens of sessions is a rule that will never hold as prose, and it moves into the path that week.

# teardown is always allowed, reaping strays must never be blocked
printf '%s' "$cmd" | grep -Eq '(^|[^a-zA-Z])(stop|kill|pkill|down|teardown|reap)([^a-zA-Z]|$)' && exit 0

for pat in "$server_launch" "$pkg_dev" "$http_mod" "$browser_runner"; do
  if printf '%s' "$cmd" | grep -Eq "$pat"; then
    deny "this launches a dev server or browser runner, an opt-in check. Prefer the repo
          gate, reading the changed call paths, or one-shot commands against real input.
          If the user asked for a running check, re-run with CLAUDE_VERIFY_OK=1."
  fi
done

Teardown verbs pass unconditionally, because a guard that blocks cleanup manufactures the orphaned processes it was meant to prevent. There is exactly one escape hatch, an environment prefix, so the legitimate case costs a keystroke instead of a fight. The denial names the cheaper alternatives, since a hook that only says no sends the agent hunting for a way around it.

The second hook came from watching my laptop swap itself to death.

Linting a large monorepo end to end peaks near 3.6 gigabytes. Run several agents in parallel, tell each to verify before reporting, and every one of them re-checks the entire tree: the same work, N times over, all of it resident, on one machine. The obvious fix is to tell the agents not to do that, and I did, and it held about as well as the last rule did.

So there is a machine-wide lock now, one full-tree verification at a time across every session and both of my profiles. A scoped run, one that names a workspace or a path, passes straight through and is never gated, so narrowing a check is the fast path rather than a penalty.

# mkdir is the atomic primitive here on purpose: shlock validates a lock by kill(2)-ing
# the PID it stored, and a PreToolUse hook exits the instant it returns, so shlock would
# judge the lock stale on the very next check and never serialize anything.
if mkdir "$LOCK_DIR" 2>/dev/null; then claim_lock; exit 0; fi

# Re-entrant for the owning session: a session running its own checks back to back must
# not deadlock against itself if a release was ever missed.
[ "$holder" = "$session_id" ] && { claim_lock; exit 0; }

# A crashed or SIGKILLed session never fires PostToolUse, so the lock must be self-healing
# or it wedges every sibling session permanently.
if [ "$held_for" -gt "$LOCK_TTL_SECONDS" ]; then
  rm -rf "$LOCK_DIR"; mkdir "$LOCK_DIR" && claim_lock; exit 0
fi

A hook process exits the instant it returns, so any lock validated by checking whether its owning process is alive judges itself stale a second later and serializes nothing at all. It looks like it works, because nothing errors and every check still runs.

Every session ends with a hook that appends it to a queue, and once a week a loop reads the new entries, clusters them into candidate lessons, and routes each one by altitude: a rule file, a repository’s own instructions, the second brain, project memory. Building it cost four rounds of it being confidently wrong about me.

Every proposal is numbered, carries its evidence, and waits. The cursor tracking how far the queue has been read advances only after I answer, so declining a batch does not quietly discard it. Every count in this piece comes from the fixed version of that loop.

The harvester was mining whole transcripts for corrections, including transcripts of its own runs, where phrases like stop doing that appear constantly as quoted evidence. It was reading its own output as fresh signal about me. Now it counts only my turns, and the loop’s own sessions are marked and skipped outright.

Counted raw, my most urgent recurring correction was a single sentence about not syncing something to Google, which appeared in 41 sessions. It was typed once. Resumed sessions carry their earlier turns forward, so one instruction from one afternoon reappears in every later transcript of that workstream and reads as a standing complaint.

Every bar in that chart is one sentence I typed once. The whole ranking is an artifact of how sessions resume, which is why the same 399 correction lines collapse to 88 once you count distinct instruction plus context. A real pattern looks different: the same complaint in five distinct phrasings across four repositories. The second and last bars survived both counts. They are the rule that became a hook.

The knowledge lane returned 70 keepers from 160 sessions. They were true, durable, nicely phrased restatements of things the vault already held, because an agent judging novelty without seeing the corpus cannot tell new from known and keeps whatever sounds worth keeping. The full inventory of existing note names goes into every batch now, and each candidate comes back marked new, extends an existing note, or already covered. A healthy run is mostly extends.

For a while the knowledge half was a separate skill that this loop deferred to, and that skill recorded 0 invocations across 406 sessions while its own queue sat at 505 unprocessed entries out of 506. Deferring to it was, in practice, throwing away everything it was meant to receive. I merged it back into the loop that actually fires.

I do not run this reflection loop on a schedule anymore. Scheduled runs on this machine are unreliable for reasons that have nothing to do with agents, and a background job that quietly does not fire is worse than no job, because you stop checking.

Even when it fires, it can fail in a shape that looks like working. For a stretch the digest step produced files on every run, on time, with content in them. The content was a small model politely explaining that it could not see a transcript in the message.

# Run a command with a hard wall-clock bound (no coreutils `timeout` on this machine).
# NOTE: the job is backgrounded, and a background job in a NON-interactive shell inherits
# stdin from /dev/null. Never feed a bounded command through stdin: pass its input as an
# argument. (Observed 2026-08-01: `_bounded 90 claude -p "$P" <"$raw"` reached the model
# with empty stdin, so every digest stored haiku replying "I don't see a transcript in
# your message." An isolated interactive test did NOT reproduce it: job control differs.)
_bounded() {
  local secs="$1"; shift
  "$@" & local pid=$!
  ( sleep "$secs"; kill -9 "$pid" 2>/dev/null ) & local watchdog=$!
  wait "$pid" 2>/dev/null; local rc=$?
  kill "$watchdog" 2>/dev/null; wait "$watchdog" 2>/dev/null
  return "$rc"
}

Each digest is backgrounded to bound its runtime, and a backgrounded job in a non-interactive shell takes its input from nowhere, so the transcript I was piping in never arrived. Testing it by hand did not reproduce it, because job control differs in an interactive shell. Nothing errored, nothing came back empty, and every file on disk was junk. It runs when I run it now, weekly, about ten minutes of skimming.

I also cut a guard the agent proposed before every production deploy, because redeploying that site is a non-issue and the guard would have taxed something that happens weekly and breaks nothing.

The one that still bothers me failed silently. I renamed my hook scripts mid-session to fit a naming convention, and the configuration kept pointing at paths that no longer existed. A hook that cannot launch does not block anything, it does nothing, so the session carried on with its guards off and nothing on screen said so. Failing open is the right default for a guard, so after I touch one I make it deny something before I trust it again.

The problem I have not solved is review. A long autonomous run produces a large diff, and a large diff is a worse artifact for a human reviewer than five small ones, even when every line is correct. My current answer is to let it run to completion in an isolated worktree, then split the result into layered pull requests once I know what the end looks like. Splitting after the fact is fiddly and conflict-prone. Doing it as it goes would need the agent to leave breadcrumbs marking the load-bearing changes while it works, and I have not built that.

Everything in this series is machinery: context an agent is missing, a memory that outlives the session, a tool layer, a harness that checks itself.

No model inherits my standards, and no amount of capability produces them. A rubric is my taste written down in a form a machine can apply, which is the only reason any of that machinery knows what it is aiming at.

No system I run volunteers a stop. Every autonomous run I have built ends because I decided in advance what finished looks like, and every one that went badly went badly at that decision.

Reading the transcripts matters more than the rubric or the stop rule, and it is the part I still have to make myself do. The machinery is ordinary: a few rules, a place to keep what I learn, some hooks in the path, a loop that reads its own logs on Sundays. Twenty minutes a week of reading what my agents actually did has changed more of it than any model upgrade has.