Briefing · quality controls for agent-written code

gap-trap

gap-trap enforces best practices in a way agents can better understand and consistently follow.

It is also self-improving. Agents will always make some mistakes, so every mistake that gets through becomes a new rule and check.

github.com/pliablepixels/gap-trap

Writes the rules for coding agents from the codebase, enforces them with scripts in CI, and has every bug fix add a rule and a check, so the rules improve over time. In spec-driven development it is the loop inside each task.

Before · SDD with an instructions file

Thin instructions, followed some of the time

Spec-driven development (SDD) defines each task well. Inside the task, two things go wrong. The instructions are thin: people write a short file from memory, so many of the codebase's real rules never make it in. Agents drift from them: following instructions is probabilistic, and it gets less reliable as the context fills up.

STEP 1 OF 4Spec and tasks. The spec states the goal and the acceptance criteria, and gets broken into tasks.
STEP 2 OF 4The agent builds the task. Its instructions cover the few rules someone wrote down. It runs the tests, linter and type checker, and CI runs them again. Those catch broken code. They enforce a project rule only if someone already wrote a lint rule or test for it, so a raw fetch() or a stray console.log usually passes.
STEP 3 OF 4PR review. A person checks the result against the spec. Agents write code faster than people can read it, so reviews get lighter.
STEP 4 OF 4Drift builds up. Duplicate helpers, tests that pass on broken code, rules the agent ignored. A fix may get a note in the instructions file, but a note is text, not a check, so the next session can repeat the mistake.

After · gap-trap as the inner loop of SDD

A rulebook agents can follow, enforced in CI

The spec and tasks stay the same. Inside each task gap-trap adds three things: a rulebook written for agents, checks in CI that enforce it, and self-improvement, where every miss adds a new rule and check.

STEP 1 OF 6Spec and tasks, unchanged. SDD still defines what to build. gap-trap works inside the "build a task" step.
STEP 2 OF 6The agent loads the rulebook. gap-trap wrote it from the code and git history. Rules are numbered, and any rule a script can check names that check. Contracts give one approved path per area. For example, all HTTP goes through http.ts, which adds the auth token, and all logging goes through the logger, which redacts tokens. The always-loaded part has a word cap.
STEP 3 OF 6A check rejects the change. While debugging, the agent called fetch() directly and added console.log(response), which would put the auth token in production logs. Two grep checks fail, one per contract, and the output names each rule and the file. The agent fixes it and tries again.
STEP 4 OF 6Checks pass and the PR merges. All four checks are green. The PR description must include the spec's acceptance criteria, and a feature PR must link its spec or say why it has none. CI only checks that these are filled in. Whether the code meets them is the reviewer's call.
STEP 5 OF 6Self-improving: a miss becomes a rule. When a bug, a review comment, or a wasted session shows a gap, the rulebook tells the agent to add a rule in the same PR as the fix, plus a check if a script can catch it. A person approves the rule change. After that, CI catches a repeat.
STEP 6 OF 6Periodic sweep. Someone runs /gap-trap refine. It reads git history since the last sweep, finds reverts and areas that keep getting fixed, and opens one PR that proposes new checks and deletes rules that never prevented anything.

Part 1 · a rulebook written for agents

gap-trap writes the instructions, in a shape agents follow

Typical hand-written CLAUDE.md

## Commands
- npm run dev
- npm test        # vitest
- npm run lint

## Stack
- React + TypeScript, Zustand for state
- API client in src/lib/http.ts

## Conventions
- Use the http client, don't call fetch directly
- Use the logger, not console.log
- Write tests for new features
- Keep components small

Reasonable rules, but they're advice. "Small" has no number, "write tests" has no bar, and nothing fails when a rule is broken or when http.ts moves. Lessons from past incidents usually aren't in it.

Written by gap-trap

- I2. Destructive operations need error handling
  and a recovery path. Lost user data cannot be
  patched later.
- P2. Test first: a failing test precedes every
  feature and bugfix. Gate: proven-red CI job.

### HTTP
Owns:  all network requests, auth headers
Path:  src/lib/http.ts
Never: raw fetch or axios
Gate:  instruction-gate test

### Logging
Owns:  all log output, token redaction
Path:  src/lib/logger.ts
Never: console.* outside the logger
Gate:  instruction-gate test

Written from the code. Setup reads the codebase to find the real helpers and modules, and turns reverts in git history into "don't retry this" notes.

One fixed shape. Every rule has an ID, one statement, and a short reason. If a script can enforce it, the rule names that check. Docs and agent briefs cite "P2" instead of copying the text, so a rule changes in one place.

Small core, details on demand. The always-loaded files stay under a word cap. Topic playbooks load only for work in their area, which leaves more of the context for the task.

Existing rules kept. Rules from an existing CLAUDE.md or .cursorrules are migrated one at a time, and the setup plan shows each one. Other tools' files become pointers to the single rulebook.

The hand-written file is an illustrative example. The gap-trap rules are from its templates, with the wording lightly shortened.

Part 2 · checks in CI

Four checks, with and without

1

Contracts

grep over the source

One approved module per subsystem. Any other way fails the build.

Without

Debugging a sync bug, the agent calls fetch() directly and adds console.log(response). Tests pass, so it ships, and auth tokens start landing in production logs.

With

Both contract checks fail and list the file: HTTP: no raw fetch or axios outside lib/http Logging: no console calls outside the logger

2

Proven-red tests

tests rerun on pre-change code

A new test must fail on the code from before the change, or it isn't testing the change.

Without

A rounding fix comes with a test that only checks the total renders. It passes with or without the fix.

With

CI runs that test on the pre-fix code, where it passes, so the job fails: 1 changed test file(s) pass on the pre-change code

3

Ratchets

baseline file in the repo

Counts of known problems may go down, never up.

Without

37 files are over 400 lines. The rule is advice because old code breaks it. Next quarter it's 45.

With

The baseline says 37. A PR that adds a 38th fails: oversizedFiles: 37 allowed, 38 found (+1)

4

Rulebook checks

tests on the instruction files

Every file and function the rulebook names must exist, and the rulebook has a word cap.

Without

http.ts is renamed to api-client.ts. The rulebook still points at http.ts, so agents follow a rule for a file that's gone.

With

The rename PR fails until the contract is updated: HTTP: path src/lib/http.ts missing

Every check also confirms it measured something. Scanning zero files, a crashed counter, or a test runner that never started is an error, not a pass. Numbers, file names, and the logger's redaction here are examples; the error text is what the scripts print.

Part 3 · self-improving

Every miss becomes a new check

Checks only catch what someone thought of. When something gets past them, the rulebook tells the agent to add a rule in the same PR as the fix, plus a check if a script can catch it. A person approves it. A periodic sweep finds lessons nobody recorded and proposes deleting rules that never caught anything.

Without

An agent formats a date with the wrong timezone, and it ships

↓

Someone fixes it in a chat session

↓

The session ends, and the lesson goes with it

↓

Next month another agent makes the same mistake

With

MissSame timezone bug ships.
Fix PRFixes the bug and adds a contract: dates go through lib/date.ts, never toLocaleString. Adds the grep check for it.
ReviewA maintainer approves the rule change like any diff.
EnforcedThe next agent that formats a date by hand gets a CI failure that points it at lib/date.ts.

Periodic sweep · `/gap-trap refine` reads git history since the last sweep

Found a reverted change→adds a "don't retry this" note citing the commit

Found four fixes in the auth code→records the fact those fixes kept missing

Found a rule with no incident behind it→proposes deleting the rule

Every sweep proposal must cite a commit, a count, or check output. The always-loaded files have a word cap, so new rules compete for space. The timezone story is an illustration.

Limits

What still needs people

What gap-trap does on every pull request

Every change, whether written by an agent or a person, goes in as a pull request (PR). On each PR gap-trap:

runs the four checks, and blocks the merge if any fail
gives the PR description a template with sections for the spec's acceptance criteria and a link to the spec
fails if the acceptance section is empty, or if a feature PR has no spec link

A person still approves the PR. Scripts can't judge everything, so the reviewer covers the rest.

When reviewing a PR, a person checks

Bug-fix PRs add a rule. The agent is told to add a rule and a check with every fix, so the bug can't come back. No script forces it.
Rulebook edits don't weaken a rule. CI flags a rule that points at deleted code. It won't flag "never" being softened to "avoid".
The acceptance criteria are right. CI only checks the section isn't empty. Whether the criteria match the spec, and whether the code meets them, is the reviewer's call.
Existing tests weren't weakened. Proven-red judges new tests closely. An existing test that was edited can slip through if another test in its file fails.

Someone on the team owns

Making the checks required. CI checks block a merge only if the repo's branch protection marks them as required.
Running the sweep. Nothing schedules /gap-trap refine. Someone runs it after an incident or about once a month.
Reviewing the first contracts. Setup writes them by reading the code. A wrong one misleads every later session, so check them carefully once.
Project-specific setup. Which problem counts the ratchet tracks, and an optional mutation test (breaking code on purpose to confirm the tests notice).

Summary

SDD alone vs. SDD with gap-trap

	SDD + instructions file	SDD + gap-trap
What to build	The spec	The spec (unchanged)
Following the rules	Probabilistic, less reliable as context fills	Checkable rules must pass CI
Test quality	Tests must pass; nothing checks they can fail	New tests must fail on the old code
Known problems	Drift up	Can only go down
Writing the instructions	Short file written from memory	Generated from the code and history, one fixed rule format
Keeping them current	Grow and go stale	Word-capped, every reference verified
Learning from mistakes	Up to whoever fixes it, often a text note	Fix PR adds a rule and check; sweep proposes more and prunes
Misconfigured checks	Can pass silently	Fail CI

gap-trap

Thin instructions, followed some of the time

A rulebook agents can follow, enforced in CI

gap-trap writes the instructions, in a shape agents follow

Typical hand-written CLAUDE.md

Written by gap-trap

Four checks, with and without

Every miss becomes a new check

Without

With

Periodic sweep · /gap-trap refine reads git history since the last sweep

What still needs people

What gap-trap does on every pull request

When reviewing a PR, a person checks

Someone on the team owns

SDD alone vs. SDD with gap-trap

Periodic sweep · `/gap-trap refine` reads git history since the last sweep