Skip to content

feat(proof): mark a Decision suspect when the code it rests on moved - #301

Draft
tonyketcham wants to merge 1 commit into
claude/agent-memory-context-potency-6sniud-p4-buildfrom
claude/agent-memory-context-potency-6sniud-p5-suspect
Draft

tonyketcham wants to merge 1 commit into
claude/agent-memory-context-potency-6sniud-p4-buildfrom
claude/agent-memory-context-potency-6sniud-p5-suspect

Conversation

@tonyketcham

Copy link
Copy Markdown
Collaborator

Stacked on #300.

Why

In #300's feature-building run, agents followed an accepted Decision ("shortcuts are registered in keys/keymap.ts") that a refactor had made wrong. They did it even after seeing the anchor note "registerBinding renamed to legacyAction". An anchor note is too small to override a Decision.

What

A live Decision or Constraint is suspect when any of these holds:

  1. A tied code Citation is stale at HEAD. Its path or symbol is gone or renamed. A Citation counts as tied when it is linked to the record within 2 hops, or when the record's text names the Citation's symbol as code.
  2. A file it names is gone, or largely rewritten since the record. Largely rewritten means at least max(10, min(100, ¼ of its lines)) lines changed.
  3. It rests on dead evidence.

The warning appears before the body in get, find cards, brief, state and records, and lists the reasons:

SUSPECT — the code this Decision rests on changed after it was recorded: src/keys/keymap.ts: #registerBinding renamed to legacyAction in bc1c5b5 "Split key bindings by scope"; src/keys/keymap.ts changed in 4 commits since (43 of 60 lines): … Read the code first; where they disagree, the code is the current behaviour.

Other changes:

  • records gains --suspect.
  • The skill docs say what suspect means, and that the code wins.
  • The proof-build dump arm (inject.mjs) now asks the CLI for these warnings instead of reading record files directly.
  • Git work is batched and capped, like the other history reads.

Measured on the benchmark bed (v2)

Flags Precision Recall
Code reasons 3/3 3 of 8 drifted records
All, including dead evidence 3/5 —

The 5 misses are Decisions whose text names no code and that link to no Citation. Counting trailer-commit files instead catches all 8, but with 27 false positives, so it isn't shipped.

Checks

  • AVA: 307 passed, including a new proof-suspect.test.ts with 6 temp-repo tests. These cover each trigger, a small edit that doesn't flag, and the surfaces where the warning appears.
  • pnpm build, typecheck, prettier and skills:sync.

The paid comparison against #300's old reads comes next.


Generated by Claude Code

The feature-building benchmark showed agents following an accepted
Decision past a stale-anchor note. A live Decision or Constraint is now
suspect when a code Citation tied to it is stale at HEAD (path or symbol
gone or renamed), when a file it names is gone or largely rewritten since
it was recorded (at least max(10, min(100, a quarter of its lines))
changed), or when it rests on dead evidence. The warning comes before the
body in get, find cards, brief, state and records, with the reasons and
"read the code first; where they disagree, the code is the current
behaviour". records gains --suspect. The proof-build dump arm now asks
the CLI for the same warnings instead of reading record files directly.

On the benchmark bed it flags 3 of 8 drifted records with no false code
flags (the misses name no code and link to no Citation); git work is
batched and capped as the other history reads are.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants