Skip to content

feat(proof): add proof state, one read for counts, contests and blockers - #289

Draft
tonyketcham wants to merge 1 commit into
claude/agent-memory-context-potency-6sniud-real-harnessfrom
claude/agent-memory-context-potency-6sniud-state
Draft

tonyketcham wants to merge 1 commit into
claude/agent-memory-context-potency-6sniud-real-harnessfrom
claude/agent-memory-context-potency-6sniud-state

Conversation

@tonyketcham

Copy link
Copy Markdown
Collaborator

Summary of changes

This is layer 9 of a stack. It sits on #288.

proof find raised accuracy on the 10x graph (#286). But it still misses most questions about counts, open and closed sets, status history, blockers, and live records that rest on dead evidence, because no single record states those. proof state <effortId> prints them for one Effort. It is built only from frontmatter and links, never from body text.

What it shows:

  1. Counts: exact numbers by kind and standing. These are never cut.
  2. Open Issues and blockers: blockers first, each with the Decisions it gates.
  3. Risks and Constraints: Risks, and the Constraints in force.
  4. Contested topics: every question with more than one Decision, and every replacement chain. Each lists what rejected, replaced or disproved each version, and when.
  5. Closed without a fix: Issues resolved with no fix.
  6. Dead evidence: live records resting on replaced, disproved, rejected or retracted ones.
  7. Dated changes, and retractions.

Size:

  • Settled records cost only their counts.
  • The Markdown aims for 15,000 characters. On the 10x graph's Efforts it runs from 6,900 to 16,100.
  • Longer lists shrink in a fixed order. Each cut prints "… N more" and sets capped with cap_reasons: state_limit.

Other details:

  • --as-of is refused, with a pointer to proof find --as-of.
  • The read server serves it, and bin/flatbread.js forwards it.
  • The skill docs say when to use state and when to use find.

Benchmark:

  • state-recall.mjs is a free offline check of whether the answer appears in the state output.
  • A findstate strategy adds state to the find task text.
  • grade.mjs and compare.mjs take --split tune|holdout.

Offline result, reported plainly. On the 10x graph's held-out half, state holds the answer for 55% of count, status-history, blocker and dead-evidence questions under the strict scoring rule. That is below its own 60% bar, so state alone fails it. On those 31 held-out questions, find alone covers 15 and one of the two covers 24, so state adds to find. The paid Run D tests that: findstate against find, twice, on the 97 questions in those categories. Its results will come in a follow-up PR. The README records both scoring rules and how the strict one came about.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • I added doc comments to any new public exports, and inline comments to any hard-to-understand areas
  • My changes generate no new console errors locally
  • If applicable, try to include a test that fails without this PR but passes with it. state.test.ts covers each section on a small graph, and serve.test.ts checks that served state output matches local output. 253 AVA tests pass, along with typecheck, lint and check.mjs.

Does this introduce any non-backwards compatible changes?

  • Yes
  • No

Does this include any user config changes?

  • Yes
    • If so, I have updated the relevant areas of documentation
  • No

Generated by Claude Code

…ckers

`proof find` answers questions about what records say. It misses
counts, open and closed sets, status history, blockers and live
records that rest on dead evidence, because no single record states
those. `proof state <effortId>` prints them for one Effort, built only
from frontmatter and links:

- exact counts by kind and standing;
- open Issues and blockers, with what each gates;
- Risks, and the Constraints in force;
- contested topics: every question with more than one Decision, and
  every replacement chain, with what rejected or replaced each version
  and when;
- Issues closed without a fix;
- dead evidence: live records resting on replaced, disproved, rejected
  or retracted ones;
- dated changes, and retractions.

Settled records cost only their counts. The Markdown aims for 15,000
characters; longer lists shrink in a fixed order, and every cut prints
"N more" and sets `capped`. `--as-of` is refused with a pointer to
`proof find --as-of`. The read server serves it.

The benchmark gains `state-recall.mjs`, a free offline check of whether
the answer is in the state output, and a `findstate` strategy. On the
10x graph's held-out half the check finds the answer for 55% of count,
status-history, blocker and dead-evidence questions, against a 60% bar,
so `state` alone fails it. `find` alone covers 15 of those 31 and
`find` or `state` 24. The README records both rules and how the strict
one came about. `grade.mjs` and `compare.mjs` take `--split`.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants