Skip to content

Add Proof compaction records, proof brief, and a recall benchmark - #281

Draft
tonyketcham wants to merge 27 commits into
mainfrom
claude/agent-memory-context-potency-6sniud
Draft

Add Proof compaction records, proof brief, and a recall benchmark#281
tonyketcham wants to merge 27 commits into
mainfrom
claude/agent-memory-context-potency-6sniud

Conversation

@tonyketcham

@tonyketcham tonyketcham commented Oct 18, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary of changes

This branch does five things.

  1. Compaction records and proof brief. Proof is Flatbread's Git-native agent memory. A compaction is a Finding with kind: compaction that sums up an Effort (a unit of work) without hiding its history.
    • The new WriteCompaction mutation stores a covers list of <id>@<standing> entries. Standing is a record's state when the compaction was written, such as open, accepted, or superseded.
    • It supersedes every current head compaction, so each compaction replaces the old one rather than editing it.
    • derives_from holds at most 10 ids and may not name another compaction.
    • flatbread proof brief <effortId> prints the head compaction, older ones as lineage, the records that changed since, and any derivations that now rest on dead records. It suggests a new compaction after 5 changes.
    • Guards stop WriteFinding, Supersede, and Invalidate from making or targeting compactions by the back door.
  2. Retracted records can be listed, always marked. proof records <effortId> --retracted include|only lists them; the default still hides them. Every digest marks a retracted record with (retracted) on its index line, a RETRACTED banner with date and reason, and a retracted N count in the summary.
  3. flatbread proof serve keeps the graph loaded and serves reads only. With FLATBREAD_PROOF_SERVER set to its socket, the six read commands go to it before the CLI bundle loads; writes, and any read when the socket is unreachable, run locally. A write bumps the journal generation, so the next served read rebuilds. On a 4,980-record graph a read drops from 6 to 9 seconds to 0.2 to 0.4 seconds. A second server refuses to take a live server's socket.
  4. GitHub stacked pull requests replace Mergify stacks. The Mergify stack hooks and skill are gone. A gh-stack skill and a section in AGENTS.md explain gh stack. Mergify's merge queue stays.
  5. A Proof recall benchmark in benchmarks/proof-recall/: a fictional game studio's graph of 498 records across five Efforts, built by replaying specs through the real Proof writer, with 246 machine-checked questions from simple lookups to four-hop chains.
    • scripts/run.mjs gives each question its own fresh, headless claude -p session per strategy, with only Bash, Read, and Write. Runs share one read-only graph copy per build and one read server. scripts/compare.mjs pairs two runs with a sign test. build.mjs --scale N grows the graph with inert filler that changes no answer but one.
    • All 246 questions on Haiku: 221.4 correct with brief, 226.3 without (p = 0.24). brief read 23% fewer tokens and 70% fewer on compaction questions.
    • The same on a graph 10 times larger: both strategies lost about ten points (197.4 and 198.7 of 237), mostly by giving up. brief gained nothing measurable (26 questions better, 22 worse, p = 0.67) and still read 28% less. So graph size does not explain the missing accuracy gain; the next test is a text search read.
    • Three smaller pilots (Opus, Sonnet, Haiku) shaped the protocol. The runs exposed and fixed six flawed questions, one wrong graph fact, two grader rules, and a gap in check.mjs's scope warning.

The branch also records the compaction Decision, the retracted-listing Decision, Issues for gaps found along the way, a new compaction, and a measurement Finding per run in the repo's own .flatbread-proof/.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

  • I added doc comments to any new public exports, and inline comments to any hard-to-understand areas
  • My changes generate no new console errors locally
  • If applicable, try to include a test that fails without this PR but passes with it (compaction.test.ts, planner and schema tests, end-to-end proof brief and retracted-listing tests, and serve.test.ts for the read server)

Does this introduce any non-backwards compatible changes?

  • Yes
  • No

WriteFinding now rejects kind: compaction; use WriteCompaction instead.

Does this include any user config changes?

  • Yes
    • If so, I have updated the relevant areas of documentation
  • No

The Mergify stack git hooks are removed from .husky/. Contributors who used mergify stack should install gh stack instead (see AGENTS.md). FLATBREAD_PROOF_SERVER is a new, optional environment variable.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW

claude added 26 commits October 17, 2026 09:08
@claude
feat(proof): add compaction records and the brief read
76e8d92 
An agent that resumes an Effort must read every live record to learn
where the work stands. This adds a short, current account of an Effort
that never hides the records behind it.

- WriteCompaction (the 18th mutation) creates a Finding with
  kind: compaction. The writer stores `covers`, a list of which records
  were live and where each stood, and supersedes every earlier head
  compaction. `derives_from` holds at most 10 ids and may not name
  another compaction. No other mutation can create, supersede, or chain
  a compaction.
- `flatbread proof brief <effortId>` (the 6th read) prints the head
  compaction in full, older compactions as one-line checkpoints, and
  each record whose standing changed since `covers` was taken. The
  envelope's `compaction` field says whether the head is stale and
  whether to write a new one. Reads never print `covers`.
- The skill, reference, glossary, and README explain when to compact,
  when to supersede, and when a hand edit is enough.

Proof records: Decision dec-compact-an-effort-with-writer-built-
superseding (accepted), the first compaction of the agent-memory
Effort, a measurement Finding (brief is 84 percent smaller than the
status-briefing read on that Effort), and a new version of the
mutation-count Constraint that names 18 mutations.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
Change-Id: I6d8587f3d01cd6ad96278300a9e2a651c107ee9d
@claude
chore: replace Mergify stacks with GitHub stacked pull requests
c05bc16 
Stacked pull requests now go through GitHub's own `gh stack` extension
instead of `mergify stack`.

- Remove the Mergify stack Git hooks. They added a `Change-Id` trailer
  to every commit and steered `git push` toward `mergify stack push`.
  The lint `pre-commit` hook stays.
- Replace the vendored `mergify-stack` agent skill with GitHub's
  `gh-stack` skill, installed with the `skills` CLI and recorded in
  `skills-lock.json`.
- Rewrite the AGENTS.md section to explain the `gh stack` workflow and
  the flags agents must pass so commands never wait on a prompt.

Mergify's merge queue (`.mergify.yml`) and its `mergify-*` skills stay.
`gh stack merge` only knows GitHub's own merge queue, so AGENTS.md says
to keep merging through Mergify.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
proof: record three gaps in agent memory and recompact
e972577 
Record what the review of agent-memory research found worth acting on, as
open Issues in the agent-memory Effort:

- Proof has no recall benchmark, so no change to reads can show that
  answers stay correct. This blocks the compaction Decision's reversal
  test.
- Records are not tied to the code they govern, so a Decision can describe
  code that has since changed. The Issue sketches Git-hash anchors and a
  CI check.
- Memory reaches an agent only when the agent asks for it.

Four Citations hold the outside sources. Two are papers read only from
search summaries; their Citation bodies say so.

The new records pushed `brief` past its threshold of 5 changes, so a
fresh compaction supersedes the first one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): add the Proof recall benchmark harness
37212c5 
Add benchmarks/proof-recall, the start of the recall benchmark the open
Proof Issue asks for. It measures how well an agent answers questions
from a Proof graph, and at what cost in tokens and tool calls.

- build.mjs replays each Effort's spec, an ordered list of mutations,
  through the real Proof writer, with fixed ids and timestamps, so the
  graph is valid and identical on every build.
- check.mjs proves each question's answer and path against the built
  graph and prints coverage by tier, hops, and category.
- grade.mjs scores a run; prepare-run.mjs makes a clean folder that
  holds the graph and question text but no answers.
- README.md sets the authoring contract; WORLD.md sets the fictional
  game every Effort describes.

The built graph is kept out of Prettier, because it must match the
writer's output byte for byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): add the tools and netcode Efforts and shortcut warnings
5b2ba0d 
Add the two small benchmark Efforts: tools (34 records, 16 questions,
ends completed) and netcode (32 records, 15 questions, ends
abandoned). They supply distractors and test Efforts that default
reads no longer list.

check.mjs now finds the shortest route a reader can follow between a
question's entry record and its answer, using only links stored on the
record in hand. It warns when that route is shorter than the stated
path, since the question is then easier than its tier claims. Three
questions had such shortcuts and are re-tiered to their real routes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): add the ports Effort
227950a 
Add the medium benchmark Effort for the Meridian and Skiff ports: 88
records and 34 questions across all five tiers and 18 categories.

It holds the deliberate distractors that other Efforts reference by
title: the "Skiff memory budget" and "Frame budget" Constraints, the
on-device pipeline cache that prose also calls a shader cache, and the
Decision to run Tern on Skiff at a lower preset. A retracted "Frame
budget" record carries the render Effort's 8.3 ms figure as a trap.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): add the world Effort
6c0e008 
Add the first complex benchmark Effort, Kestrel's world streaming and
simulation core: 165 records and 81 questions.

It holds a six-version job-system chain that moves from a thread pool
to fibers and back, three versions of the streaming pool budget, a
three-version tide drift measurement, 12 invalidated records, and 7
retracted look-alikes. Its head compaction is not stale, but 7 records
changed after it, and two of its claims are contradicted by records it
does not cite.

Two facts now agree with the ports Effort. The streaming pool cut in
June cites the Skiff memory cut that ports made in February, instead
of reporting it as new. Ports now gives Skiff internal storage as about
200 MB/s, so world's 190 MB/s retail measurement sits under it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): add the render Effort and the built graph
945d2b0 
Add the second complex benchmark Effort, the Lantern renderer overhaul:
174 records and 83 questions. It holds a six-version visibility buffer
chain, three versions of a "Frame budget" Constraint plus a retracted
look-alike with the Skiff figure, 9 invalidated records, 3 reopened
Decisions, and 13 Blobs of capture data. Its head compaction is stale:
a record it cites changed standing after it was written.

Commit the built graph in .flatbread-proof/ so a run needs no build. A
rebuild from spec/ is byte-identical. The full benchmark now holds 498
records and 229 questions across five Efforts, and check.mjs proves
every answer and path against the graph.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): add cross-Effort questions
aac41f6 
Add 17 questions that need two or more Efforts. Efforts link only by
title in prose, so each question joins records that no edge connects:
the two live "Frame budget" Constraints, the shader caches that prose
names alike, the tide Decision behind the netcode abandonment, and the
Skiff memory cut behind the streaming pool cut.

Checks gain two features for this. A `select` can now scan a list of
Efforts, or all of them, and a new `efforts` check counts Efforts by
status. Run folders now tell the answering agent the form each answer
takes, without its content.

The streaming pool Constraint now dates the Skiff memory cut to
February, matching the ports Effort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
fix(bench): let scripts be imported from inline code
f18a66f 
The run-as-script guard read process.argv[1], which is undefined under
`node -e`, so importing check.mjs from inline code crashed. Check that
it exists first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
fix(bench): grade short answers that add detail
269adbd 
The first pilot showed agents answering "46 player-proximity reads"
for "46" and "textures (3,184 MB)" for "textures". The grader marked
these wrong. It now accepts a value answer that equals the gold value,
starts with it, or carries the same leading number and unit. It also
reads "true" and "false" given as strings, and counts given with a
noun after them. Authoring checks stay exact.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): record the first pilot and fix the run protocol
61c5056 
Two fresh agents answered the same stratified sample of 46 questions,
one session each. Starting from `proof brief`, one scored 46 of 46
(46 tool calls, 203,004 tokens). Without `brief`, the other scored 45
of 46 (27 tool calls, 239,556 tokens). Audits of both agents' tool
calls found no access to answers.

Both read most of the graph early and answered from it, so the
benchmark hit its ceiling and the tiers stopped mattering. The README
now asks for one session per question, one graph command per tool
call, and token and tool-call counts from the harness, since the
agents' own tallies were 2.4 and 4.5 times too high.

The runs are saved under runs/. A Proof Finding records the result
against the open recall-benchmark Issue, which stays open until a run
shows whether answers differ between read strategies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): collect run cost from agent transcripts
eb7b0c2 
Add collect.mjs, which builds a run file from one-question-per-session
runs. It reads each session's answer from its folder and its cost from
the agent's transcript: tool calls, turns, tokens that entered the
context, and tokens processed including cache reads. The pilot showed
agents' own tallies cannot be trusted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
fix(bench): make every question name its Effort
21c4cea 
Ten questions said "this Effort". That read fine inside a question
file grouped by Effort, but a run shows each question alone, so two
answering agents could not tell which Effort was meant. Each question
now names its Effort by title, and check.mjs rejects "this Effort".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): run pilot 2 with one Sonnet session per question
d6f8894 
Ask the pilot's 46 questions again, one fresh Sonnet session per
question and per strategy: 92 sessions, with cost taken from each
session's transcript.

Five questions did not name their Effort, so the agents could not tell
which Effort they meant; both strategies missed all five. Sixteen
questions now name their Effort, and check.mjs warns when an
Effort-wide check would also find answers in another Effort. A
question whose wording names its Effort another way carries
"scoped": true. One rubric now also accepts the tide Decision's id.

On the other 41 questions, starting from `proof brief` scored 38 and
going without it scored 39. The extra miss gave the right record's id
where the question asked for its title. `brief` cut read tokens by
17.5 percent on average (15,365 against 18,624), but the median
question saved about 1,000 tokens, and `brief` was cheaper on only 22
of 41.

A Proof Finding records this against the compaction Decision. That
Decision says to revisit it if agents answer no better from `brief`,
and on this evidence they do not.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): run pilot 3 with one Haiku session per question
089056a 
Repeat pilot 2 on Haiku: 46 questions, one fresh session per question
and per strategy. On the 44 sound questions, the brief arm answered all
44 and the no-brief arm 41. Brief also cut graph reads by 26 percent on
average (median saving about 4,000 tokens).

Fix three benchmark flaws the run exposed:

- render-040 now names its Effort. Unscoped, the ports compaction was a
  fair answer.
- cross-017 now asks about the record titled exactly as quoted, since
  browse reads hide the retracted duplicate.
- The Skiff beta retrospective no longer puts its 6-week cost on a span
  the record dates make about 18 weeks.

Record the result as a Proof Finding against the compaction Decision.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
fix(bench): warn when a single-record check also answers elsewhere
97bbe3f 
The scope warning in check.mjs only re-ran Effort-wide checks in other
Efforts. Questions whose `asof`, `head`, or `chain` check starts from one
record slipped past it, as render-040 did in pilot 3.

For those checks, look in every other Effort for records a reader could
take for the start record: the same kind and title, or another
compaction. If the check answers from any of them, the question must name
its Effort. The new rule flagged four more questions, which now do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): run sessions headless with only Bash, Read, and Write
e845626 
Benchmark agents launched as subagents could reach session-control
tools, and Haiku agents called them in about half of pilot 3's sessions:
messages to sessions that do not exist, one empty cloud session, and one
scheduled reminder.

scripts/run.mjs now starts one `claude -p` session per question and per
strategy. Each session gets only Bash, Read, and Write, no MCP servers,
no settings files, its own session id, and none of the environment
variables that tie a process to the session that launched it. The runner
saves each transcript beside its answer, resumes an interrupted run, and
writes both run files.

Also:

- scripts/task.mjs holds the task text each session reads.
- scripts/compare.mjs pairs two runs: scores, questions only one got
  right, a sign test, and read tokens.
- collect.mjs records each session's first turn as `fixed_tokens`, so
  read tokens no longer rest on an estimate.
- prepare-run.mjs, collect.mjs, and grade.mjs can now be imported.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
fix(bench): grade a boolean answer by its leading word
07f35b7 
Haiku agents often add evidence after the verdict, as in "false,
fnd-...". The grader took only an exact "true" or "false" and scored
those answers wrong. It now reads the leading word. No earlier pilot's
score changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): run all 246 questions on Haiku
0663f49 
Run every question once with `proof brief` and once without, one
headless Haiku session each: 492 sessions, all of which answered.

- Accuracy: 221.4 correct with brief, 226.3 without. Twelve questions
  were right only without brief and six only with it (sign test
  p = 0.24). Pilot 3's 3-to-0 edge for brief came from its sample.
- Reading: brief cut read tokens by 23 percent on average and 5,300 on
  the median question, and was cheaper on 157 of 246. On compaction
  questions it read 70 percent less.
- Eight questions ask about retracted records, which no read command
  can list. Every run answered "unknown" to all of them.

The runner now alternates strategies question by question, so a change
in API speed over a run falls on both alike.

A new Proof Finding supersedes pilot 3's, whose accuracy claim the full
run reverses.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(proof): list retracted records on request, always marked
481723b 
Browse reads hid retracted records with no way to ask for them, and
Retract strips a record's id from every other record, so nothing could
find one without already knowing its id. In the recall benchmark, every
run answered "unknown" to all eight questions about retracted records.

`proof records <effortId>` takes `--retracted exclude|include|only`. The
default, exclude, is unchanged and keeps existing query hashes. Every
digest marks a retracted record so it cannot pass for a live one:

- its index line ends in "(retracted)";
- a RETRACTED banner with the date and reason sits under its heading,
  in browse reads and in `get`;
- the envelope summary counts "retracted N".

The benchmark task text now mentions the flag. A rerun of the eight
retraction questions on Haiku scored 7 of 8 in each strategy, up from
0 of 8. The decision is recorded and accepted in Proof.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): build the graph at any scale with inert filler
4bc98fe 
To test whether `brief` helps once an Effort is too large to read whole,
the benchmark needs a bigger graph whose answers do not change.

- `build.mjs --scale N --out <dir>` merges filler into each Effort until it
  holds about N times its own records. Filler (spec/_filler.mjs) is
  seeded, so builds repeat exactly. It is routine measurements, captures,
  and small question-to-decision threads about subsystems no spec or
  question names. It never links to a real record and is all written
  before the Effort's head compaction, which therefore covers it. The
  build refuses filler whose subsystem appears in real content.
- Each Effort now builds in its own folder and then moves into place. The
  writer reads every record under its root on each mutation, so a shared
  root made later Efforts pay for earlier ones. The unscaled build is
  byte-identical to the committed graph.
- `run.mjs --graph <dir>` runs against another build. Sessions now link to
  one shared public copy of the graph instead of copying it, and the run
  warns if that copy changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(proof): serve reads from memory with flatbread proof serve
e63c87f 
Every Proof read built the whole graph and exited. On the benchmark's 10x
graph (4,980 records) that cost 6 to 9 seconds per read.

`flatbread proof serve [--socket path]` keeps the graph loaded in one
process and answers the six read commands over HTTP on a Unix socket. It
serves nothing that writes. With FLATBREAD_PROOF_SERVER set to the socket,
`bin/flatbread.js` forwards those reads before loading the CLI bundle and
prints exactly what a local read prints; anything else, writes included,
runs locally, as does any read when the socket is unreachable.

The in-memory graph is keyed by the graph root's real path and its journal
generation, so folders that link to one graph share it, and a write makes
the next served read rebuild. Digests are still written under the caller's
folder. On the 10x graph a later `records` read takes 0.4 s and a `get`
0.17 s.

The benchmark runner starts one server per run (`--no-serve` turns it
off) and points each session's `./flatbread` at it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): share one read server and graph copy across runs
75fb84a 
Each run started its own `proof serve` and made its own graph copy, so
the 6-second graph load and 500 MB of memory came back with every run.

run.mjs now keeps both under <tmp>/flatbread-proof-recall/:

- graphs/<hash>/ holds one read-only public copy of each built graph,
  named by a hash of its contents. A copy whose files no longer match its
  name is made again, and the post-run tamper check compares against the
  same hash.
- proof.sock is one server for every run. A run reuses it if it answers,
  and otherwise starts it detached and leaves it running. A lock folder
  makes runs that start together start one server.
  `run.mjs --stop-server` stops it.

`proof serve` now refuses to start on a socket a live server answers on.
Before, it deleted the socket file, which left the old server running
but unreachable.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
feat(bench): run every question on a graph ten times larger
5e3866b 
The leading theory for why `proof brief` did not raise accuracy was that
each Effort was small enough to read whole. This run tests it on a graph
built with `--scale 10` (498 to 4,980 records), asking all 246 questions
on Haiku in both strategies through one read server.

On the 237 questions comparable with the 1x run:

- Accuracy fell about ten points in both: 220.4 to 197.4 with brief,
  225.3 to 198.7 without. "Unknown" answers nearly doubled; agents named
  a filler record once in 474 sessions.
- Brief gained nothing measurable: per question it held up better on 26
  and worse on 22 (sign test p = 0.67).
- Agents read about the same amount at both sizes, so more noise meant
  more misses. Brief still read 28 percent less.

So graph size does not explain the missing gain; the limit is finding a
record by its text. The README now proposes testing a search read next.
A Proof Finding records the result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@claude
fix(proof): rebuild a served graph when its record files change
5968731 
`proof serve` keyed its in-memory graph on the journal generation. That
counter lives in a gitignored folder and only `proof write` bumps it, so
after a pull, merge, or checkout the server kept serving the old graph
until it restarted.

The key now also holds a fingerprint of the record files: each file's
path, size, and modification time. Statting a few thousand files takes
milliseconds, far less than rebuilding the graph. A new test changes a
record file directly, as a pull would, and checks that the next served
read sees it; it fails with the old key.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
This was referenced Oct 19, 2026
Draft
Draft
Draft
Draft
@claude
Merge main into the compaction branch
8e03d30 
main now allows `derives_from` links across Efforts (#280). Six files
conflicted:

- `digest.ts`: the index keeps main's filter for foreign records and
  this branch's `(retracted)` marker.
- `SKILL.md` and `glossary.md`: the Retract text says both that the
  writer strips links across every Effort and that `--retracted`
  lists retracted records.
- The Constraint "Mutation enum stays deliberately small" had two new
  versions, one from each side. The chain is now one line: main's
  version replaces the original, and this branch's version, which adds
  `WriteCompaction`, replaces main's and takes its new Retract wording.

`WriteCompaction` now calls `assertDerivesFrom` without an Effort,
matching main's rule that `derives_from` may cross Efforts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DVVnhYL1Bz7mCfEKDoyLMW
@tonyketcham tonyketcham mentioned this pull request Oct 19, 2026
Draft
4 of 8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@tonyketcham @claude