docs(bench): record Run C, Sonnet, Run D and a held-out check on real graphs

.flatbread-proof/findings/fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj.md

@@ -0,0 +1,17 @@
+---
+id: fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj
+effort: eff-effort-graph-memory-and-agent-wedge--szeqvmgqjqnhd002
+title: 'A state read adds to search on counts, blockers and status history'
+kind: measurement
+created_at: '2026-10-19T12:41:23.560Z'
+derives_from:
+  - fnd-search-cards-raise-accuracy-and-cut-reading-on-t--ydqm2ybyqjedzw9y
+---
+
+`proof state <effortId>` prints what search cannot: exact counts, open blockers and what they gate, contested topics, Issues closed without a fix, dead evidence and dated changes. On the benchmark graph ten times larger, Haiku answered the 97 count, status-history, blocker and dead-evidence questions twice with `find` plus `state` and twice with `find` alone.
+
+- Accuracy rose by about 5 questions per run: 181 against 171 of 194 question-runs. Over both runs, only `find` plus `state` was right 15 times and only `find` 5 times (sign test p = 0.041).
+- It read 18 percent fewer tokens (15,578 against 18,907) and made 9 tool calls instead of 12.
+- On the half of the questions `state` was not shaped on, it led 83 to 79 of 88 (5 to 1). That points the same way but is too small to show on its own.
+
+Offline, `state` alone held the answer for 55 percent of these questions on that half, below its 60 percent bar. It earns its place by covering what `find` misses, not by answering alone. The plan's mark (at least 4 more per run, no rise in tokens) is met.

.flatbread-proof/findings/fnd-on-two-real-graphs-search-still-wins-and-the-sta--9qssh2p2w9tevfpw.md

@@ -0,0 +1,17 @@
+---
+id: fnd-on-two-real-graphs-search-still-wins-and-the-sta--9qssh2p2w9tevfpw
+effort: eff-effort-graph-memory-and-agent-wedge--szeqvmgqjqnhd002
+title: On two real graphs search still wins and the state read is neutral
+kind: measurement
+created_at: '2026-10-19T12:54:53.992Z'
+derives_from:
+  - fnd-a-state-read-adds-to-search-on-counts-blockers-a--hy9fx5hrw0gbcwzj
+  - fnd-search-cards-raise-accuracy-and-cut-reading-on-t--ydqm2ybyqjedzw9y
+---
+
+Two real Proof graphs from private projects (200 and 134 records) served as a held-out check that nothing was tuned on. An author blind to `find` and `state` output wrote 39 and 41 machine-checked questions; they stay outside the repository. Haiku answered each set twice with no-brief, `find`, and `find` plus `state`.
+
+- `find` beat no-brief on both graphs: 71.7 against 66.3 of 78, and 75 against 71 of 82. Over both, it alone was right 13 times and no-brief alone 3 times (sign test p = 0.021). It read about half the tokens.
+- Adding `state` changed nothing measurable: 1 to 2 and 3 to 4 against `find`, at the same tokens.
+
+So the search gain holds on real memory. The `state` gain seen on the 10x graph's counts and blockers does not appear on graphs a twenty-fifth that size or less, where paging records is cheap; `state` neither helps nor costs there.

.flatbread-proof/findings/fnd-search-keeps-its-gain-on-look-alike-filler-and-s--rtg5zjs9rndhqe5v.md

@@ -0,0 +1,16 @@
+---
+id: fnd-search-keeps-its-gain-on-look-alike-filler-and-s--rtg5zjs9rndhqe5v
+effort: eff-effort-graph-memory-and-agent-wedge--szeqvmgqjqnhd002
+title: Search keeps its gain on look-alike filler and saves reading on Sonnet
+kind: measurement
+created_at: '2026-10-19T12:33:23.668Z'
+derives_from:
+  - fnd-search-cards-raise-accuracy-and-cut-reading-on-t--ydqm2ybyqjedzw9y
+---
+
+Two more single runs of `proof find` against no-brief on the benchmark graph ten times larger, 245 comparable questions each.
+
+- **Look-alike filler (Haiku).** With 30 percent of filler records reusing real subsystem names, `find` got 233.7 right and no-brief 214.7: a gap of 19, the same as on the plain graph (19 to 1 on the questions only one got right, p < 0.0001). `find` read 0.38 of no-brief's tokens. So the earlier gain was not an artifact of filler that is easy to skip.
+- **Sonnet.** `find` got 233.4 and no-brief 226.7, a gap of 6.8 that one run cannot separate from noise (11 to 5, p = 0.21). `find` still read 44 percent fewer tokens (9,280 against 16,674) and made 7 tool calls instead of 18. On 24 questions that name no Effort, `find` got 24 and no-brief 17.
+
+A stronger model crawls the graph well enough without search, so it gains less accuracy and keeps most of the token saving. Search helps every model most when the question does not say which Effort it belongs to.

benchmarks/proof-recall/README.md

@@ -872,6 +872,52 @@ What this does not show:
   finds the right Effort, not that agents handle a hard question without a
   named Effort.
 
+### Look-alike filler, and Sonnet
+
+Two more single runs of `find` against no-brief, with the same task texts
+and `find_hash` as above, on 270 questions each (the 246 and the 24
+ambiguous-Effort ones). Figures are for the 245 comparable questions.
+
+**Run C: Haiku on the look-alike graph** (`--scale 10 --filler-overlap 0.3`,
+where 30% of filler records reuse real subsystem names). Run files:
+`runs/x10adv-find-r1-*.json`.
+
+| Measure                           | `find` | no-brief |
+| --------------------------------- | ------ | -------- |
+| Correct                           | 233.7  | 214.7    |
+| Questions only this arm got right | 19     | 1        |
+| Mean read tokens                  | 13,835 | 36,891   |
+| Mean tool calls                   | 9      | 22       |
+| "Unknown" answers                 | 18     | 33       |
+
+The gap is 19 questions (p < 0.0001), the same as on the plain graph
+(about 19 per run), and `find` read 0.38 of no-brief's tokens. The plan's
+pass mark for Run C was half the Run A gain and at most 0.75 of the
+tokens, so it passes easily: filler that looks real did not hurt search.
+On the ambiguous set `find` got 22 of 24 and no-brief 15 (7 to 0,
+p = 0.016).
+
+**Sonnet on the plain 10x graph.** Run files: `runs/x10-sonnet-r1-*.json`.
+
+| Measure                           | `find` | no-brief |
+| --------------------------------- | ------ | -------- |
+| Correct                           | 233.4  | 226.7    |
+| Questions only this arm got right | 11     | 5        |
+| Mean read tokens                  | 9,280  | 16,674   |
+| Mean tool calls                   | 7      | 18       |
+| "Unknown" answers                 | 16     | 27       |
+
+Sonnet without `find` does far better than Haiku without it (226.7
+against about 211), so the accuracy gap shrinks to 6.8 questions and is
+not significant in one run (p = 0.21). `find` still read 44% fewer tokens
+and made fewer than half the calls. On the ambiguous set, where the
+question names no Effort, the gap stays large: `find` 24 of 24, no-brief
+17 (7 to 0, p = 0.016).
+
+So a stronger model gains less accuracy from search and keeps most of the
+saving; a weaker one gains both. Timeouts (10 minutes) hit 3 Haiku and 5
+Sonnet sessions, all but one in the no-brief arm.
+
 ### The offline gate for `proof state`
 
 `proof state <effortId>` is one read, built from frontmatter and links,
@@ -965,5 +1011,61 @@ The find gate, rerun with the same `find_hash` (`e5be6532…`), still gives
 81% evidence in the top 5 and 62% answers on a card on the stripped set.
 
 The paid Run D compares `findstate` against `find` on these questions.
-Before paying for it, decide whether a read that fails its own bar is
-worth a paid run.
+It was run anyway, because `find` and `state` together cover more than
+either alone; see below.
+
+### `find` plus `state` against `find` (Run D)
+
+Haiku on the 10x graph, twice, on the 97 questions in the aggregate,
+dead-evidence, lifecycle and blocker categories (all but `world-077`):
+`run.mjs --strategies findstate,find --ids <those 97>`. The `findstate`
+arm adds one line to the `find` task text: use `proof state <effortId>` for
+counts, open and closed sets, status history, blockers and dead evidence.
+Run files: `runs/x10-rund-r{1,2}-*.json`.
+
+Pooled over both runs (194 question-runs per arm):
+
+| Measure                              | `findstate` | `find` |
+| ------------------------------------ | ----------- | ------ |
+| Correct                              | 181         | 171    |
+| Runs of a question only this arm got | 15          | 5      |
+| Mean read tokens                     | 15,578      | 18,907 |
+| Mean tool calls                      | 9           | 12     |
+
+The pooled sign test gives p = 0.041; counting each question once, 9 to 4
+(p = 0.27). The plan kept `state` only if it gained at least 4 questions
+per run on this subset with no rise in tokens. It gained 5 per run and read
+18% fewer tokens, so it passes.
+
+`state` was shaped on the tune questions, so the holdout half is the clean
+measure: there `findstate` got 83 and `find` 79 of 88 (5 to 1, p = 0.22),
+against 98 and 92 of 106 on tune (10 to 4). The direction holds on
+holdout, but two runs on 44 questions cannot show it alone. Two sessions
+timed out, both in the `find` arm.
+
+### Two private project graphs, as a held-out check
+
+Two real Proof graphs from private projects (200 records across 11 Efforts,
+and 134 across 6) tested whether the gains above hold on memory nobody tuned
+against. An author who never saw `find` or `state` output wrote 39 and 41
+machine-checked questions about them. The questions, answers, graph copies
+and run files live outside this repository (see "Question sets kept outside
+the repository"); only these totals are published. Before any paid run,
+only category totals of the offline gates were read, never a card.
+
+Haiku, two runs per graph, three arms each, pooled:
+
+| Graph | `find`     | no-brief | `findstate` | Read tokens, `find` / no-brief |
+| ----- | ---------- | -------- | ----------- | ------------------------------ |
+| A     | 71.7 of 78 | 66.3     | 71.3        | 12,488 / 27,975                |
+| B     | 75 of 82   | 71       | 74          | 12,230 / 24,389                |
+
+- `find` against no-brief: 7 to 1 on A and 6 to 2 on B; 13 to 3 over both
+  (sign test p = 0.021). `find` read about half the tokens on both.
+- `findstate` against `find`: 1 to 2 on A and 3 to 4 on B, with the same
+  tokens. On these small graphs `state` neither helps nor costs.
+
+So the search gain carries over to real memory at about half the reading.
+The gain from `state` on the 10x graph's counts and blockers does not show
+on graphs a twenty-fifth
+that size or less, where paging `records` is cheap.