feat(bench): add a free offline gate for proof find

[tonyketcham]

Summary of changes

Layer 3 of a stack. It sits on #282 (proof find), which sits on #281.

This adds a check that costs nothing to run. It asks whether proof find puts the right record, and the answer itself, in its top five results. It runs no model. The rule is simple: no paid benchmark run until this check passes.

1. scripts/find-recall.mjs runs every benchmark question through proof find in three ways:

   • the raw question, searched across all Efforts (--all);
   • the question with its question words stripped, across all Efforts;
   • the question scoped to its Effort (--effort).

   An Effort is Proof's top-level record for one piece of work. For each question the script records four things:

   • whether an evidence record shows in the top five;
   • whether the expected answer appears on the returned cards, matched as whole tokens. Questions whose answer would match by chance more than 10% of the time are set aside;
   • whether the right Effort comes first;
   • how many characters the reply holds.

   The check passes when the stripped set has evidence in the top five for at least 75% of questions and the answer on a card for at least 60%.

2. task.mjs adds a find strategy. Its prompt tells the agent to read with find, then get, and to use records only to count. The brief and nobrief prompts are byte-identical to before.

3. collect.mjs stores find_hash, a sha256 of find.ts, in each run file's header. compare.mjs warns when two runs used different find code. It can now also pool several run files, and it refuses the same file twice.

4. run.mjs records find_hash too.

5. The README records the first result. On the 10x graph the check fails:

   Measure (stripped set)	Result	Bar
   Evidence in top five	81% (200/246)	75%
   Answer on a card	56% (102/182)	60%
   Right Effort first	81%	—

   The weak categories are Blob questions (22% answer on card, though 100% evidence), counts (24%), lifecycle (46%), and retraction (40%). So no paid runs were made. The next step is to quote the linked Blob passage on a card, then rerun this check.

To run it: node benchmarks/proof-recall/scripts/find-recall.mjs, with --graph <dir> for the 10x graph.

Please don't delete this checklist! Before submitting the PR, please make sure you do the following:

• I added doc comments to any new public exports, and inline comments to any hard-to-understand areas
• My changes generate no new console errors locally
• If applicable, try to include a test that fails without this PR but passes with it (benchmark scripts only; check.mjs and the single-file compare.mjs output are unchanged)

Does this introduce any non-backwards compatible changes?

• Yes
• No

Does this include any user config changes?

• Yes
  • If so, I have updated the relevant areas of documentation
• No

---

Generated by Claude Code