diff --git a/.agents/skills/proof/SKILL.md b/.agents/skills/proof/SKILL.md index 65b8e104..a3381bdb 100644 --- a/.agents/skills/proof/SKILL.md +++ b/.agents/skills/proof/SKILL.md @@ -175,13 +175,14 @@ Pick the read by what you need, in this order: records by their title, full body, and Blob payload, and prints the hit cards inline in the JSON, so you need no second Read. Each card shows the newest version of the record (or, with `--as-of`, the version current - then), its standing, the passage that matched, its supersession chain - with dates, and its links in both directions. Leave out `--effort` when - you do not know the Effort: hits then group by Effort, at most 3 each - unless one Effort clearly leads. The `missing` terms name query words no - hit contains. Ordinary words often land there; a missing distinctive - term, such as a name, a number, or a code reference, is the cue that the - answer may not be recorded. + then), its standing, the passages that matched (up to two, plus one + from an attached Blob payload such as a capture or table), its + supersession chain with dates, and its links in both directions. Leave + out `--effort` when you do not know the Effort: hits then group by + Effort, at most 3 each unless one Effort clearly leads. The `missing` + terms name query words no hit contains. Ordinary words often land there; + a missing distinctive term, such as a name, a number, or a code + reference, is the cue that the answer may not be recorded. 2. **Read one record in full: `proof get `.** 3. **Count or page through records: `proof records`.** `find` shows only the top hits, so never count from it. diff --git a/.agents/skills/proof/reference.md b/.agents/skills/proof/reference.md index b484f68c..b8719a47 100644 --- a/.agents/skills/proof/reference.md +++ b/.agents/skills/proof/reference.md @@ -348,10 +348,31 @@ flatbread proof serve [--socket .flatbread/proof/serve.sock] Effort. Unless the top Effort's sum is at least 1.5 times the next one's, at most 3 hits come from one Effort at first; slots still empty then go to the next hits by score. +- **Passages:** a card quotes up to two passages of the matched version's + body, each about 160 characters. The first is the best: the window whose + query words weigh most, counting rare words more (by the same inverse + document frequency that ranks hits) and counting words the card's title + already shows only to break ties. The matches sit in the middle of the + window, and the passage runs a little past 160 characters to finish a + short word right after the last match, so a value after a matched label + (`"fuse_ms": 42`) is not cut off. A second passage appears only when + another part of the body holds a query word the first passage and the + title lack; it may come from earlier in the body, as its offsets show. + When the shown version carries a Blob (a Citation's `blob`), or cites a + Citation that does, and that payload holds a query word, the card also + quotes the payload's best passage on a `blob :` line. A Blob that is a + hit itself, or that an earlier card already quotes, is not quoted again. + The Citation and the Blob must be current: not superseded, not written + after `--as-of`, and not retracted unless `--retracted include` or `only` + lets retracted records in; then the line reads `blob (retracted):`. + Query words found only in a quoted Blob count as matched, not missing. - **Caps:** `--limit` 1 to 10, default 5. Each card holds at most 4 chain versions (the head and the shown and matched versions always appear), 3 - links per relation, and 6 code references. When five cards run past about - 4,000 characters, the cards shrink in this order: link titles, the + links per relation, and 6 code references. The cards aim for 300 + characters plus 600 per card, plus the length of every passage they + quote: about 4,000 characters for five cards with one passage each. + Passages are never cut, so quoting more never pushes out a chain or a + link. Past the aim, the cards shrink in this order: link titles, the `efforts by match` line, the middle of each chain, then link entries. A relation with no entry left shows its count, as `cited_by +4`. When more records matched than are shown, the envelope has `complete: false` and @@ -364,10 +385,12 @@ Each card has these lines: 1. | [:] | [(head)] | | score "" matched: v<n> <id> (only when an older version matched) - "…<160-character passage with the most query words>…" [chars <from>-<to> of <length>] + "…<best passage of the matched version>…" [chars <from>-<to> of <length>] + "…<second passage, when the body matches in a second place>…" [chars <from>-<to> of <length>] + blob <id> [(retracted)]: "…<passage of an attached Blob payload>…" [chars <from>-<to> of <length>] chain: v1 <id> <standing> <date> -> (n more) -> v4 <id> <standing> <date> (head) out: <relation> <id> "<title, 60 chars>" (<standing>), … +n more; <relation> +n - in: <derived_by|cited_by|resolves|rejects|mitigates|evidence_for|blob_of> <id> "<title>" (<standing>) + in: <resolves|rejects|mitigates|evidence_for|cited_by|blob_of|derived_by> <id> "<title>" (<standing>) code: <paths, symbol() names, SHAs, #numbers the chain's records name> ``` diff --git a/benchmarks/proof-recall/README.md b/benchmarks/proof-recall/README.md index 7f719e34..f784bea2 100644 --- a/benchmarks/proof-recall/README.md +++ b/benchmarks/proof-recall/README.md @@ -278,7 +278,9 @@ A benchmark agent must never see `spec/`, `questions/`, or `keymap.json`. the `find` source. The gate passes when, on the stripped set, the evidence is in the top 5 for at least 75% of questions and the answer is on a card for at least 60% of those it applies to; if it fails, do not - pay for sessions. + pay for sessions. `--split tune` or `--split holdout` asks only half the + questions, chosen by a hash of each id. Read tune misses to decide what + to change in `find`; run holdout once the change is done. 3. Run the benchmark: @@ -611,3 +613,94 @@ Next steps, in order: 2. Decide whether counting and lifecycle questions need their own read, such as the `proof state` view the design review proposed, instead of search. + +#### Cards that quote more + +`find-recall.mjs --split tune|holdout|all` now keeps half the questions +back. A question is in `tune` when the first byte of the SHA-256 of its id +is even, and in `holdout` when it is odd: 130 and 116 questions. Only tune +misses were read to decide what to change in this step; holdout was scored +once the code was done. Holdout is unseen only question by question, though: +the plan to quote Blob payloads came from the diagnosis of all 246 +questions in the step before, made before the split existed. + +On the 10x graph, 43 tune questions missed the answer on the cards. Why: + +| Where the answer was | Misses | +| ----------------------------------------------------------- | ------ | +| Not a hit, and not one link from a hit | 20 | +| In a hit's body, outside the passage or cut off at its edge | 3 | +| In a Blob that is a hit, outside its passage | 3 | +| In a Blob attached to a hit, which the card did not quote | 2 | +| A record linked from a hit, cut by the per-line link cap | 3 | +| A chain version the card left out as `(n more)` | 3 | +| A retracted record, which `find` leaves out by default | 3 | +| In a linked record's text or author | 2 | +| Not written anywhere; it must be worked out | 2 | +| A hit's author, which cards do not show | 1 | +| On the card, but as a bare number the gate does not count | 1 | + +The 1x graph gave 42 misses with the same causes. The cards changed in +four ways: + +1. The best passage is the window whose query words weigh most, rare words + more, and words the title already shows count only to break ties. It + comes first on the card. +2. The matches sit in the middle of the window, and the passage runs on to + finish a short word right after the last match, so a value after its + label stays. +3. A second passage appears when another part of the body holds a query + word the first lacks, and a card quotes the best passage of a Blob that + the hit carries or cites through a Citation. Only a current, unretracted + Citation and Blob count, and a Blob quoted on one card is not quoted + again on a later one. Passages add their length to the size aim, so they + never push out a chain or a link. +4. Reverse links list `resolves`, `rejects`, and the like before + `derived_by`, which on a much-used record fills the line. + +Showing each card's author was tried and dropped: it pushed chains off the +cards and put a name on every card, so name answers matched by chance. + +The chance rates now come from every question's cards, even with +`--split`, so each split's figures below are its part of the `all` run. +The tables also print the largest reply next to the mean. + +Holdout is the clean measure. Stripped set, answer on a card, before and +after (`find_hash` +`e5be653299abe9b2329822d98ffe09e90ee7cfe96f8711c813a51dc6500b887f`): + +| Graph | Split | Evidence in top 5 | Answer before | Answer after | Mean reply chars | Largest reply | +| ----- | ------- | ----------------- | ------------- | ------------- | ---------------- | -------------- | +| 10x | holdout | 85% (98/116) | 58% (51/88) | 66% (58/88) | 6,084 to 6,365 | 6,509 to 7,112 | +| 10x | tune | 79% (102/130) | 54% (51/94) | 59% (53/90) | 6,080 to 6,369 | 6,700 to 7,360 | +| 10x | all | 81% (200/246) | 56% (102/182) | 62% (111/178) | 6,082 to 6,368 | 6,700 to 7,360 | +| 1x | holdout | 84% (97/116) | 57% (50/88) | 65% (57/88) | 6,093 to 6,411 | 6,502 to 7,056 | +| 1x | tune | 79% (102/130) | 55% (52/94) | 59% (54/91) | 6,085 to 6,403 | 6,698 to 7,358 | +| 1x | all | 81% (199/246) | 56% (102/182) | 62% (111/179) | 6,089 to 6,407 | 6,698 to 7,358 | + +Search ranking did not change, so evidence in the top 5 is the same before +and after. On the 10x holdout, which the code was not tuned on, the answer +is on a card for 7 more questions and for none fewer: 66% against the 60% +bar. On the 1x holdout it is also 7 more and none fewer. The gate on all +246 questions of the 10x graph passes by two points, but that figure +includes the 130 tune questions, which four rounds of changes were measured +against (55%, 56%, 57%, then 59% on the 10x tune split). + +The tune gain is small partly because longer cards match more answers by +chance. On the 10x graph, 3 tune answers that were on a card are now set +aside as chance matches (ports-001, render-003, world-065), and on the 1x +graph 2 are. One tune miss, world-052, also left the count the same way, +which nudges the rate up. To check that the set-aside list did not make +the gain, one list, the union of both runs' set-asides, was applied to +before and after: on the 10x graph the answer is on a card for 99/178 (55.6%) before +and 111/178 (62.4%) after; holdout 51/88 to 58/88; tune 48/90 to 53/90. On +the 1x graph, all goes from 100/179 (55.9%) to 111/179 (62.0%). One 1x tune +answer, ports-026, was lost when two equally good passages of a Blob +swapped places. Most misses left are records no hit reaches, mainly +counting and lifecycle questions, which a `proof state` read should answer +instead. + +The largest reply grew by about 700 characters, to 7,360. That is close +to 7,500 characters, the most this step allowed for the mean reply. A later +change that adds passage lines should watch the largest reply, not only the +mean. diff --git a/benchmarks/proof-recall/scripts/find-recall.mjs b/benchmarks/proof-recall/scripts/find-recall.mjs index c8a0c3b9..9c3869b7 100644 --- a/benchmarks/proof-recall/scripts/find-recall.mjs +++ b/benchmarks/proof-recall/scripts/find-recall.mjs @@ -3,7 +3,7 @@ // whether its top hits hold the evidence, before any paid agent run. // // node scripts/find-recall.mjs [--graph <built-graph-dir>] [--ids a,b] -// [--out <json>] [--jobs 8] +// [--split tune|holdout|all] [--out <json>] [--jobs 8] // // The gate verdict that counts is the one on the 10x graph, given with // `--graph <x10 build>`. The default, the committed 1x graph, is a quick @@ -37,10 +37,15 @@ // 10% of the other questions' cards in the same set // (`chance_rate`), since a card shows those by // accident: "2", "6", "accepted". With --ids, the -// rates come from the chosen questions only. +// rates come from the chosen questions only. With +// --split, they still come from every question, so a +// split's rows are the same as its part of an `all` +// run. // right_effort_first the top Effort in `efforts by match` is the // question's own (single-Effort questions, --all sets) // reply_chars, cards_chars +// the tables print the mean reply size and the +// largest one // // The gate passes when, on the stripped set, evidence_top5 holds for at least // 75% of questions and answer_on_card for at least 60% of those it applies @@ -48,9 +53,19 @@ // records `find_hash`, a hash of the `find` source, so a later run file can // be checked against the search this gate measured. // +// `--split` keeps half the questions back. A question belongs to `tune` when +// the first byte of the SHA-256 of its id is even, and to `holdout` when it is +// odd, so the split never changes and needs no list. Read the misses of the +// tune split to decide what to change in `find`; run the holdout split only +// after the change is done, to see whether it holds on questions nobody +// looked at. The default, `all`, scores every question. Every split asks +// every question, because the chance rates above need all the cards; the +// rows outside the split are dropped after that. +// // The graph is copied, without its key map, into a temporary folder, and one // `proof serve` there answers every query. The folder is removed at the end. import { spawn } from 'node:child_process'; +import { createHash } from 'node:crypto'; import { mkdtemp, rm, writeFile } from 'node:fs/promises'; import { request } from 'node:http'; import { tmpdir } from 'node:os'; @@ -73,6 +88,14 @@ const GATE = { evidence_top5: 0.75, answer_on_card: 0.6 }; // A value answer that matches more than this share of other questions' cards // is set aside: a card shows it by accident. const CHANCE_CUTOFF = 0.1; +const SPLITS = ['tune', 'holdout', 'all']; + +/** `tune` or `holdout`, from the first byte of the SHA-256 of the id. */ +export function splitOf(id) { + return createHash('sha256').update(id).digest()[0] % 2 === 0 + ? 'tune' + : 'holdout'; +} function args(argv) { const out = { @@ -80,6 +103,7 @@ function args(argv) { ids: [], out: join(BENCH, 'runs', 'find-recall.json'), jobs: 8, + split: 'all', }; for (let i = 0; i < argv.length; i++) { const flag = argv[i]; @@ -87,8 +111,11 @@ function args(argv) { else if (flag === '--ids') out.ids = argv[++i].split(','); else if (flag === '--out') out.out = resolve(argv[++i]); else if (flag === '--jobs') out.jobs = Number(argv[++i]); + else if (flag === '--split') out.split = argv[++i]; else throw new Error(`Unknown argument ${flag}`); } + if (!SPLITS.includes(out.split)) + throw new Error(`--split must be one of ${SPLITS.join(', ')}`); return out; } @@ -268,6 +295,7 @@ function tally(rows) { mean_reply_chars: Math.round( chars.reduce((a, b) => a + b, 0) / (chars.length || 1) ), + max_reply_chars: Math.max(0, ...chars), }; } @@ -281,7 +309,8 @@ function printTable(label, groups) { 'ev@5'.padStart(7) + 'card'.padStart(7) + 'effort1'.padStart(9) + - 'chars'.padStart(8) + 'chars'.padStart(8) + + 'max'.padStart(7) ); for (const [name, t] of Object.entries(groups)) console.log( @@ -290,7 +319,8 @@ function printTable(label, groups) { pct(t.evidence_top5).padStart(7) + pct(t.answer_on_card).padStart(7) + pct(t.right_effort_first).padStart(9) + - String(t.mean_reply_chars).padStart(8) + String(t.mean_reply_chars).padStart(8) + + String(t.max_reply_chars).padStart(7) ); } @@ -365,6 +395,9 @@ async function main() { } setAsideChance(rows); + const inSplit = (id) => + options.split === 'all' || splitOf(id) === options.split; + rows.splice(0, rows.length, ...rows.filter((r) => inSplit(r.id))); const order = new Map(jobs.map((job, i) => [`${job.set} ${job.q.id}`, i])); rows.sort( (a, b) => order.get(`${a.set} ${a.id}`) - order.get(`${b.set} ${b.id}`) @@ -372,7 +405,11 @@ async function main() { const bySet = Object.fromEntries( SETS.map((set) => [set, tally(rows.filter((r) => r.set === set))]) ); - printTable(`graph ${options.graph}`, bySet); + const questions = new Set(rows.map((r) => r.id)).size; + printTable( + `graph ${options.graph}, split ${options.split} (${questions} questions)`, + bySet + ); console.log( `Value answers set aside by chance: ${SETS.map( (set) => `${set} ${bySet[set].set_aside_by_chance}` @@ -407,6 +444,8 @@ async function main() { `${JSON.stringify( { graph: options.graph, + split: options.split, + questions, find_hash, gate: { ...GATE, chance_cutoff: CHANCE_CUTOFF, pass }, bySet, diff --git a/packages/flatbread/src/proof/find.test.ts b/packages/flatbread/src/proof/find.test.ts index e8840be9..f15c36d1 100644 --- a/packages/flatbread/src/proof/find.test.ts +++ b/packages/flatbread/src/proof/find.test.ts @@ -411,7 +411,7 @@ test('as-of scores and quotes only versions written by then', (t) => { const [hit] = result.hits; t.is(hit.record.id, 'dec-v1'); t.is(hit.matched.id, 'dec-v1'); - t.false(hit.snippet.text.includes('zeta')); + t.false(hit.passages[0].text.includes('zeta')); t.deepEqual(result.missing, ['zeta']); }); @@ -468,7 +468,7 @@ test('long chains keep the head and the matched version, and snippets near the t ), ]); const result = findRecords(index, 'narwhal beta', { effort: 'eff-a' }); - t.is(result.hits[0].snippet.start, 0); + t.is(result.hits[0].passages[0].start, 0); const cards = renderFindCards(index, result); t.regex(cards, /"As of beta the narwhal/); t.regex( @@ -508,3 +508,295 @@ test('five dense cards stay within the inline budget', (t) => { // A relation with no entry left still names itself, with its count. t.false(cards.includes('in other relations')); }); + +test('the passage leads with query words the title does not already show', (t) => { + const filler = 'Nothing else to note in this sentence. '.repeat(6); + const body = `The walrus kiln layout sets the firing order. ${filler}Past the ridge the kiln stops at 40 m.`; + const index = buildFindIndex([ + rec('eff-a'), + rec('dec-kiln', { title: 'Walrus kiln layout', body }), + ]); + const [hit] = findRecords(index, 'walrus kiln ridge', { + effort: 'eff-a', + }).hits; + // The title shows walrus and kiln, so the passage goes where ridge is. + t.regex(hit.passages[0].text, /ridge the kiln stops at 40 m/); + t.is(hit.passages.length, 1); +}); + +test('the passage centers its matches and keeps the word after the last one', (t) => { + const before = 'Earlier words that set the scene for the reader. '.repeat(6); + const body = `${before}The kiln ridge sits high. ${'Later words. '.repeat( + 20 + )}`; + const at = body.indexOf('kiln'); + const centered = bestSnippet(body, new Set(['kiln', 'ridg'])); + t.true(centered.start < at - 40, `starts at ${centered.start}`); + t.true(centered.end > at + 60, `ends at ${centered.end}`); + // A label right at the window's edge keeps the value after it. + const edge = `kiln ${'a'.repeat(140)} ridge: 4242 and more text after it`; + const snippet = bestSnippet(edge, new Set(['kiln', 'ridg'])); + t.regex(snippet.text, /ridge: 4242/); +}); + +test('a body that matches in two places gets a second passage', (t) => { + const filler = 'Nothing else to note in this sentence. '.repeat(10); + const body = `The kiln fires at dawn. ${filler}The ridge vent closes at noon.`; + const index = buildFindIndex([ + rec('eff-a'), + rec('fnd-two', { title: 'Firing notes', body }), + rec('fnd-one', { title: 'Short notes', body: 'The kiln fires again.' }), + ]); + const result = findRecords(index, 'kiln ridge', { effort: 'eff-a' }); + const two = result.hits.find((hit) => hit.record.id === 'fnd-two'); + const one = result.hits.find((hit) => hit.record.id === 'fnd-one'); + // Ridge is the rarer word, so its passage is the best one and comes first. + t.is(two?.passages.length, 2); + t.regex(two?.passages[0].text ?? '', /ridge vent closes/); + t.regex(two?.passages[1].text ?? '', /kiln fires at dawn/); + t.true((two?.passages[1].end ?? 0) <= (two?.passages[0].start ?? 0)); + // No second place to quote, no second passage. + t.is(one?.passages.length, 1); + const cards = renderFindCards(index, result); + t.regex( + cards, + /"…[^"\n]*The ridge vent closes at noon\." \[chars \d+-\d+ of \d+\]/ + ); +}); + +test('a card quotes the best passage of a Blob its Citation carries', (t) => { + const index = buildFindIndex([ + rec('eff-a'), + rec('fnd-cost', { + title: 'Kiln firing cost', + body: 'The full table is in the attached capture.', + relations: { cites: ['cit-cap'] }, + }), + { + ...rec('cit-cap', { title: 'Capture', body: 'https://example.test/c' }), + frontmatter: { + title: 'Capture', + effort: 'eff-a', + created_at: '2026-01-01T00:00:00.000Z', + blob: 'blb-cap', + }, + }, + rec('blb-cap', { + title: 'Capture payload', + body: '{ "firing": { "glaze_ms": 3.1, "quench_ms": 1.9 } }', + }), + ]); + // Findings only, so the Blob is not a hit and the finding's card quotes it. + const findings = findRecords(index, 'kiln firing quench', { + effort: 'eff-a', + kinds: ['finding'], + }); + t.is(findings.hits[0].attached?.id, 'blb-cap'); + t.regex( + renderFindCards(index, findings), + /blob blb-cap: "\{[^\n]*"quench_ms": 1\.9[^\n]*\[chars 0-51 of 51\]/ + ); + // A query word found only in the quoted Blob is not reported missing. + t.deepEqual(findings.missing, []); + // When the Blob is a hit, its own card quotes it instead. + const all = findRecords(index, 'kiln firing quench', { effort: 'eff-a' }); + t.true(all.hits.some((hit) => hit.record.id === 'blb-cap')); + t.true(all.hits.every((hit) => hit.attached === undefined)); +}); + +test('a quoted Blob must be live: not retracted, superseded, or written after as-of', (t) => { + const cit = ( + id: string, + blob: string, + extra: Record<string, unknown> = {}, + relations: ReadRecord['relations'] = {} + ): ReadRecord => { + const base = rec(id, { title: `Capture ${id}`, body: 'https://x.test' }); + return { + ...base, + frontmatter: { ...base.frontmatter, blob, ...extra }, + relations, + }; + }; + const index = buildFindIndex([ + rec('eff-a'), + rec('fnd-cost', { + title: 'Kiln firing cost', + created_at: '2026-01-05T00:00:00.000Z', + body: 'The full table is in the attached capture.', + relations: { cites: ['cit-old', 'cit-new', 'cit-v1'] }, + }), + cit('cit-old', 'blb-old', { + retracted: true, + retracted_reason: 'wrong build', + }), + rec('blb-old', { title: 'payload', body: '{ "quench_ms": 9.9 }' }), + cit('cit-new', 'blb-new', { created_at: '2026-03-01T00:00:00.000Z' }), + rec('blb-new', { + title: 'payload', + created_at: '2026-03-01T00:00:00.000Z', + body: '{ "quench_ms": 1.9 }', + }), + cit('cit-v1', 'blb-v1', {}, { superseded_by: ['cit-v2'] }), + cit( + 'cit-v2', + 'blb-v2', + { created_at: '2026-04-01T00:00:00.000Z' }, + { supersedes: ['cit-v1'] } + ), + rec('blb-v1', { title: 'payload', body: '{ "quench_ms": 5.5 }' }), + rec('blb-v2', { + title: 'payload', + created_at: '2026-04-01T00:00:00.000Z', + body: '{ "quench": "see v2" }', + }), + ]); + const findings = { effort: 'eff-a', kinds: ['finding' as PrimitiveKind] }; + // The retracted Citation and the superseded one are not quoted. + const now = findRecords(index, 'kiln firing quench', findings); + t.is(now.hits[0].attached?.id, 'blb-new'); + // Before cit-new was written, cit-v1 was still the newest version. + const then = findRecords(index, 'kiln firing quench', { + ...findings, + asOf: '2026-02-01', + }); + t.is(then.hits[0].attached?.id, 'blb-v1'); + // With retracted records let in, a retracted source is marked. + const index2 = buildFindIndex( + [...index.records.values()].filter( + (record) => !['cit-new', 'cit-v1', 'cit-v2'].includes(record.id) + ) + ); + const hidden = findRecords(index2, 'kiln firing quench', findings); + t.is(hidden.hits[0].attached, undefined); + t.deepEqual(hidden.missing, ['quench']); + const shown = findRecords(index2, 'kiln firing quench', { + ...findings, + retracted: 'include', + }); + t.is(shown.hits[0].attached?.retracted, true); + t.regex(renderFindCards(index2, shown), /blob blb-old \(retracted\): /); +}); + +test('a Blob quoted on one card is not quoted again on a later card', (t) => { + const index = buildFindIndex([ + rec('eff-a'), + ...['fnd-a', 'fnd-b'].map((id) => + rec(id, { + title: `Kiln firing ${id}`, + body: 'See the capture.', + relations: { cites: ['cit-cap'] }, + }) + ), + { + ...rec('cit-cap'), + frontmatter: { ...rec('cit-cap').frontmatter, blob: 'blb-cap' }, + }, + rec('blb-cap', { body: '{ "quench_ms": 1.9 }' }), + ]); + const result = findRecords(index, 'kiln firing quench', { + effort: 'eff-a', + kinds: ['finding'], + }); + t.deepEqual( + result.hits.map((hit) => hit.attached?.id), + ['blb-cap', undefined] + ); + t.deepEqual(result.missing, []); +}); + +test('a passage finishes a short word after its last match, not a long token', (t) => { + const short = bestSnippet( + `kiln ${'a'.repeat(148)} ridge 42 tail`, + new Set(['kiln', 'ridg']) + ); + t.regex(short.text, /ridge 42/); + const token = bestSnippet( + `kiln ridge ${'x'.repeat(3000)} tail`, + new Set(['kiln', 'ridg']) + ); + t.true(token.end - token.start <= 160, `${token.end - token.start}`); + const rules = bestSnippet( + `kiln ridge ${'| --- | --- | '.repeat(100)}zeta`, + new Set(['kiln', 'ridg']) + ); + t.true(rules.end - rules.start <= 160, `${rules.end - rules.start}`); +}); + +test('the best passage comes first, even when the second sits earlier', (t) => { + const filler = 'Nothing else to note in this sentence. '.repeat(8); + const body = `The ridge vent closes at noon. ${filler}The kiln and the glaze fire at dawn.`; + const index = buildFindIndex([ + rec('eff-a'), + rec('fnd-two', { title: 'Notes', body }), + ]); + const [hit] = findRecords(index, 'kiln glaze ridge', { + effort: 'eff-a', + }).hits; + t.is(hit.passages.length, 2); + t.regex(hit.passages[0].text, /kiln and the glaze/); + t.regex(hit.passages[1].text, /ridge vent/); + t.true(hit.passages[1].end <= hit.passages[0].start); +}); + +test('reverse links list verdicts before derived records', (t) => { + const index = buildFindIndex([ + rec('eff-a'), + rec('dec-pick', { title: 'Pick the yak kiln', state: 'accepted' }), + ...[1, 2, 3, 4].map((n) => + rec(`fnd-use-${n}`, { relations: { derives_from: ['dec-pick'] } }) + ), + ...[1, 2].map((n) => + rec(`dec-alt-${n}`, { + state: 'rejected', + relations: { rejected_by: ['dec-pick'] }, + }) + ), + ]); + const result = findRecords(index, 'yak kiln', { effort: 'eff-a' }); + t.deepEqual( + result.hits[0].inbound.map((link) => link.relation), + ['rejects', 'derived_by'] + ); + t.regex( + renderFindCards(index, result), + /in: rejects dec-alt-1 [^;]*dec-alt-2 [^;]*; derived_by / + ); +}); + +test('quoting a second passage never pushes out a chain or a link', (t) => { + const ids = [1, 2, 3, 4, 5, 6].map((n) => `dec-c${n}`); + const records: ReadRecord[] = [rec('eff-a')]; + for (let h = 0; h < 5; h++) + ids.forEach((id, i) => + records.push( + rec(`${id}-${h}`, { + title: `Kiln schedule ${h} version ${i} with a long enough title`, + body: `The kiln schedule fires at dawn. ${'Filler words here. '.repeat( + 20 + )}The ridge vent opens at noon.`, + created_at: `2026-0${i + 1}-01T00:00:00.000Z`, + relations: { + ...(i ? { supersedes: [`${ids[i - 1]}-${h}`] } : {}), + ...(i < 5 ? { superseded_by: [`${ids[i + 1]}-${h}`] } : {}), + derives_from: [`fnd-base-${h}`], + }, + }) + ) + ); + for (let h = 0; h < 5; h++) records.push(rec(`fnd-base-${h}`)); + const index = buildFindIndex(records); + const frame = (query: string) => + renderFindCards(index, findRecords(index, query, { effort: 'eff-a' })) + .split('\n') + .filter((line) => /^ {3}(chain|out|in):/.test(line)); + const one = frame('kiln schedule'); + const two = frame('kiln schedule ridge'); + t.true( + renderFindCards( + index, + findRecords(index, 'kiln schedule ridge', { effort: 'eff-a' }) + ).includes('The ridge vent opens at noon.') + ); + t.deepEqual(two, one); +}); diff --git a/packages/flatbread/src/proof/find.ts b/packages/flatbread/src/proof/find.ts index 658ee2d8..e3146175 100644 --- a/packages/flatbread/src/proof/find.ts +++ b/packages/flatbread/src/proof/find.ts @@ -21,6 +21,14 @@ const TITLE_WEIGHT = 2; /** Extra weight for entity terms: numbers with units, people, code refs. */ const ENTITY_BOOST = 1.5; const SNIPPET_CHARS = 160; +/** + * A passage runs on to finish the word after its last match only when that + * word starts within NEXT_WORD_GAP characters of the match and ends within + * NEXT_WORD_CHARS of the window's end. A long token, such as a hash, or a run + * of table rules after a match keeps the plain end instead. + */ +const NEXT_WORD_GAP = 24; +const NEXT_WORD_CHARS = 32; const LINKS_PER_RELATION = 3; const LINK_TITLE_CHARS = 60; const HITS_PER_EFFORT = 3; @@ -33,16 +41,20 @@ const EFFORTS_SHOWN = 5; const CHAIN_VERSIONS_SHOWN = 4; const CODE_REFS_SHOWN = 6; /** - * Characters the cards aim for: a header share plus a share per hit, so five - * hits aim for about 4,000 characters (near 1,000 tokens at 4 characters a - * token). The renderer shrinks link titles, the Effort ranking line, chain - * lines, and link lists to fit, in that order. Titles, snippets, and code - * lines are never cut further, so a card set can still run past the aim. On - * the 10x benchmark graph the whole JSON envelope for five hits ran about - * 5,700 characters at the median and up to about 6,900. + * Characters the cards aim for. The aim has a header share, a share per hit + * for the card's frame (its header, title, chain, and links), and the length + * of every passage the card quotes. Passages count apart because they are + * what matched and are never cut, so a longer or an extra passage never + * pushes out a chain or a link. The frame share was 750 when it also held + * the one passage, about 150 characters; it is 600 now that passages count + * apart. Five frames with one passage each still aim for about 4,000 + * characters (near 1,000 tokens at 4 characters a token). The renderer + * shrinks link titles, the Effort ranking line, chain lines, and link lists + * to fit, in that order. Titles, passages, and code lines are never cut + * further, so a card set can still run past the aim. */ const CARDS_HEADER_CHARS = 300; -const CARD_CHARS = 750; +const CARD_CHARS = 600; const STOPWORDS = new Set( ( @@ -294,15 +306,21 @@ interface FindDoc { readonly length: number; } -/** Relations whose stored ids, read backward, give a record's inbound links. */ +/** + * Relations whose stored ids, read backward, give a record's inbound links, + * in the order a card lists them. The links that pass a verdict on a record + * (it resolves, rejects, or mitigates another) come first. `derived_by` + * comes last: a much-used record collects dozens of derived records, and + * listed first they would fill the line's few entries. + */ const INBOUND_LABELS: Readonly<Record<string, string>> = { - derives_from: 'derived_by', - cites: 'cited_by', resolved_by: 'resolves', rejected_by: 'rejects', mitigated_by: 'mitigates', evidence: 'evidence_for', + cites: 'cited_by', blob: 'blob_of', + derives_from: 'derived_by', }; /** Outbound links a card lists. Supersession shows on the chain line instead. */ const OUTBOUND_RELATIONS: readonly ReadRelation[] = [ @@ -457,17 +475,39 @@ export interface FindHit { readonly chain: readonly ReadRecord[]; readonly score: number; readonly effort: string | null; - readonly snippet: { - readonly text: string; - readonly start: number; - readonly end: number; - readonly length: number; + /** + * Passages of the matched version's body: the best one, then a second + * when another part of the body holds query words the first lacks. The + * second may come from earlier in the body; the offsets say where. Empty + * when the body is. + */ + readonly passages: readonly FindPassage[]; + /** + * The best passage of a Blob attached to the shown version: its own `blob` + * (a Citation), or the `blob` of a Citation it `cites`. Only when that + * payload holds a query word, and the Blob is neither a hit itself nor + * quoted on an earlier card. `retracted` is set when the Blob, or the + * Citation that leads to it, is retracted, which only `--retracted + * include` or `only` lets through. + */ + readonly attached?: { + readonly id: string; + readonly passage: FindPassage; + readonly retracted?: true; }; readonly outbound: readonly FindLink[]; readonly inbound: readonly FindLink[]; readonly code: readonly string[]; } +/** A window of a body, with its offsets in the normalized body. */ +export interface FindPassage { + readonly text: string; + readonly start: number; + readonly end: number; + readonly length: number; +} + export interface FindEffortMass { readonly id: string; readonly title: string; @@ -485,9 +525,12 @@ export interface FindResult { * mass, the summed scores of each one's best 3 hits. */ readonly efforts?: readonly FindEffortMass[]; - /** Query words found in at least one shown hit, as written. */ + /** + * Query words found in at least one shown hit or a Blob a card quotes, as + * written. + */ readonly found: readonly string[]; - /** Query words that no shown hit contains. */ + /** Query words that no shown hit, nor a Blob a card quotes, contains. */ readonly missing: readonly string[]; } @@ -515,62 +558,158 @@ export function queryTerms(query: string): QueryTerm[] { return terms; } +/** BM25's inverse document frequency: rare terms weigh more. */ +function idf(index: FindIndex, term: string): number { + const n = index.docs.size; + const df = index.df.get(term) ?? 0; + return Math.log(1 + (n - df + 0.5) / (df + 0.5)); +} + function bm25( index: FindIndex, doc: FindDoc, terms: readonly string[] ): number { let score = 0; - const n = index.docs.size; for (const term of terms) { const tf = doc.tf.get(term); if (!tf) continue; - const df = index.df.get(term) ?? 0; - const idf = Math.log(1 + (n - df + 0.5) / (df + 0.5)); score += - (idf * tf * (K1 + 1)) / + (idf(index, term) * tf * (K1 + 1)) / (tf + K1 * (1 - B + (B * doc.length) / (index.averageLength || 1))); } return score; } -/** - * The 160-character window of `body` that holds the most distinct matched - * terms (then the most matches, then the earliest), with its offsets. - */ -export function bestSnippet( - body: string, - terms: ReadonlySet<string> -): FindHit['snippet'] { - const source = normalizeBody(body); - const hits = [...wordSpans(source), ...entitySpans(source)] +export interface PassageOptions { + /** Weight of each term, such as its IDF; 1 for a term left out. */ + readonly weights?: ReadonlyMap<string, number>; + /** + * Terms the card already shows, such as its title's. A window scores by + * the terms it adds first; these only break ties. + */ + readonly shown?: ReadonlySet<string>; +} + +interface Window { + /** Summed weight of distinct terms not already shown. */ + readonly fresh: number; + /** Summed weight of all distinct terms. */ + readonly weight: number; + /** + * The matches the passage centers on: the window's matches, less repeats + * at either end whose term also appears further in. + */ + readonly inside: readonly Span[]; + /** Characters from the first of `inside` to the end of the last. */ + readonly spread: number; +} + +/** Matched spans of a normalized body, in text order. */ +function matchSpans(source: string, terms: ReadonlySet<string>): Span[] { + return [...wordSpans(source), ...entitySpans(source)] .filter((span) => terms.has(span.term)) .sort((a, b) => a.start - b.start || a.end - b.end); - let start = 0; - if (hits.length) { - let best = { distinct: -1, count: -1, start: 0 }; - for (const anchor of hits) { - const from = Math.max(0, anchor.start - 20); - const inside = hits.filter( +} + +/** Drops matches at either end whose term the rest still holds. */ +function tighten(spans: readonly Span[]): Span[] { + const kept = [...spans]; + const repeated = (i: number) => + kept.some((span, j) => j !== i && span.term === kept[i].term); + while (kept.length > 1 && repeated(0)) kept.shift(); + while (kept.length > 1 && repeated(kept.length - 1)) kept.pop(); + return kept; +} + +/** + * The best 160-character window over `spans`: the most weight of terms not + * yet shown, then the most weight in all, then the matches set closest + * together, then the earliest. Undefined when no span is given. + */ +function bestWindow( + spans: readonly Span[], + options: PassageOptions +): Window | undefined { + const weight = (term: string) => options.weights?.get(term) ?? 1; + let best: Window | undefined; + for (const anchor of spans) { + const from = Math.max(0, anchor.start - 20); + const inside = tighten( + spans.filter( (span) => span.start >= from && span.end <= from + SNIPPET_CHARS - ); - const distinct = new Set(inside.map((span) => span.term)).size; - if ( - distinct > best.distinct || - (distinct === best.distinct && inside.length > best.count) ) - best = { distinct, count: inside.length, start: from }; + ); + // A match longer than the window, such as a long path, fits no window. + if (!inside.length) continue; + const terms = new Set(inside.map((span) => span.term)); + let fresh = 0; + let all = 0; + for (const term of terms) { + all += weight(term); + if (!options.shown?.has(term)) fresh += weight(term); } - // Start at the top when that drops only a word or two. - start = best.start <= 20 ? 0 : best.start; + const spread = + Math.max(...inside.map((span) => span.end)) - inside[0].start; + if ( + !best || + fresh > best.fresh || + (fresh === best.fresh && + (all > best.weight || (all === best.weight && spread < best.spread))) + ) + best = { fresh, weight: all, inside, spread }; + } + return best; +} + +/** + * Cuts the passage around a window's matches. The matches sit in the middle + * of the 160 characters, and the passage runs to the end of the short word + * after the last match, so the value a matched word names (`"fuse_ms": 42`) + * is not cut off. `floor` and `ceiling` keep the passage clear of another + * passage. + */ +function passageAround( + source: string, + inside: readonly Span[], + floor = 0, + ceiling = source.length +): FindPassage { + const last = inside.length + ? Math.max(...inside.map((span) => span.end)) + : floor; + let start = floor; + if (inside.length) { + const first = inside[0].start; + const slack = SNIPPET_CHARS - (last - first); + start = Math.max(floor, first - Math.floor(slack / 2)); + // Start at the top when that drops only a word or two and keeps them all. + if (floor === 0 && start <= 20 && last <= SNIPPET_CHARS) start = 0; // Start on a word boundary, without skipping past the first match. - const first = hits.find((span) => span.start >= start)?.start ?? start; - while (start > 0 && start < first && /\S/.test(source[start - 1])) start++; + while (start > floor && start < first && /\S/.test(source[start - 1])) + start++; } - let end = Math.min(source.length, start + SNIPPET_CHARS); + // Against a ceiling, give up words before the matches, not after them. + if (ceiling < source.length) + start = Math.max(floor, Math.min(start, ceiling - SNIPPET_CHARS)); + let end = Math.min(ceiling, start + SNIPPET_CHARS); if (end < source.length) { const space = source.lastIndexOf(' ', end); - if (space > start + SNIPPET_CHARS * 0.75) end = space; + if (space > Math.max(last - 1, start + SNIPPET_CHARS * 0.75)) end = space; + } + if (inside.length) { + // A copy, so the shared pattern keeps lastIndex 0 for matchAll. + const word = new RegExp(WORD.source, WORD.flags); + word.lastIndex = last; + const next = word.exec(source); + const after = next ? next.index + next[0].length : 0; + if ( + next && + after > end && + next.index - last <= NEXT_WORD_GAP && + after - end <= NEXT_WORD_CHARS + ) + end = Math.min(ceiling, after); } const window = source.slice(start, end).replace(/\s+/g, ' ').trim(); return { @@ -581,6 +720,61 @@ export function bestSnippet( }; } +/** + * The 160-character window of `body` that best covers the matched terms, + * with its offsets. Without weights or shown terms, that is the window with + * the most distinct matched terms, then the closest-set matches, then the + * earliest. + */ +export function bestSnippet( + body: string, + terms: ReadonlySet<string>, + options: PassageOptions = {} +): FindPassage { + const source = normalizeBody(body); + const best = bestWindow(matchSpans(source, terms), options); + return passageAround(source, best?.inside ?? []); +} + +/** + * Up to two passages of `body`, the best one first. The second comes from + * text outside the first and must hold a matched term that neither the first + * passage nor the shown terms hold, so it only appears when the body matches + * in two places. + */ +export function bestPassages( + body: string, + terms: ReadonlySet<string>, + options: PassageOptions = {} +): FindPassage[] { + const source = normalizeBody(body); + if (!source.trim()) return []; + const spans = matchSpans(source, terms); + const best = bestWindow(spans, options); + const first = passageAround(source, best?.inside ?? []); + if (!best) return [first]; + const seen = new Set([ + ...(options.shown ?? []), + ...spans + .filter((span) => span.start >= first.start && span.end <= first.end) + .map((span) => span.term), + ]); + const rest = spans.filter( + (span) => span.end <= first.start || span.start >= first.end + ); + const second = bestWindow(rest, { ...options, shown: seen }); + if (!second || second.fresh <= 0) return [first]; + // Keep the second window on its side of the first, so they never overlap. + const before = second.inside[0].start < first.start; + const inside = second.inside.filter((span) => + before ? span.end <= first.start : span.start >= first.end + ); + const passage = before + ? passageAround(source, inside, 0, first.start) + : passageAround(source, inside, first.end); + return [first, passage]; +} + function inScope(record: ReadRecord, options: FindOptions): boolean { const retracted = record.frontmatter.retracted === true; const mode = options.retracted ?? 'exclude'; @@ -618,6 +812,62 @@ function inboundLinks(index: FindIndex, record: ReadRecord): FindLink[] { }); } +/** + * The best passage among the Blobs attached to a record: its own `blob`, as + * a Citation has, and the `blob` of each Citation it `cites`. Only a Citation + * and a Blob that `usable` accepts count. A Blob in `skip` is left out: one + * that is a hit, since its own card quotes it, or one an earlier card quotes. + * So is one whose payload holds no query term. Ties go to the earlier Blob + * id. + */ +function attachedPassage( + index: FindIndex, + record: ReadRecord, + terms: ReadonlySet<string>, + options: PassageOptions, + skip: ReadonlySet<string>, + usable: (target: ReadRecord) => boolean +): FindHit['attached'] { + // Each Blob id, with whether it or the Citation leading to it is retracted. + const blobs = new Map<string, boolean>(); + const add = (via: ReadRecord | undefined) => { + const id = via?.frontmatter.blob; + if (!via || typeof id !== 'string' || !usable(via)) return; + const blob = index.records.get(id); + if (!blob || !usable(blob) || skip.has(id)) return; + const retracted = + via.frontmatter.retracted === true || blob.frontmatter.retracted === true; + blobs.set(id, (blobs.get(id) ?? true) && retracted); + }; + add(record); + for (const id of record.relations.cites ?? []) add(index.records.get(id)); + let best: + | { id: string; window: Window; source: string; retracted: boolean } + | undefined; + for (const [id, retracted] of [...blobs].sort(([a], [b]) => + compareIds(a, b) + )) { + const blob = index.records.get(id) as ReadRecord; + const source = normalizeBody(blob.body_excerpt); + const window = bestWindow(matchSpans(source, terms), options); + if ( + window && + (!best || + window.fresh > best.window.fresh || + (window.fresh === best.window.fresh && + window.weight > best.window.weight)) + ) + best = { id, window, source, retracted }; + } + return best + ? { + id: best.id, + passage: passageAround(best.source, best.window.inside), + ...(best.retracted ? { retracted: true as const } : {}), + } + : undefined; +} + /** * Runs a query. Each supersession chain is one hit, scored by its best * member. Only versions that pass the query's scope count: the Effort, the @@ -719,6 +969,25 @@ export function findRecords( ); } else selected = selected.slice(0, limit); const querySet = new Set(termList); + const weights = new Map(termList.map((term) => [term, idf(index, term)])); + // A Citation or Blob a card quotes through must be live at the query's + // time: not retracted unless the retracted mode lets retracted records in, + // not superseded by a version written by then, and written by then itself. + // The kind and Effort filters do not apply to it. + const mode = options.retracted ?? 'exclude'; + const usable = (target: ReadRecord) => { + if (mode === 'exclude' && target.frontmatter.retracted === true) + return false; + if (asOf !== undefined && time(target) > asOf) return false; + const chain = index.chains.get(index.chainOf.get(target.id) ?? ''); + if (!chain || chain.length < 2) return true; + const written = + asOf === undefined ? chain : chain.filter((m) => time(m) <= asOf); + return headsOf(written).some((member) => member.id === target.id); + }; + // Blobs not to quote: hits, which quote themselves, and, card by card, + // Blobs an earlier card already quotes. + const skipBlobs = new Set(selected.map((hit) => hit.record.id)); const foundTerms = new Set<string>(); const hits: FindHit[] = selected.map((hit) => { for (const member of hit.versions) @@ -731,6 +1000,35 @@ export function findRecords( )) if (!code.some((seen) => seen.toLowerCase() === ref.toLowerCase())) code.push(ref); + // The card prints the shown version's title, so the passages lead with + // the query words the title does not already show. + const titleText = text(hit.record.frontmatter.title); + const title = new Set( + [...wordSpans(titleText), ...entitySpans(titleText)].map( + (span) => span.term + ) + ); + const passages = bestPassages(hit.matched.body_excerpt, querySet, { + weights, + shown: title, + }); + const onCard = new Set(title); + for (const passage of passages) + for (const span of matchSpans(passage.text, querySet)) + onCard.add(span.term); + const attached = attachedPassage( + index, + hit.record, + querySet, + { weights, shown: onCard }, + skipBlobs, + usable + ); + if (attached) { + skipBlobs.add(attached.id); + for (const term of termList) + if (index.docs.get(attached.id)?.tf.has(term)) foundTerms.add(term); + } return { record: hit.record, head: hit.head, @@ -738,7 +1036,8 @@ export function findRecords( chain: hit.chain, score: hit.score, effort: effortOf(hit.record), - snippet: bestSnippet(hit.matched.body_excerpt, querySet), + passages, + ...(attached ? { attached } : {}), outbound: outboundLinks(hit.record), inbound: inboundLinks(index, hit.record), code, @@ -869,6 +1168,26 @@ function chainLine(hit: FindHit, shape: CardShape): string | undefined { return ` chain: ${parts.join(' -> ')}`; } +function passageLine(passage: FindPassage): string { + return ` "${passage.text}" [chars ${passage.start}-${passage.end} of ${passage.length}]`; +} + +function attachedLine(attached: NonNullable<FindHit['attached']>): string { + const { id, passage } = attached; + return ` blob ${id}${attached.retracted ? ' (retracted)' : ''}: "${ + passage.text + }" [chars ${passage.start}-${passage.end} of ${passage.length}]`; +} + +/** Characters of a card's passage lines, with their line breaks. */ +function passageChars(hit: FindHit): number { + const lines = [ + ...hit.passages.map(passageLine), + ...(hit.attached ? [attachedLine(hit.attached)] : []), + ]; + return lines.reduce((sum, line) => sum + line.length + 1, 0); +} + function renderCard( index: FindIndex, hit: FindHit, @@ -901,11 +1220,8 @@ function renderCard( ...(hit.matched.id !== record.id ? [` matched: v${version + 1} ${hit.matched.id}`] : []), - ...(hit.snippet.length - ? [ - ` "${hit.snippet.text}" [chars ${hit.snippet.start}-${hit.snippet.end} of ${hit.snippet.length}]`, - ] - : []), + ...hit.passages.map(passageLine), + ...(hit.attached ? [attachedLine(hit.attached)] : []), ]; for (const line of [ chainLine(hit, shape), @@ -984,7 +1300,10 @@ export function renderFindCards(index: FindIndex, result: FindResult): string { if (!result.hits.length) lines.push('no records matched'); return lines.join('\n'); }; - const limit = CARDS_HEADER_CHARS + CARD_CHARS * result.hits.length; + const limit = + CARDS_HEADER_CHARS + + CARD_CHARS * result.hits.length + + result.hits.reduce((sum, hit) => sum + passageChars(hit), 0); let markdown = ''; for (const shape of SHRINK_STEPS) { markdown = render(shape); diff --git a/packages/proof/skills/proof/SKILL.md b/packages/proof/skills/proof/SKILL.md index 65b8e104..a3381bdb 100644 --- a/packages/proof/skills/proof/SKILL.md +++ b/packages/proof/skills/proof/SKILL.md @@ -175,13 +175,14 @@ Pick the read by what you need, in this order: records by their title, full body, and Blob payload, and prints the hit cards inline in the JSON, so you need no second Read. Each card shows the newest version of the record (or, with `--as-of`, the version current - then), its standing, the passage that matched, its supersession chain - with dates, and its links in both directions. Leave out `--effort` when - you do not know the Effort: hits then group by Effort, at most 3 each - unless one Effort clearly leads. The `missing` terms name query words no - hit contains. Ordinary words often land there; a missing distinctive - term, such as a name, a number, or a code reference, is the cue that the - answer may not be recorded. + then), its standing, the passages that matched (up to two, plus one + from an attached Blob payload such as a capture or table), its + supersession chain with dates, and its links in both directions. Leave + out `--effort` when you do not know the Effort: hits then group by + Effort, at most 3 each unless one Effort clearly leads. The `missing` + terms name query words no hit contains. Ordinary words often land there; + a missing distinctive term, such as a name, a number, or a code + reference, is the cue that the answer may not be recorded. 2. **Read one record in full: `proof get <id>`.** 3. **Count or page through records: `proof records`.** `find` shows only the top hits, so never count from it. diff --git a/packages/proof/skills/proof/reference.md b/packages/proof/skills/proof/reference.md index b484f68c..b8719a47 100644 --- a/packages/proof/skills/proof/reference.md +++ b/packages/proof/skills/proof/reference.md @@ -348,10 +348,31 @@ flatbread proof serve [--socket .flatbread/proof/serve.sock] Effort. Unless the top Effort's sum is at least 1.5 times the next one's, at most 3 hits come from one Effort at first; slots still empty then go to the next hits by score. +- **Passages:** a card quotes up to two passages of the matched version's + body, each about 160 characters. The first is the best: the window whose + query words weigh most, counting rare words more (by the same inverse + document frequency that ranks hits) and counting words the card's title + already shows only to break ties. The matches sit in the middle of the + window, and the passage runs a little past 160 characters to finish a + short word right after the last match, so a value after a matched label + (`"fuse_ms": 42`) is not cut off. A second passage appears only when + another part of the body holds a query word the first passage and the + title lack; it may come from earlier in the body, as its offsets show. + When the shown version carries a Blob (a Citation's `blob`), or cites a + Citation that does, and that payload holds a query word, the card also + quotes the payload's best passage on a `blob <id>:` line. A Blob that is a + hit itself, or that an earlier card already quotes, is not quoted again. + The Citation and the Blob must be current: not superseded, not written + after `--as-of`, and not retracted unless `--retracted include` or `only` + lets retracted records in; then the line reads `blob <id> (retracted):`. + Query words found only in a quoted Blob count as matched, not missing. - **Caps:** `--limit` 1 to 10, default 5. Each card holds at most 4 chain versions (the head and the shown and matched versions always appear), 3 - links per relation, and 6 code references. When five cards run past about - 4,000 characters, the cards shrink in this order: link titles, the + links per relation, and 6 code references. The cards aim for 300 + characters plus 600 per card, plus the length of every passage they + quote: about 4,000 characters for five cards with one passage each. + Passages are never cut, so quoting more never pushes out a chain or a + link. Past the aim, the cards shrink in this order: link titles, the `efforts by match` line, the middle of each chain, then link entries. A relation with no entry left shows its count, as `cited_by +4`. When more records matched than are shown, the envelope has `complete: false` and @@ -364,10 +385,12 @@ Each card has these lines: 1. <id> | <kind>[:<kind field>] | <standing> [(head)] | <created date> | score <n> "<title>" matched: v<n> <id> (only when an older version matched) - "…<160-character passage with the most query words>…" [chars <from>-<to> of <length>] + "…<best passage of the matched version>…" [chars <from>-<to> of <length>] + "…<second passage, when the body matches in a second place>…" [chars <from>-<to> of <length>] + blob <id> [(retracted)]: "…<passage of an attached Blob payload>…" [chars <from>-<to> of <length>] chain: v1 <id> <standing> <date> -> (n more) -> v4 <id> <standing> <date> (head) out: <relation> <id> "<title, 60 chars>" (<standing>), … +n more; <relation> +n - in: <derived_by|cited_by|resolves|rejects|mitigates|evidence_for|blob_of> <id> "<title>" (<standing>) + in: <resolves|rejects|mitigates|evidence_for|cited_by|blob_of|derived_by> <id> "<title>" (<standing>) code: <paths, symbol() names, SHAs, #numbers the chain's records name> ```