Repository navigation
test(search): measure typo forgiveness only when strict finds nothing - #37
Merged
Merged
Conversation
Adds the typo runner: strict first, and only on an empty result a known word beginning becomes word* and an unknown word its nearest archive words. Reruns strict, legacy, combined and --discover on the same store, sweeps 33 guards on dev, adds six unlabelled absent-near-word diagnostics and a top-ten parity check against the correctly spelled query. Held-out ran once on the frozen guard and is informational (the split was reused). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Variant C runs the narrow step and, only when it still finds nothing, falls back to an any-word OR, an all-but-one-word OR, or the production --discover path, listing each hit's missing words. On dev the discover fallback equals legacy in every category and pays legacy's 80/80 no-answer hits; held-out ran once on the frozen choice (informational). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The first run (B only) stays as typo-dev.json and typo-heldout.json; the run with the partial fallbacks is typo-c-dev.json and typo-c-heldout.json. The README names each variant, the runner hash behind each file and the commands for dev and the single held-out run. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
leemour
marked this pull request as ready for review
October 12, 2026 22:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task
Measure the narrower first step for typo-tolerant default search in cli-messaging (help only when strict search finds nothing), next to strict, legacy, combined and
--discover.Why
The owner wants typo-tolerant search by default and
--language legacygone. The full combined switch failed its gates; this gives the measured numbers for the plan in cli-messaging.What
performance/search/message-search/typo.tsand atypoentry inrun.py.typo-dev.json: 33 guards on dev.typo-heldout.json: one held-out run of the frozen guard (informational, the split was reused).Testing
run.py typecheckpasses. Measured against cli-messaging 0cc88b1f (origin/main). Synthetic store only.🤖 Generated with Claude Code