Repository navigation
perf: certify full-text Top-K and reuse write buffer settings - #1245
Merged
Merged
Conversation
JingsongLi
force-pushed
the
codex/fts-bounded-topk
branch
from
October 28, 2026 09:17
87dc89c to
2d31b97
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Closes #1243.
Global full-text search retains every archive's candidates before applying the caller's Top-K. Bound per-shard candidates when equivalence to the existing final result can be certified, and provide a reproducible comparison against exhaustive collection.
Scalar global-index sorting also reuses Java's
write-buffer-sizesetting instead of the recently introduced Rust-only sort-memory option.Brief change log
internal-apifeature for benchmarks and tests. The production optimization has no table-level enable/disable option.write-buffer-sizefor scalar sorting, with Java's 256 MiB default. Preserve builder-option overrides, spill/merge behavior, oversized-key errors, and FM's existing streaming path.Validation
A local all-targets attempt exceeded available disk during linking. The complete core library and related integration target passed; the remaining integration/platform matrix is left to CI.
API and Format
The existing Java option
write-buffer-sizereplacessorted-index.sort-memoryfor scalar sorting; the default changes from 64 MiB to 256 MiB. The optional search-task candidate limit has a default for older frames. Index archives and score semantics are unchanged.Documentation
Document the automatic optimization, internal benchmark command and Java-compatible sort-memory setting. Preserve the historical performance sample with its original source revision.