Repository navigation
Conversation
Folding many small digests into one means flushing each and calling merge_digests, which sorts, compresses and allocates per digest even when it holds only a few dozen values. TDigest::absorb merges another digest in place. If the other digest has never flushed, its buffered values are pushed into the receiver, exactly as if they had been pushed there directly. Otherwise both are flushed and combined with merge_digests. The result keeps the larger max_size. Adds unit and property tests for both paths and an absorb_small_digests bench comparing absorb with flush plus merge_digests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Folding many small digests into one, for example per-batch digests built with
push, currently means flushing each one and callingmerge_digests. That sorts, compresses and allocates for every digest, even when it holds only a few dozen values.This PR adds
TDigest::absorb(&mut self, other: TDigest):otherhas never flushed, its buffered values are pushed intoself. The result is exactly what pushing them intoselfdirectly would give.merge_digests.max_size. As withpush, callflushbefore queries.This is a new public method, so it's a minor-version addition with no existing behavior changed.
Tests
PartialEq); absorbing a flushed digest equalsmerge_digests; empty digests in both positions; the largermax_sizeis kept.absorbonto the merge path makes the buffered-path tests fail.Bench and docs
absorb_small_digestsbench: 10,000 values in 64-value digests,absorbvs. flush +merge_digests.README.mdanddocs/architecture.mddescribe the method, and the cost table has a row for it.Checked locally:
cargo test(debug and release, all feature sets), clippy-D warnings, rustdoc-D warnings,no_std, and the MSRV 1.62 checks from CI.This is independent of the integer-key sort PR. If both land,
absorbpushes through whichever buffer representation is in place.🤖 Generated with Claude Code