Skip to content

fix: support all non-negative int64 deletion vector positions - #917

Draft
wirybeaver wants to merge 1 commit into
apache:mainfrom
wirybeaver:fix/dv-full-positive-int64
Draft

wirybeaver wants to merge 1 commit into
apache:mainfrom
wirybeaver:fix/dv-full-positive-int64

Conversation

@wirybeaver

@wirybeaver wirybeaver commented Sep 5, 2026 •

Copy link
Copy Markdown

Summary

Use sparse ordered high-32-bit buckets to represent every non-negative int64_t deletion-vector position without storage proportional to the largest key.

  • preserve portable Roaring/Puffin encoding and serialize populated buckets in order

  • avoid overflow when a position-delete range ends at INT64_MAX; document the half-open endpoint limitation

  • retain fixed framing-size prechecks before allocation/output creation, without adding a configurable test-only writer factory

  • make emptiness checks constant-time under the nonempty-bucket invariant and reuse the computed framing length during serialization

DV-PR-2 in #916. Based directly on main at 3c5715c; no fixture-only or footer-validation dependency. Existing independent raw Roaring fixtures remain, but the standalone fixtures from #913 are not included.

Test plan

GCC 14 focused checks on the four-feature integration tree based on 3c5715c; the final changed branch sources were verified to match the tested tree.

  • util_test --gtest_filter='RoaringPositionBitmap*:*PositionDeleteIndex*:*PositionDeleteRangeConsumer*': 64 passed
  • data_test --gtest_filter='DVWriterTest.*': 11 passed

Existing sparse roundtrip coverage also checks restored cardinality; no new exhaustive boundary matrix was introduced.

  • Repository pre-commit checks and git diff --check: passed

@wirybeaver
wirybeaver force-pushed the fix/dv-full-positive-int64 branch 2 times, most recently from 0a6cc1a to 5c494b6 Compare September 5, 2026 23:16
@wirybeaver
wirybeaver force-pushed the fix/dv-full-positive-int64 branch from 5c494b6 to 948bfeb Compare October 1, 2026 19:57
@wirybeaver
wirybeaver force-pushed the fix/dv-full-positive-int64 branch from 948bfeb to 002ccd9 Compare October 1, 2026 21:45

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant