Skip to content

Subtract zipper - #94

Merged
imlvts merged 15 commits into
masterfrom
subtract_zipper
Oct 23, 2026
Merged

imlvts merged 15 commits into
masterfrom
subtract_zipper

Conversation

@marcin-rzeznicki

Copy link
Copy Markdown
Collaborator

Summary

This PR adds SubtractZipper<V, A, B>, a virtual zipper exposing the materialized algebraic subtraction:

A - B

without eagerly constructing the resulting trie.

The zipper exposes the same observable Zipper state as a zipper over a materialized subtraction result:

  • values are computed using psubtract;
  • LHS-only subtrees survive unchanged;
  • shared branches are retained only when their subtree difference is non-empty;
  • path_exists, is_val, child_mask, child_count, and val reflect the resulting virtual trie.

Traversal

SubtractZipper implements ZipperMoving and ZipperIteration.

Several movement operations are specialized because the default implementations would repeatedly recompute the virtual subtraction state after every intermediate byte movement.

The optimized implementations move the two backing zippers in sync and refresh the cached virtual state only at externally observable stopping points.

This includes optimized implementations of:

descend_to_existing
descend_to_val
descend_to_existing_byte
descend_until
descend_until_max_bytes
ascend_until
ascend_until_branch
to_next_sibling_byte
to_prev_sibling_byte
to_next_step
to_next_val
descend_first_k_path
to_next_k_path

Overlapping subtrees are inspected lazily and short-circuit as soon as the difference is known to be non-empty. Once RHS no longer contains the current path, traversal can treat the remaining subtree as plain LHS.

val_count() is computed lazily and cached.

ZipperValues split

The PR also separates current-focus value access from arbitrary-path value access.

Previously ZipperValues<V> required both:

val() -> Option<&V>
val_at(path) -> Option<&V>

This is problematic for computed zippers: a value at the current focus can be cached and returned by reference, while val_at() may need to synthesize a new value with no stable backing storage.

The traits are therefore split into:

pub trait ZipperValues<V> {
    fn val(&self) -> Option<&V>;
}

pub trait ZipperValuesAt<V>: ZipperValues<V> {
    fn val_at<K: AsRef<[u8]>>(&self, path: K) -> Option<&V>;
}

This lets computed zippers such as SubtractZipper remain compatible with generic algorithms that only require val(), while concrete zippers can continue supporting arbitrary-path borrowed access.

@imlvts

imlvts commented Sep 11, 2026

Copy link
Copy Markdown
Collaborator

Writing a fuzzer with materialized subtraction as an oracle reveals a few bugs:

use pathmap::{PathMap, zipper::*};

fn check(label: &str, actual: impl std::fmt::Debug + PartialEq<bool>) {
    println!("{label} {actual:?} (expected false)");
    assert_eq!(actual, false, "{label}");
}

fn main() {
    // The trace format records the boolean; the path is intentionally unchanged.
    let lhs = PathMap::from_iter([([2], 7u64)]);
    let rhs = PathMap::<u64>::new();
    let mut zipper = SubtractZipper::new(lhs.read_zipper(), rhs.read_zipper());
    zipper.descend_to([2]);
    check("zero-k result", zipper.descend_first_k_path(0));
    assert_eq!(zipper.path(), [2], "zero-k must not move");

    // This exercises seek_k_path exhaustion and refresh-on-failure.
    let lhs = PathMap::from_iter([([10, 20], 1u64), ([10, 21], 2)]);
    let rhs = PathMap::from_iter([([10, 20], 1u64), ([10, 21], 2)]);
    let mut zipper = SubtractZipper::new(lhs.read_zipper(), rhs.read_zipper());
    zipper.descend_to([10]);
    check("k-path result", zipper.descend_first_k_path(2));
    check("k-path cache valid", zipper.path_exists());
    check("k-path result", zipper.descend_first_k_path(2));
    check("k-path state valid", zipper.path_exists());

    // to_next_k_path must refresh after advance_to_next_subtree fails.
    let lhs = PathMap::from_iter([([10, 20], 1u64), ([10, 21], 2)]);
    let rhs = lhs.clone();
    let mut zipper = SubtractZipper::new(lhs.read_zipper(), rhs.read_zipper());
    zipper.descend_to([10, 20]);
    check("next-k result", zipper.to_next_k_path(2));
    check("next-k state valid", zipper.path_exists());

    // empty result, so result.read_zipper().path_exists() reported true.
    let empty_lhs = PathMap::<u64>::new();
    let empty_rhs = PathMap::<u64>::new();
    let result = empty_lhs.subtract(&empty_rhs);
    let result_zipper = result.read_zipper();
    check("empty-map subtraction", result_zipper.path_exists());
}

@marcin-rzeznicki

Copy link
Copy Markdown
Collaborator Author

Thanks @imlvts I will turn these into tests and fix'em

@marcin-rzeznicki

Copy link
Copy Markdown
Collaborator Author

@imlvts The first failure is somewhat interesting:

check("zero-k result", zipper.descend_first_k_path(0));

The docs say that it must return true if the zipper successfully descended k steps (you can argue that descending 0 steps is always a success - I return true in this case), and then it contradicts itself saying that returning false means that zipper did not move (it obviously did not have to move when k=0). @luketpeterson ?

@marcin-rzeznicki

Copy link
Copy Markdown
Collaborator Author

@imlvts This passes: // This exercises seek_k_path exhaustion and refresh-on-failure.

@marcin-rzeznicki

Copy link
Copy Markdown
Collaborator Author

@imlvts This also passes: // to_next_k_path must refresh after advance_to_next_subtree fails

@luketpeterson

luketpeterson commented Sep 12, 2026 via email

Copy link
Copy Markdown
Collaborator

@adamv-symbolica

Copy link
Copy Markdown
Contributor

@imlvts @luketpeterson can we get this decided on and merged?

@imlvts

imlvts commented Sep 23, 2026

Copy link
Copy Markdown
Collaborator

I'm good with this. seems reasonable. but will need merges.

@marcin-rzeznicki

Copy link
Copy Markdown
Collaborator Author

I am still working on one improvement (survival witness - makes some benchmarks faster) and benchmarks

marcin-rzeznicki and others added 3 commits October 1, 2026 18:59
Conflicts:
- src/utils/mod.rs: keep the branch's u64_4_raw_clear_bit helper in
  clear_bit, and add master's new toggle_bit.
- src/zipper.rs: take master's doc example for val_at.
- src/prefix_zipper.rs: take master's side (the ZipperValues test import
  is unused after master's changes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When the child being ascended out of is not part of the subtraction (an
off-trie focus, or a path the RHS removed), has_surviving_sibling() counted
the parent's one remaining child as a sibling, so the ascent stopped at a
non-branch. Such a parent is a branch only with two surviving children; one
is enough to prove it survives for the next step.

Fixes the six *ascend_until_from_off_trie* shared-suite tests that master
added, and adds a test for the subtracted-child case against the
materialized subtraction.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Bench A/B vs base: done

job log · 2026-10-23 21:55:05 UTC

base 70d3dd3 → head 280c63d, 3 round(s), median of each run averaged; negative is faster

bench cases geomean largest gain largest loss >5% faster >5% slower
shakespeare 7 -4.3% -10.7% shakespeare/shakespeare_sentences_val_count +0.2% shakespeare/shakespeare_sentences_val_count_act 3 0
cities 5 +2.5% -0.2% cities/cities_insert +8.1% cities/cities_get_act 0 1
sparse_keys 96 -0.2% -8.7% sparse_iter/160 +4.4% sparse_iter/20 1 0
binary_keys 77 -0.7% -5.4% binary_set_val_at_short_replace/0 +7.9% binary_insert/1600 1 1
superdense_keys 104 -0.6% -5.8% superdense_zipper_cursor/800 +2.5% superdense_val_count_bench/400 2 0
act_paths 46 -0.7% -8.9% shape_map_to_act_cata/random_long +3.9% size_map_to_paths/100000 1 0
zipper_head_owned 16 +2.2% -6.6% zipper_head_owned/map_reader_val +19.4% zipper_head_owned/map_reader_is_val 1 3
product_zipper 4 -1.9% -5.2% product_zipper/introspecting_pathmap_pathmap -0.1% product_zipper/generic_pathmap_act 1 0
15 case(s) moved more than 5%
bench case base head change
zipper_head_owned zipper_head_owned/map_reader_is_val 4.18 µs 4.98 µs +19.4%
shakespeare shakespeare/shakespeare_sentences_val_count 2.97 ms 2.65 ms -10.7%
zipper_head_owned zipper_head_owned/shared_parent_reader_val 1.33 µs 1.47 µs +10.5%
shakespeare shakespeare/shakespeare_sentences_get 51.68 ms 46.75 ms -9.5%
act_paths shape_map_to_act_cata/random_long 44.37 ms 40.43 ms -8.9%
sparse_keys sparse_iter/160 4.18 µs 3.81 µs -8.7%
cities cities/cities_get_act 677.53 ms 732.30 ms +8.1%
binary_keys binary_insert/1600 162.87 µs 175.77 µs +7.9%
zipper_head_owned zipper_head_owned/map_reader_val 2.94 µs 2.74 µs -6.6%
zipper_head_owned zipper_head_owned/owned_head_write_creation_cleanup_unchecked 14.60 µs 15.52 µs +6.3%
shakespeare shakespeare/shakespeare_sentences_insert 54.94 ms 51.59 ms -6.1%
superdense_keys superdense_zipper_cursor/800 5.55 µs 5.23 µs -5.8%
superdense_keys superdense_zipper_cursor/400 2.67 µs 2.52 µs -5.5%
binary_keys binary_set_val_at_short_replace/0 3 ns 3 ns -5.4%
product_zipper product_zipper/introspecting_pathmap_pathmap 454.70 ms 430.90 ms -5.2%

Full tables per bench are in the job log and the bench-out artifact.

@imlvts
imlvts merged commit 8600ef0 into master Oct 23, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants