Repository navigation
perf(query): collapse single-row relations into constants and scope predicate terms - #39
Closed
tiensonqin wants to merge 1 commit into
Closed
tiensonqin wants to merge 1 commit into
tiensonqin wants to merge 1 commit into
Conversation
…redicate terms First slice of the v3 query-model migration from docs/query_planner.md, reducing binding-map churn in the relation evaluator: - When the joined relation reduces to a single row, its attrs are constant: propagate them into the remaining clauses (substitute_row_consts) so later patterns resolve through bounded index seeks instead of hash/stream joins over the whole attribute. not/not-join bodies keep their vars (anti_join and the insufficient-bindings check are var-based); attr-valued consts stay join vars since QAttr carries no value constraint in e/v positions. - Predicate/filter terms the relation cannot bind (unbound vars, idents, lookup refs, sources, wildcards) evaluate to the same result on every row, so they resolve once lazily instead of rebuilding a binding map and evaluating per row. All 34 native test suites pass; q1-q4/qpred/q2pred bench deltas are within run-to-run noise.
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
First slice of the v3 query-model migration in
docs/query_planner.md: reduce binding-map churn in theimpl/query_where.mlrelation evaluator. Two changes, both semantics-preserving:1. Single-row relations collapse into constants (
substitute_row_consts)When the joined relation reduces to one row, every attr is a constant — upstream
rel->consts. Instead of keeping a 1-row relation and building a binding map per subsequent clause, the row's values are substituted into the remaining clauses viabound_relation_clause(moved earlier for declaration order; body unchanged), so later patterns resolve through bounded index seeks rather than hash/stream joins over the whole attribute.not/not-joinbodies keep their vars:anti_joinandensure_not_has_outer_bindingare var-based, and the projected join vars already carry the constants.Result_attr) stay join vars: substituting them into e/v positions producesQAttr, whichquery_value_termignores and direct rows never re-check — the join key already compares attr values correctly (Result_attr a≡Keyword aviaquery_results_equivalent).2. Predicate/filter terms evaluated only against the relation that binds them (
filter_term_value_getter)eval_query_termonly consults bindings forQVars present in them, so terms the relation cannot bind — unbound vars, idents, lookup refs, sources, wildcards — produce the same value on every row. They now resolve once lazily (raising on first-row use exactly as before) instead of rebuilding arow_bindingmap and evaluating per row via thevalue_of_relation_term/relation_comparison_matchesfallback (both deleted).Applied at the top of all three
applyloops (eval_relation_from_relation,eval_relation_from_empty,eval_relation_clausesinner), so a 1-row intermediate anywhere in the clause chain triggers the collapse.Test
All 34 native test suites pass (
test_*binaries, built and run individually —dune build test/needs lein). One initial regression intest_logseq_query_planners(attr-valued const substituted into v position produced unconstrainedQAttr→ over-acceptance) was fixed by keepingResult_attrbindings as join vars.Bench
bench/bench_ocaml.exe --size 5000 --warmup-ms 200 --sample-ms 400 --samples 5, native, main vs this branch (two runs):Deltas are within run-to-run noise (±3–4%): the bench queries don't produce single-row intermediate relations, so the collapse is inert there; qpred terms are relation-bound and take the same getter path minus the per-row
row_bindingallocation. The win shows on workloads whose joins reduce to 1-row intermediates, where subsequent clauses now use bounded seeks.Link to Devin session: https://app.devin.ai/sessions/0b8c5abdc6994feaad86a56c0cca5db6
Open in Devin Desktop: https://app.devin.ai/desktop/session/0b8c5abdc6994feaad86a56c0cca5db6?variant=devin
Requested by: @tiensonqin