Skip to content

perf(query): collapse single-row relations into constants and scope predicate terms - #39

Closed
tiensonqin wants to merge 1 commit into
mainfrom
devin/query-v3-consts
Closed

tiensonqin wants to merge 1 commit into
mainfrom
devin/query-v3-consts

Conversation

@tiensonqin

Copy link
Copy Markdown
Contributor

Summary

First slice of the v3 query-model migration in docs/query_planner.md: reduce binding-map churn in the impl/query_where.ml relation evaluator. Two changes, both semantics-preserving:

1. Single-row relations collapse into constants (substitute_row_consts)

When the joined relation reduces to one row, every attr is a constant — upstream rel->consts. Instead of keeping a 1-row relation and building a binding map per subsequent clause, the row's values are substituted into the remaining clauses via bound_relation_clause (moved earlier for declaration order; body unchanged), so later patterns resolve through bounded index seeks rather than hash/stream joins over the whole attribute.

  • not/not-join bodies keep their vars: anti_join and ensure_not_has_outer_binding are var-based, and the projected join vars already carry the constants.
  • Attr-valued consts (Result_attr) stay join vars: substituting them into e/v positions produces QAttr, which query_value_term ignores and direct rows never re-check — the join key already compares attr values correctly (Result_attr a ≡ Keyword a via query_results_equivalent).

2. Predicate/filter terms evaluated only against the relation that binds them (filter_term_value_getter)

eval_query_term only consults bindings for QVars present in them, so terms the relation cannot bind — unbound vars, idents, lookup refs, sources, wildcards — produce the same value on every row. They now resolve once lazily (raising on first-row use exactly as before) instead of rebuilding a row_binding map and evaluating per row via the value_of_relation_term/relation_comparison_matches fallback (both deleted).

Applied at the top of all three apply loops (eval_relation_from_relation, eval_relation_from_empty, eval_relation_clauses inner), so a 1-row intermediate anywhere in the clause chain triggers the collapse.

Test

All 34 native test suites pass (test_* binaries, built and run individually — dune build test/ needs lein). One initial regression in test_logseq_query_planners (attr-valued const substituted into v position produced unconstrained QAttr → over-acceptance) was fixed by keeping Result_attr bindings as join vars.

Bench

bench/bench_ocaml.exe --size 5000 --warmup-ms 200 --sample-ms 400 --samples 5, native, main vs this branch (two runs):

query before after (run 1) after (run 2)
q1 0.13500 0.13622 0.13611
q2 0.43155 0.44865 0.43441
q3 0.71576 0.72881 0.72844
q4 1.02 1.06 1.03
qpred1 0.51159 0.52928 0.52941
qpred2 0.51547 0.52703 0.52255
q2pred 0.16685 0.16925 0.17481
q5-shortcircuit 0.01075 0.01092 0.01079
get-page-data 0.52737 0.52381 0.51386
add-1 90.54 90.00 90.05
add-5 106.23 106.42 108.55
add-all 112.74 113.40 114.00

Deltas are within run-to-run noise (±3–4%): the bench queries don't produce single-row intermediate relations, so the collapse is inert there; qpred terms are relation-bound and take the same getter path minus the per-row row_binding allocation. The win shows on workloads whose joins reduce to 1-row intermediates, where subsequent clauses now use bounded seeks.

Link to Devin session: https://app.devin.ai/sessions/0b8c5abdc6994feaad86a56c0cca5db6
Open in Devin Desktop: https://app.devin.ai/desktop/session/0b8c5abdc6994feaad86a56c0cca5db6?variant=devin
Requested by: @tiensonqin

…redicate terms

First slice of the v3 query-model migration from docs/query_planner.md,
reducing binding-map churn in the relation evaluator:

- When the joined relation reduces to a single row, its attrs are
  constant: propagate them into the remaining clauses (substitute_row_consts)
  so later patterns resolve through bounded index seeks instead of
  hash/stream joins over the whole attribute. not/not-join bodies keep
  their vars (anti_join and the insufficient-bindings check are
  var-based); attr-valued consts stay join vars since QAttr carries no
  value constraint in e/v positions.
- Predicate/filter terms the relation cannot bind (unbound vars, idents,
  lookup refs, sources, wildcards) evaluate to the same result on every
  row, so they resolve once lazily instead of rebuilding a binding map
  and evaluating per row.

All 34 native test suites pass; q1-q4/qpred/q2pred bench deltas are
within run-to-run noise.
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@tiensonqin tiensonqin closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant