Skip to content

feat(entity): lazy attribute view with bounded per-attr index seeks - #35

Merged
tiensonqin merged 1 commit into
mainfrom
devin/entity-lazy-view
Oct 5, 2026
Merged

tiensonqin merged 1 commit into
mainfrom
devin/entity-lazy-view

Conversation

@tiensonqin

Copy link
Copy Markdown
Contributor

Summary

Makes entity a lazy attribute view matching upstream datascript/entity.cljc, closing the "Entity Construction Is Not Lazy" divergence recorded in docs/upstream_differences.md.

Call-surface change for entity_attr: before, the first forward-attr lookup forced raw_attrs — a scan of all of the entity's EAVT datoms grouped per-attr via assoc/remove_assoc (O(D²) on wide entities), even when the caller only wanted one attr. Now:

  • entity construction: holds db + eid only — O(1) plus a bounded numeric-eid-exists?-style peek (unchanged, upstream-parity).
  • entity_attr on a forward attr: a bounded (eid, attr) EAVT index seek via the new datoms_by_entity_attr context hook (datoms db Eavt ~e ~a — upstream's db/-search [eid attr]). Resolved values are cached per-attr on the entity (upstream cache).
  • entity_attr on a reverse :_ attr: still a bounded seek on the ref attribute via datoms_by_avet_ref (upstream -lookup-backwards → db/-search [nil attr eid]).
  • entity_attrs/touch (materialize_attrs): materializes on demand — one bounded entity slice — caches the result, and marks the entity materialized (upstream touched), so later lookups of absent forward attrs answer from the cache without another seek.

raw_forward_entity_attrs grouping is now consecutive runs over the attr-ordered EAVT slice (upstream partition-by :a in datoms->cache), removing the per-datom assoc rebuild.

Public API: unchanged — entity, entity_attr, entity_attr_raw, entity_attrs, touch signatures all stable. Entity.context gains one field, datoms_by_entity_attr.

Before / after cost

bench/entity_view_bench.exe (new, committed): db with one wide entity (N attrs) + 2000 unrelated datoms; median-per-call over 200 iters.

metric N before after speedup
entity + single entity_attr 200 218.0 µs 2.8 µs ~79×
entity + single entity_attr 2000 20,049 µs 5.0 µs ~4000×
entity_attrs (all attrs) 200 0.26 ms 0.075 ms ~3.5×
entity_attrs (all attrs) 2000 21.3 ms 1.7 ms ~12.6×
entity construct — ~0.5 µs ~0.5 µs already bounded

Before: single-attr read scales quadratically with entity width (218 µs → 20 ms going 200→2000 attrs, ~92× for 10× datoms). After: flat (~5 µs — one index seek).

Tests

  • test_entity__test_entity_attr_lookup_is_lazy extended with per-entry-point counters: asserts entity_attr calls datoms_by_entity_attr exactly once and never datoms_by_entity; repeated lookups hit the entity cache; entity_attrs scans the entity slice once then caches; post-materialization lookups don't re-seek.
  • All 34 native test suites pass (test/test_*.exe built and run individually — dune runtest requires lein for the upstream cljs bundle, per repo convention).

Docs: docs/upstream_differences.md — removed the resolved "Entity Construction Is Not Lazy" blocker entry and marked follow-up item 7 done. docs/design.md needed no change (no eager-entity mention).

Link to Devin session: https://app.devin.ai/sessions/3477a4224ebc4d8bb979c85ebf56acc8
Open in Devin Desktop: https://app.devin.ai/desktop/session/3477a4224ebc4d8bb979c85ebf56acc8?variant=devin
Requested by: @tiensonqin

Upstream datascript/entity.cljc holds only db + eid and resolves each
attr on demand. Route single forward-attr lookups through a new
datoms_by_entity_attr context hook — a bounded (eid, attr) EAVT index
seek — instead of materializing all forward attrs on first lookup.
Reverse :_ lookups keep their bounded seek on the ref attribute.

entity_attrs/touch still materialize on demand, now caching the result
on the entity and marking it materialized (upstream 'touched'), so
later lookups of absent forward attrs answer from the cache without
another index seek.

Also replace the per-datom assoc/remove_assoc grouping in
raw_forward_entity_attrs (O(D^2) on wide entities) with consecutive-run
grouping — the EAVT entity slice is already attr-ordered (upstream's
partition-by :a).
@devin-ai-integration

Copy link
Copy Markdown

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@tiensonqin
tiensonqin merged commit 4fe87ec into main Oct 5, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant