Repository navigation
feat(entity): lazy attribute view with bounded per-attr index seeks - #35
Merged
Merged
Conversation
Upstream datascript/entity.cljc holds only db + eid and resolves each attr on demand. Route single forward-attr lookups through a new datoms_by_entity_attr context hook — a bounded (eid, attr) EAVT index seek — instead of materializing all forward attrs on first lookup. Reverse :_ lookups keep their bounded seek on the ref attribute. entity_attrs/touch still materialize on demand, now caching the result on the entity and marking it materialized (upstream 'touched'), so later lookups of absent forward attrs answer from the cache without another index seek. Also replace the per-datom assoc/remove_assoc grouping in raw_forward_entity_attrs (O(D^2) on wide entities) with consecutive-run grouping — the EAVT entity slice is already attr-ordered (upstream's partition-by :a).
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Makes
entitya lazy attribute view matching upstreamdatascript/entity.cljc, closing the "Entity Construction Is Not Lazy" divergence recorded indocs/upstream_differences.md.Call-surface change for
entity_attr: before, the first forward-attr lookup forcedraw_attrs— a scan of all of the entity's EAVT datoms grouped per-attr viaassoc/remove_assoc(O(D²) on wide entities), even when the caller only wanted one attr. Now:entityconstruction: holds db + eid only — O(1) plus a boundednumeric-eid-exists?-style peek (unchanged, upstream-parity).entity_attron a forward attr: a bounded(eid, attr)EAVT index seek via the newdatoms_by_entity_attrcontext hook (datoms db Eavt ~e ~a— upstream'sdb/-search [eid attr]). Resolved values are cached per-attr on the entity (upstreamcache).entity_attron a reverse:_attr: still a bounded seek on the ref attribute viadatoms_by_avet_ref(upstream-lookup-backwards→db/-search [nil attr eid]).entity_attrs/touch(materialize_attrs): materializes on demand — one bounded entity slice — caches the result, and marks the entity materialized (upstreamtouched), so later lookups of absent forward attrs answer from the cache without another seek.raw_forward_entity_attrsgrouping is now consecutive runs over the attr-ordered EAVT slice (upstreampartition-by :aindatoms->cache), removing the per-datom assoc rebuild.Public API: unchanged —
entity,entity_attr,entity_attr_raw,entity_attrs,touchsignatures all stable.Entity.contextgains one field,datoms_by_entity_attr.Before / after cost
bench/entity_view_bench.exe(new, committed): db with one wide entity (N attrs) + 2000 unrelated datoms; median-per-call over 200 iters.entity_attrentity_attrentity_attrs(all attrs)entity_attrs(all attrs)entityconstructBefore: single-attr read scales quadratically with entity width (218 µs → 20 ms going 200→2000 attrs, ~92× for 10× datoms). After: flat (~5 µs — one index seek).
Tests
test_entity__test_entity_attr_lookup_is_lazyextended with per-entry-point counters: assertsentity_attrcallsdatoms_by_entity_attrexactly once and neverdatoms_by_entity; repeated lookups hit the entity cache;entity_attrsscans the entity slice once then caches; post-materialization lookups don't re-seek.test/test_*.exebuilt and run individually —dune runtestrequires lein for the upstream cljs bundle, per repo convention).Docs:
docs/upstream_differences.md— removed the resolved "Entity Construction Is Not Lazy" blocker entry and marked follow-up item 7 done.docs/design.mdneeded no change (no eager-entity mention).Link to Devin session: https://app.devin.ai/sessions/3477a4224ebc4d8bb979c85ebf56acc8
Open in Devin Desktop: https://app.devin.ai/desktop/session/3477a4224ebc4d8bb979c85ebf56acc8?variant=devin
Requested by: @tiensonqin