Repository navigation
Native: record ids go to the engine as a parameter, not in the statement text - #202
Conversation
…ent text Port of humemai/arcadedb-embedded-python#529 (branch perf/native-rid-param, never merged there) onto current main. On the native build a record id was pasted into the SQL text, so each traversal step and each lookup_by_rid was a new statement text. The engine caches a parsed statement by its text (arcadedb.sqlStatementCache, 300 entries) and its literal parameterizer keeps a literal in a FROM target, so 25 records meant 25 cache entries. The id now goes over as a parameter (the RID the codec already tags as R): lookup_by_rid (SELECT FROM ?), Vertex.get_out_edges/get_in_edges/get_both_edges, Vertex.new_edge, Edge.get_in/get_out, save() (UPDATE :rid MERGE :row) and delete() (DELETE FROM :rid). Id validation and error messages are unchanged. test_native_statement_texts.py counts the distinct statement texts (native only, not collected on JPype); the new behaviour tests in test_graph_api.py run on both builds. Untimed counts only: a timed A/B decides it before merge. Refs #19 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…2 % slower there First timed A/B (ab.py, laptop, native wheel of main vs this PR, 12 clean pairs): lookup_rid_us 1.022 [1.019, 1.025] (slower), out_edges_us 0.792 [0.790, 0.794], edge_ends_us 0.792 [0.789, 0.794], save_us 0.805 [0.801, 0.809] (faster). Guards flat, A/A clean. Narrow the PR to the paths that got faster; lookup_by_rid returns to main's text. database.py is now identical to main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Second timed A/B, on this narrowed head
Result: ok (all 6 B/A figures have >= 10 clean pairs and CI width <= 0.08, the A/A control is clean). Verdict by that rule: SHIP. The edge getters and edge ends are about a fifth faster and One figure to keep in view: |
…both native-only test ignores) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Conflicts resolved: - src/arcadedb/_native/records.py: keeps this branch's _save_update (lazy fingerprint, refuses a stale overwrite) and applies #202's rule inside it: the record id goes to the engine as the :rid parameter for both the stored-record read (_SELECT = "SELECT FROM :rid") and the UPDATE .. MERGE .. RETURN AFTER (_UPDATE now carries RETURN AFTER, which only _save_update uses). Without this, #202's test_native_statement_texts fails on the pasted ids in the save path. - tests/conftest.py, tests/test_graph_api.py: both sides appended; kept both. The duplicate `import pytest` from the two sides is dropped and main's import order kept (local isort disagrees with main's CI order; left as main has it). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No textual conflicts: main's #202 (record ids as a :rid parameter, src/arcadedb/_native/records.py and database.py) and #201 (run_in_transaction witness) touch other hunks than this branch. The record-id rule of #202 is kept: this branch pastes no record id into SQL text (the lazy per-hit vector load, if present, goes through lookup_by_rid, which keeps its literal id by design, see tests/test_native_statement_texts.py). Static checks after the merge: compile() and pyflakes of the changed files (the unused typing.Optional in records.py is on main too), pre-commit black and isort pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
No textual conflicts: main's #202 (record ids as a :rid parameter, src/arcadedb/_native/records.py and database.py) and #201 (run_in_transaction witness) touch other hunks than this branch. The record-id rule of #202 is kept: this branch pastes no record id into SQL text (the lazy per-hit vector load, if present, goes through lookup_by_rid, which keeps its literal id by design, see tests/test_native_statement_texts.py). Static checks after the merge: compile() and pyflakes of the changed files (the unused typing.Optional in records.py is on main too), pre-commit black and isort pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Port of humemai/arcadedb-embedded-python#529 (branch
perf/native-rid-param, closed there when the bindings moved, never merged) onto current main, narrowed after a first timed A/B. Refs #19. Opened as a draft: a timed A/B decides it before merge.On the native build a record id was pasted into the SQL text, so each traversal step was a new statement text. The engine's parsed-statement cache is keyed by the statement text (
arcadedb.sqlStatementCache, default 300) and its literal parameterizer keeps every literal in aFROMtarget, soSELECT FROM #1:0andSELECT FROM #1:1are two entries; a loop over more than 300 records evicts the statements an application re-runs. In this PR the id goes over as a parameter (theRIDthe codec already tags asR) forVertex.get_out_edges/get_in_edges/get_both_edges,Vertex.new_edge,Edge.get_in/get_out,save()(UPDATE :rid MERGE :row) anddelete()(DELETE FROM :rid).NativeDatabase.lookup_by_ridkeeps main's literal text (see the A/B below). Python only, no image rebuild, and_native/database.pyis identical to main. One paragraph on this is in theRecord ids in rowssection ofhow-to/databases-and-queries.adoc.First timed A/B (
tools/ab.py, laptop, cores 6 and 7, main's native wheel against the same wheel with this PR's first head185ef1f26bswapped in, 12 clean pairs per figure, the A/A control clean, guards flat). Ratios are PR over main, below 1 is faster:lookup_by_ridget_out_edges('Knows')Edge.get_in()lookup_by_rid(...).modify(),set,save()The pre-written rule dropped the PR because the
lookup_by_ridheadline got slower. This head therefore narrows the change to the paths that got faster and putslookup_by_ridback to main's text.new_edgeanddelete()have no timed figure of their own; they follow the same pattern as the paths that were timed and are kept for that reason. A second run on the narrowed head, with its rule written before it, decides whether this leaves draft.Untimed counts (25 records,
tests/test_native_statement_texts.py, native only and not collected on JPype):lookup_by_ridstill one text per record, as on main;get_out_edges("Knows"),get_in_edges()andget_both_edges()over the 24 linked vertices 72 calls and now 3 texts;Edge.get_in/get_outover 24 edges 48 calls and now 2 texts;save()anddelete()one text each;new_edgetwo texts (with and withoutCONTENT) per label instead of one per pair. The behaviour tests added totests/test_graph_api.py(traversal over a 12-vertex chain, save and delete by id,new_edgewith and without properties, the refusals oflookup_by_rid) run on both builds.The statement-cache finding was read from the engine source at 26f0bf889b (26.10.x); main now links a newer engine, so the text-count test is also the check that the engine still keeps a record id in a
FROMtarget in the cache key. Draft PRs skip thetest-native.ymllegs, so the PR's wheel and the parity run come fromgh workflow run test-native.yml --ref perf/native-rid-param. Checks run locally on the changed files: compile, pyflakes (only the findings main already has). The tests have not been run on this port; the parity run and the JVM legs are the first run.🤖 Generated with Claude Code