Skip to content

Native: bulk-copy reply text out of the image (draft, measurement pending) - #520

Closed
tae898 wants to merge 2 commits into
native-previewfrom
native-result-copy
Closed

tae898 wants to merge 2 commits into
native-previewfrom
native-result-copy

Conversation

@tae898

@tae898 tae898 commented Oct 25, 2026 •

Copy link
Copy Markdown

DRAFT. Do not merge until the image build, the byte-identity digests and the before/after clocks below are filled in.

ArcadeNative.out(String) (every JSON reply) and the float32-tail reply of arcadedb_n_exec (flag 4) copied the reply into the malloc'd block one byte per Java loop turn (p.write(i, b[i]), then a loop for the zero padding and one for the floats). Both now fill a direct ByteBuffer view of the block (CTypeConversion.asByteBuffer(p, n).put(b, 0, n), Unsafe.copyMemory). The bytes written are the same.

New test bindings/python/tests/test_native_result_transport.py: replies of one row, of several batches (first reply 256 rows with vectors in the float tail, then 2048-row arcadedb_n_next batches), multi-byte text (2-, 3-, 4-byte UTF-8, quotes, control characters), rows of about 1.2 MB, empty results. It passes against the current (old-loop) library and must pass against the new image.

Why this was looked at: the question was what share of the native wheel's warm query time is JSON text. Study with probe, rule and tables: .notes/bench/repros/native-result-transport-20261025/README.md (notes repo, local).

Untimed, indicative numbers (not ab.py, no night lock; unmodified native wheel, 100k documents with a 32-float vector, microseconds per warm db.query(...).to_list()):

query wall engine in-image reply (encode + getBytes + copy) Python JSON decode
point lookup by id, record row 36 10 (27%) 7.5 (21%) 9.0 (25%)
point lookup, scalar columns 29 12 (41%) 3.3 (11%) 5.3 (18%)
2-hop Cypher, 100 rows 367 324 (88%) small 54 (15%)
aggregation, 10 rows 50,887 100% 0% 0%
wide SELECT 10k rows, 6.9 MB 158,614 54,950 (35%) 21,534 (14%) 78,534 (50%)
wide SELECT 100k rows, 68 MB 1,676,369 511,536 (31%) 240,638 (14%) 886,790 (53%)

The copy loop is a part of the "in-image reply" column and cannot be split out of the unmodified image. The instrumented image (this commit plus an overlay that adds out_timers_ns to arcadedb_n_diag and a run-time switch back to the old loop) is queued behind other jobs on the laptop's 20 GB build budget; its old-loop vs bulk-copy clocks and the sha256 of the reply content (text + float tail, old wheel vs old loop vs bulk) will be added here. The larger finding is on the Python side: json.loads with the object_hook is 50% of the wide queries, because only the first 256 rows use the float32 tail and arcadedb_n_next replies carry embeddings as number text. That is a separate change and is not in this PR.

ArcadeLib.java has a third copy of the same byte loop (toC); it is not part of the preview image, so it is left alone.

Instrumented build result (2026-10-25, untimed)

With out_timers_ns in an instrumented image of this branch, the old malloc + per-byte copy is 0-2% of warm wall in every query class (point lookups 0.1-0.35 us of 42-57 us; wide10k_proj 3.7 ms of 235 ms; wide100k_proj 33 ms of 2.53 s). So this PR is a cleanup with a small gain, not the fix for native read speed: the cost is JSON encode (8-36%) plus Python decode (18-53%). Full tables: notes folder native-result-transport-20261025 (verdict section). #521 (stacked on this one) carries the measured win: every result batch gets the binary float tail, wide reads about 2x faster, rows bit-identical to the JVM wheel.

🤖 Generated with Claude Code

tae898 and others added 2 commits October 25, 2026 15:32
…op turn

out() and the float32-tail reply wrote the reply with p.write(i, b[i]) in a Java loop. Both now fill a direct ByteBuffer view of the
malloc'd block (Unsafe.copyMemory); the bytes written are the same. New test: replies of one, several and many batches, multi-byte text,
megabyte rows and vectors round-trip exactly.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The previous commit widened off + 4 * sink.n to long, which UnmanagedMemory.malloc does not accept (found by the first image build).

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
tae898 added a commit to ArcadeData/arcadedb-python that referenced this pull request Oct 25, 2026
…bulk

From humemai/arcadedb-embedded-python#520. out() and the tail block of withFloatTail wrote the reply one byte (and one float) at a time with p.write / fp.write. They now use CTypeConversion.asByteBuffer(...).put(...) (Unsafe.copyMemory): the text, the zero padding up to the 4-byte boundary, then the float32 tail in native order. This is the state that both arms of the A/B of arcadedb_n_next_bin ran. tests/test_native_result_transport.py gets the round-trip tests of #520 (several batches, a small reply with a vector, rows of about a megabyte of multi-byte text, empty and one-row replies) next to the vector tests.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@tae898

tae898 commented Oct 26, 2026

Copy link
Copy Markdown
Author

Closing here: the bindings moved to ArcadeData/arcadedb-python, and this change is merged there as ArcadeData/arcadedb-python#76 (it carries the bulk copy of reply text together with the float32 tail). The branch stays.

@tae898 tae898 closed this Oct 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant