Skip to content

Prefer Google Highway native SIMD by default - #751

Merged
MarkWolters merged 14 commits into
mainfrom
native-by-default
Oct 15, 2026
Merged

MarkWolters merged 14 commits into
mainfrom
native-by-default

Conversation

@tlwillke

@tlwillke tlwillke commented Oct 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Prefer Google Highway native SIMD by default on supported Linux x86-64/AArch64 systems running Java 22+ with the Vector API enabled. Existing Java fallbacks remain available. Closes #750.

Compatibility

  • Opt out with -Djvector.enable_native_vectorization=false. The deprecated experimental alias remains supported; the new property wins. Explicit provider selection takes precedence over both.
  • Before selecting native, require ABI 1 in the library's supported set, all bound symbols, and a successful ISA accessor call. Unavailable or incompatible libraries and denied native access fall back to Panama; explicitly forcing native fails at startup.
  • Native selection uses MemorySegmentVectorFloat. Use VectorFloat and FloatArray.array() instead of concrete array casts. copyFrom supports both built-in storage types; native arithmetic still requires provider-created vectors.
  • Includes Repros 751 #754's repros and fixes, security-policy handling, best-effort temporary-library cleanup, and user/developer documentation.

Kernel implementations and ISA dispatch are unchanged by this PR. #753's additional kernels are already merged into main.

Validation

All four checks passed on the remote Linux x86-64 server with c0095f06 combined with main 8c33f97a. Native tests used the rebuilt library.

Check Result
Java 22 provider, numeric, and storage tests 33 passed, 2 skipped
Java 25 full native reactor tests 465 passed, 5 skipped
Java 17 / Java 11-compatible profile tests 13 passed, 16 skipped
Java 25 packaging, Javadoc, and license checks Passed

Performance

A separate study compared integrated Highway improvements (4e7d397c) with the Panama candidate from #755 on an Intel Xeon 6985P-C. Highway had higher observed QPS in all 32 comparisons across four million-vector datasets, JDK 22/25, 1/8 query threads, and overquery factors 1/2.

Dataset JDK 22 QPS gain JDK 25 QPS gain
MPNet +51–84% +38–54%
Ada002 +49–91% +14–60%
Cohere +37–105% +19–46%
OpenAI3072 +17–49% +6–46%

Ranges cover the four thread/overquery combinations. Total build-process time decreased by approximately 4–36% in seven of eight cases; Cohere/JDK 22 increased by 8%.

The study includes Highway improvements outside #751 and does not isolate the default switch. Graphs/codebooks were built independently, recall was not matched, and timing variation remains.

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Before you submit for review:

  • Does your PR follow guidelines from CONTRIBUTIONS.md?
  • Did you summarize what this PR does clearly and concisely?
  • Did you include performance data for changes which may be performance impacting?
  • Did you include useful docs for any user-facing changes or features?
  • Did you include useful javadocs for developer oriented changes, explaining new concepts or key changes?
  • Did you rebase your branch onto the latest main for regression testing and PR submission?
  • Did you trigger regression testing via Run Bench Main and review results?
  • Did you adhere to the code formatting guidelines (TBD)
  • Did you group your changes for easy review, providing meaningful descriptions for each commit?
  • Did you ensure that all files contain the correct copyright header?
  • Did you add documentation for this feature to the release notes directory?

If you did not complete any of these, then please explain below.

@tlwillke tlwillke self-assigned this Oct 12, 2026
@tlwillke
tlwillke requested a review from r-devulap October 12, 2026 22:50
@tlwillke
tlwillke marked this pull request as ready for review October 12, 2026 22:50
@tlwillke tlwillke added the 4.0.2 Release label label Oct 12, 2026

@r-devulap r-devulap left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. FYI, There are 5 kernels (hamming distance, min, max, sub and scale) that are currently in Panama but not in native, we should port them to native for the sake of completeness even though they never showed up as hotspots. I will have a PR out for that shortly.

Comment thread README.md
@r-devulap

Copy link
Copy Markdown
Contributor

See #753

@MarkWolters MarkWolters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Willing to consider approval but at the least the exception handling strategy for failure to load the native library needs to be discussed.

@jshook jshook mentioned this pull request Oct 14, 2026
@tlwillke tlwillke mentioned this pull request Oct 14, 2026
@tlwillke
tlwillke requested a review from MarkWolters October 15, 2026 00:31

@MarkWolters MarkWolters left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks for adding the check in the constructor

@jshook jshook left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reapproval after fixes

@MarkWolters
MarkWolters merged commit eba7bd4 into main Oct 15, 2026
18 of 19 checks passed
@MarkWolters
MarkWolters deleted the native-by-default branch October 15, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

4.0.2 Release label

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Prefer Google Highway native SIMD by default

4 participants