diff --git a/README.md b/README.md index 3d7bb2546..6e38385b8 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,53 @@ Introductory tutorials for JVector are available in [docs/tutorials](./docs/tuto The older step-by-step guide for JV can be found [here](./docs/legacy/jvector-step-by-step.md). New users should start with the tutorials mentioned earler, but the step-by-step guide contains useful commentary for advanced users. +## SIMD provider selection + +On Java 22+ with the Vector API enabled, JVector automatically prefers its Google +Highway native provider when the native library is available. Native libraries are +bundled for Linux x86-64 (SSE4.2 or newer) and Linux AArch64. Use these JVM options: + +```text +--add-modules jdk.incubator.vector +--enable-native-access=ALL-UNNAMED +``` + +Omitting native access can produce restricted-method warnings. If native access is +denied by the runtime or JVM options, automatic selection falls back to Panama; +an explicitly forced native provider fails during initialization. For named-module +deployments, enable native access for the module containing JVector's native code. + +Operations implemented by Highway use native kernels; remaining operations use the +inherited Panama/Java Vector API implementation with native vector storage. Native +ISA selection remains automatic and respects the existing `JVECTOR_MAX_ISA` cap. + +| JVM property | Effect | +| --- | --- | +| No provider property | Prefer native on Java 22+; use Panama when native is unavailable | +| `-Djvector.enable_native_vectorization=false` | Disable automatic native selection | +| `-Djvector.vectorization_provider=panama` | Explicitly select Panama | +| `-Djvector.vectorization_provider=native` | Explicitly require native; fail if it cannot load | +| `-Djvector.vectorization_provider=default` | Explicitly select the scalar provider | + +The explicit `jvector.vectorization_provider` property takes precedence over the +native-enable property. The former +`jvector.experimental.enable_native_vectorization` name is a deprecated compatibility +alias; the new `jvector.enable_native_vectorization` name takes precedence when both +are supplied. Both names default to `true` when neither is supplied. Provider +selection happens once per JVM. Java 20/21 uses +Panama when available. Older runtimes, or runtimes without the Vector API module, +retain the scalar fallback. Neither Panama nor native is enabled by these defaults +when the existing runtime compatibility checks reject SIMD support. + +Native selection also changes the concrete vector type to `MemorySegmentVectorFloat`. +Use the `VectorFloat` interface instead of casting to `ArrayVectorFloat` or assuming +that `VectorFloat.get()` returns a `float[]`. Both built-in float vector types support +`FloatArray.array()` for array access and `copyFrom` between their storage types. +Native vector operations still require vectors created by the selected provider; +copy external vectors into provider-created vectors before passing them to those +operations. To retain the previous provider and storage choice, disable automatic +native selection with `-Djvector.enable_native_vectorization=false`. + ## The research behind the algorithms * Foundational work: [HNSW](https://ieeexplore.ieee.org/abstract/document/8594636) and [DiskANN](https://suhasjs.github.io/files/diskann_neurips19.pdf) papers, and [a higher level explainer](https://www.datastax.com/guides/hierarchical-navigable-small-worlds) @@ -95,6 +142,9 @@ bash build_native_lib.sh --auto-install-deps For other distributions the script will print the install commands needed. See [`jvector-native/src/main/native/README.md`](./jvector-native/src/main/native/README.md) for full build options, Maven integration, ISA dispatch details, and how to add new kernels. +The [native ABI compatibility guide](./jvector-native/src/main/native/README.md#native-abi-compatibility) +explains the supported ABI set, startup checks, and how to preserve compatibility +when changing native entry points. --- diff --git a/docs/release notes/4.0.2/751.feature.md b/docs/release notes/4.0.2/751.feature.md new file mode 100644 index 000000000..71c126510 --- /dev/null +++ b/docs/release notes/4.0.2/751.feature.md @@ -0,0 +1,21 @@ +Google Highway native SIMD is no longer experimental and is preferred by default +on supported Java 22+ runtimes. `jvector.enable_native_vectorization` is the supported +flag; the former name remains a deprecated compatibility alias. + +Successful native selection uses `MemorySegmentVectorFloat` instead of +`ArrayVectorFloat`. Consumers should use `VectorFloat`, or `FloatArray.array()` +when array access is needed; direct casts and assumptions about `get()`'s backing +storage must be updated. `copyFrom` supports both built-in float vector types. +Disable automatic native selection with `-Djvector.enable_native_vectorization=false` +to retain the previous provider choice. Native operations require provider-created +vectors; copying between storage types does not make mixed-storage arithmetic valid. + +Pass `--add-modules jdk.incubator.vector --enable-native-access=ALL-UNNAMED` +for classpath deployments. Missing native access can produce JVM warnings; denied +native access, an unsupported ABI, a missing ABI query, or missing required library +symbols cause automatic selection to fall +back to the Java provider. Explicitly forcing native fails during initialization. + +Java requires ABI 1 and checks the native library's explicit supported set before +calling kernels. Compatible native releases keep the same ABI ID. Libraries that +predate the ABI query are rejected; the bundled library supports ABI 1. diff --git a/jvector-base/src/main/java/io/github/jbellis/jvector/vector/ArrayVectorFloat.java b/jvector-base/src/main/java/io/github/jbellis/jvector/vector/ArrayVectorFloat.java index 46c7a59dc..e7e4da857 100644 --- a/jvector-base/src/main/java/io/github/jbellis/jvector/vector/ArrayVectorFloat.java +++ b/jvector-base/src/main/java/io/github/jbellis/jvector/vector/ArrayVectorFloat.java @@ -23,6 +23,7 @@ import java.io.IOException; import java.util.Arrays; +import java.util.Objects; /** * VectorFloat implementation backed by an on-heap float array. @@ -82,8 +83,15 @@ public VectorFloat copy() @Override public void copyFrom(VectorFloat src, int srcOffset, int destOffset, int length) { - ArrayVectorFloat csrc = (ArrayVectorFloat) src; - System.arraycopy(csrc.data, srcOffset, data, destOffset, length); + if (src instanceof FloatArray) { + System.arraycopy(((FloatArray) src).array(), srcOffset, data, destOffset, length); + } else { + Objects.checkFromIndexSize(srcOffset, length, src.length()); + Objects.checkFromIndexSize(destOffset, length, data.length); + for (int i = 0; i < length; i++) { + data[destOffset + i] = src.get(srcOffset + i); + } + } } @Override @@ -130,4 +138,3 @@ public float[] array() { return get(); } } - diff --git a/jvector-base/src/main/java/io/github/jbellis/jvector/vector/VectorizationProvider.java b/jvector-base/src/main/java/io/github/jbellis/jvector/vector/VectorizationProvider.java index c25458b67..c29f092b8 100644 --- a/jvector-base/src/main/java/io/github/jbellis/jvector/vector/VectorizationProvider.java +++ b/jvector-base/src/main/java/io/github/jbellis/jvector/vector/VectorizationProvider.java @@ -120,9 +120,10 @@ static VectorizationProvider lookup(boolean testMode) { return new DefaultVectorizationProvider(); } - // check for jvector.experimental.enable_native_vectorization - // this is an experimental feature subject to change - if (Boolean.getBoolean("jvector.experimental.enable_native_vectorization")) { + // Native uses Java 22 FFM. The new setting takes precedence over its deprecated alias. + if (runtimeVersion >= 22 && Boolean.parseBoolean(System.getProperty( + "jvector.enable_native_vectorization", + System.getProperty("jvector.experimental.enable_native_vectorization", "true")))) { try { Class clazz = Class.forName("io.github.jbellis.jvector.vector.NativeVectorizationProvider"); Constructor ctor = clazz.getConstructor(); @@ -141,7 +142,7 @@ static VectorizationProvider lookup(boolean testMode) { } catch (UnsupportedOperationException uoe) { LOG.warning("Native vector API was not enabled. " + uoe.getMessage()); } catch (ClassNotFoundException e) { - LOG.warning("Java version does not support native vector API"); + LOG.fine("Native vector provider is not on the classpath; trying the Java Vector API."); } catch (InvocationTargetException e) { // unwrap, check for UnsupportedOperationException if (e.getCause() instanceof UnsupportedOperationException) { diff --git a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/MemorySegmentVectorFloat.java b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/MemorySegmentVectorFloat.java index 4227629aa..0cda8cfab 100644 --- a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/MemorySegmentVectorFloat.java +++ b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/MemorySegmentVectorFloat.java @@ -24,6 +24,7 @@ import java.io.IOException; import java.lang.foreign.MemorySegment; import java.nio.Buffer; +import java.util.Objects; /** * VectorFloat implementation backed by an on-heap MemorySegment. @@ -107,9 +108,18 @@ public VectorFloat copy() @Override public void copyFrom(VectorFloat src, int srcOffset, int destOffset, int length) { - MemorySegmentVectorFloat csrc = (MemorySegmentVectorFloat) src; - segment.asSlice((long) destOffset * Float.BYTES, (long) length * Float.BYTES) - .copyFrom(csrc.segment.asSlice((long) srcOffset * Float.BYTES, (long) length * Float.BYTES)); + if (src instanceof MemorySegmentVectorFloat csrc) { + segment.asSlice((long) destOffset * Float.BYTES, (long) length * Float.BYTES) + .copyFrom(csrc.segment.asSlice((long) srcOffset * Float.BYTES, (long) length * Float.BYTES)); + } else if (src instanceof FloatArray csrc) { + System.arraycopy(csrc.array(), srcOffset, array(), destOffset, length); + } else { + Objects.checkFromIndexSize(srcOffset, length, src.length()); + Objects.checkFromIndexSize(destOffset, length, length()); + for (int i = 0; i < length; i++) { + set(destOffset + i, src.get(srcOffset + i)); + } + } } @Override diff --git a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorUtilSupport.java b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorUtilSupport.java index 4b627a244..9230c2f9a 100644 --- a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorUtilSupport.java +++ b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorUtilSupport.java @@ -19,7 +19,6 @@ import java.lang.foreign.MemorySegment; import java.nio.ByteOrder; -import io.github.jbellis.jvector.annotations.Experimental; import io.github.jbellis.jvector.vector.cnative.NativeSimdOps; import io.github.jbellis.jvector.vector.types.ByteSequence; import io.github.jbellis.jvector.vector.types.VectorFloat; @@ -29,10 +28,8 @@ import jdk.incubator.vector.VectorSpecies; /** - * Experimental! - * VectorUtilSupport implementation that prefers native/Panama SIMD. + * VectorUtilSupport using Highway kernels and inherited Panama operations. */ -@Experimental final class NativeVectorUtilSupport extends PanamaVectorUtilSupport { public NativeVectorUtilSupport() {} diff --git a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorizationProvider.java b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorizationProvider.java index 213e53f9a..96aefe243 100644 --- a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorizationProvider.java +++ b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/NativeVectorizationProvider.java @@ -16,19 +16,29 @@ package io.github.jbellis.jvector.vector; -import io.github.jbellis.jvector.annotations.Experimental; import io.github.jbellis.jvector.vector.cnative.LibraryLoader; +import io.github.jbellis.jvector.vector.cnative.NativeSimdOps; import io.github.jbellis.jvector.vector.types.VectorTypeSupport; +import java.lang.foreign.SymbolLookup; +import java.lang.reflect.Method; + /** - * Experimental! - * VectorizationProvider implementation that uses MemorySegment vectors and prefers native/Panama SIMD. + * VectorizationProvider using MemorySegment vectors, Highway kernels, and Panama + * for operations without native implementations. */ -@Experimental public class NativeVectorizationProvider extends VectorizationProvider { + private static final int REQUIRED_ABI = 1; + private final VectorUtilSupport vectorUtilSupport; private final VectorTypeSupport vectorTypeSupport; + /// Load the native library and create the provider. + /// + /// Throws `UnsupportedOperationException` when the architecture is unsupported, the library + /// can't be loaded, the bindings can't initialize (including when native access is denied), + /// or the loaded library doesn't support the required ABI or export every bound function, so automatic + /// selection falls back to the Java Vector API instead of failing at the first kernel call. public NativeVectorizationProvider() { var arch = System.getProperty("os.arch", ""); if (!arch.equals("amd64") && !arch.equals("x86_64") && !arch.equals("aarch64")) { @@ -36,12 +46,51 @@ public NativeVectorizationProvider() { } var libraryLoaded = LibraryLoader.loadJvector(); if (!libraryLoaded) { - throw new UnsupportedOperationException("Failed to load supporting native library."); + String reason = NativeVectorizationProvider.class.getModule().isNativeAccessEnabled() ? "" + : " Native access is not enabled for this module; pass --enable-native-access=ALL-UNNAMED" + + " (or the jvector module name)."; + throw new UnsupportedOperationException("Failed to load supporting native library." + reason); + } + var support = new NativeVectorUtilSupport(); + try { + if (SymbolLookup.loaderLookup().find("jvector_supports_abi").isEmpty()) { + throw new UnsupportedOperationException("The loaded jvector native library does not export " + + "jvector_supports_abi; compatibility with required ABI " + REQUIRED_ABI + " is unknown."); + } + if (NativeSimdOps.jvector_supports_abi(REQUIRED_ABI) != 1) { + throw new UnsupportedOperationException("The loaded jvector native library does not support required ABI " + + REQUIRED_ABI + "; a different libjvector may be on java.library.path."); + } + String missing = firstMissingSymbol(); + if (missing != null) { + throw new UnsupportedOperationException("The loaded jvector native library does not export " + missing + + "; a different libjvector may be on java.library.path."); + } + support.getActiveIsa(); + } catch (IllegalCallerException | LinkageError e) { + throw new UnsupportedOperationException("Native bindings could not initialize.", e); } - this.vectorUtilSupport = new NativeVectorUtilSupport(); + this.vectorUtilSupport = support; this.vectorTypeSupport = new MemorySegmentVectorProvider(); } + /// Returns the first function bound in [NativeSimdOps] (one per generated `name$address()` + /// accessor) that the libraries loaded by this class loader don't export, or `null` when all + /// are present. Uses a symbol lookup only, so no binding is initialized. + private static String firstMissingSymbol() { + var lookup = SymbolLookup.loaderLookup(); + for (Method method : NativeSimdOps.class.getMethods()) { + String name = method.getName(); + if (method.getParameterCount() == 0 && name.endsWith("$address")) { + String symbol = name.substring(0, name.length() - "$address".length()); + if (lookup.find(symbol).isEmpty()) { + return symbol; + } + } + } + return null; + } + @Override public VectorUtilSupport getVectorUtilSupport() { return vectorUtilSupport; @@ -51,4 +100,4 @@ public VectorUtilSupport getVectorUtilSupport() { public VectorTypeSupport getVectorTypeSupport() { return vectorTypeSupport; } -} \ No newline at end of file +} diff --git a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/LibraryLoader.java b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/LibraryLoader.java index 763eddf22..0fceb23b3 100644 --- a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/LibraryLoader.java +++ b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/LibraryLoader.java @@ -17,6 +17,7 @@ package io.github.jbellis.jvector.vector.cnative; import java.io.File; +import java.io.IOException; import java.nio.file.Files; /** @@ -48,12 +49,20 @@ static String resourceNameForArch() { return null; } + /// Load the jvector native library, from `java.library.path` first and then from the bundled + /// classpath resource. + /// + /// Returns `false` when the library is unavailable, native access is denied to this module + /// (`IllegalCallerException`, e.g. with `--illegal-native-access=deny`), or a security policy + /// blocks loading (`SecurityException`). The bundled copy isn't retried after access denial. public static boolean loadJvector() { try { System.loadLibrary("jvector"); return true; } catch (UnsatisfiedLinkError e) { // ignore + } catch (IllegalCallerException | SecurityException e) { + return false; // native access or library loading denied } try { // reinventing the wheel instead of picking up deps, so we'll just use the classloader to load the library @@ -64,18 +73,25 @@ public static boolean loadJvector() { } String baseName = resourceName.substring(1, resourceName.lastIndexOf('.')); // e.g. "libjvector-aarch64" String ext = resourceName.substring(resourceName.lastIndexOf('.')); // e.g. ".so" - File tmpLibFile = File.createTempFile(baseName, ext); - try (var in = LibraryLoader.class.getResourceAsStream(resourceName); - var out = Files.newOutputStream(tmpLibFile.toPath())) { - if (in != null) { - in.transferTo(out); - out.flush(); - } else { + try (var in = LibraryLoader.class.getResourceAsStream(resourceName)) { + if (in == null) { return false; // couldn't find library } + File tmpLibFile = File.createTempFile(baseName, ext); + try { + try (var out = Files.newOutputStream(tmpLibFile.toPath())) { + in.transferTo(out); + } + System.load(tmpLibFile.getAbsolutePath()); + return true; + } finally { + try { + Files.deleteIfExists(tmpLibFile.toPath()); + } catch (IOException | SecurityException e) { + // Best effort: cleanup failure must not invalidate a loaded library. + } + } } - System.load(tmpLibFile.getAbsolutePath()); - return true; } catch (Exception | UnsatisfiedLinkError e) { // ignore } diff --git a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/NativeSimdOps.java b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/NativeSimdOps.java index d822468a6..9fe9a6171 100644 --- a/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/NativeSimdOps.java +++ b/jvector-native/src/main/java/io/github/jbellis/jvector/vector/cnative/NativeSimdOps.java @@ -1149,6 +1149,64 @@ public static int _BITS_STDINT_LEAST_H() { */ public static final OfLong uintmax_t = NativeSimdOps.C_LONG; + private static class jvector_supports_abi { + public static final FunctionDescriptor DESC = FunctionDescriptor.of( + NativeSimdOps.C_INT, + NativeSimdOps.C_INT + ); + + public static final MemorySegment ADDR = NativeSimdOps.findOrThrow("jvector_supports_abi"); + + public static final MethodHandle HANDLE = Linker.nativeLinker().downcallHandle(ADDR, DESC, Linker.Option.critical(true)); + } + + /** + * Function descriptor for: + * {@snippet lang=c : + * int32_t jvector_supports_abi(int32_t abi) + * } + */ + public static FunctionDescriptor jvector_supports_abi$descriptor() { + return jvector_supports_abi.DESC; + } + + /** + * Downcall method handle for: + * {@snippet lang=c : + * int32_t jvector_supports_abi(int32_t abi) + * } + */ + public static MethodHandle jvector_supports_abi$handle() { + return jvector_supports_abi.HANDLE; + } + + /** + * Address for: + * {@snippet lang=c : + * int32_t jvector_supports_abi(int32_t abi) + * } + */ + public static MemorySegment jvector_supports_abi$address() { + return jvector_supports_abi.ADDR; + } + + /** + * {@snippet lang=c : + * int32_t jvector_supports_abi(int32_t abi) + * } + */ + public static int jvector_supports_abi(int abi) { + var mh$ = jvector_supports_abi.HANDLE; + try { + if (TRACE_DOWNCALLS) { + traceDowncall("jvector_supports_abi", abi); + } + return (int)mh$.invokeExact(abi); + } catch (Throwable ex$) { + throw new AssertionError("should not reach here", ex$); + } + } + private static class cosine_f32 { public static final FunctionDescriptor DESC = FunctionDescriptor.of( NativeSimdOps.C_FLOAT, diff --git a/jvector-native/src/main/native/README.md b/jvector-native/src/main/native/README.md index 91e3060ce..121afb2ae 100644 --- a/jvector-native/src/main/native/README.md +++ b/jvector-native/src/main/native/README.md @@ -385,8 +385,11 @@ Java caller system-installed `.so`), then falls back to reading `os.arch` to select the correct resource name (`libjvector-x86_64.so` or `libjvector-aarch64.so`), extracting it from the JAR, and loading it from a temp file. -3. On first call into `NativeSimdOps`, the FFM `SymbolLookup` resolves each - exported symbol directly against the loaded library. +3. The provider checks that `jvector_supports_abi` exists and returns 1 for the + ABI required by the Java bindings. See [Native ABI compatibility](#native-abi-compatibility). +4. The provider checks that all bound symbols exist and calls the ISA accessor + before making the provider available. FFM bindings resolve their function + addresses lazily; no kernel is called before the ABI check succeeds. ### ISA dispatch @@ -743,3 +746,78 @@ isa_libs += my_new_tier_lib - Update the architecture diagram in the **How it is integrated into JVector** section above. - Add the new `JVECTOR_MAX_ISA` value to the **ISA cap** section. + +## Native ABI compatibility + +### Querying support + +The Java provider requires ABI 1. Before resolving kernel bindings it calls this +export from [`src/jvector_simd.h`](src/jvector_simd.h): + +```c +int32_t jvector_supports_abi(int32_t abi); +``` + +The native library stores an explicit set of supported ABI IDs, initially `{1}`, +in [`src/jvector_simd.cpp`](src/jvector_simd.cpp). The query returns exactly 1 for +a supported ID and 0 for any other ID, including nonpositive and unknown IDs. +C and C++ consumers can call the same query before using an ABI's entry points. +The query's name, signature, and return convention are permanent bootstrap +contracts; they must remain unchanged when new ABIs are introduced. + +The Java JAR requires one ABI, rather than a particular native release number. +For example, if a future Java release requires ABI 2: + +| Native supported set | Java requires ABI 1 | Java requires ABI 2 | +| --- | --- | --- | +| `{1}` | Accept | Reject | +| `{1, 2}` | Accept | Accept | +| `{2}` | Reject | Accept | +| Query missing | Reject | Reject | + +Sets need not be contiguous. A library advertising `{1, 3}` must return 0 for +ABI 2. Never infer support for older IDs from the highest supported ID. + +An unsupported ABI or missing query causes automatic Java provider selection to +fall back to Panama. Explicitly forcing native fails during initialization. +Native libraries from before this handshake are rejected, even if their kernels +might otherwise be compatible. Each release bundles a library supporting its +required ABI. A system library found through `java.library.path` is checked too; +if it loads but fails validation, selection falls back to Panama rather than +trying the bundled native copy. + +### Evolving the native library + +ABI IDs describe calling contracts, not JVector release numbers. Optimizations, +bug fixes that preserve the contract, and compatible additions retain the existing +ABI ID. Required-symbol checks still reject older libraries missing functions +that a newer Java release needs, even when the ABI ID is supported. + +An incompatible change to a signature, data layout, or calling convention requires +a new ABI ID. To support both IDs, retain the old contract and expose changed +functions under distinct entry points, such as a new `_v2` name. Never reinterpret +an existing symbol's signature. Java bindings for each ABI must resolve the +corresponding entry points. Advertise an old ID only while its contract is still +implemented; removing it makes Java releases requiring that ID reject the library. + +When changing the native interface: + +1. Update the public declarations and native implementations. Keep the supported + set in `jvector_simd.cpp` accurate. Incompatible changes also require updating + `REQUIRED_ABI` in + [`NativeVectorizationProvider.java`](../java/io/github/jbellis/jvector/vector/NativeVectorizationProvider.java) + and adapting the Java callers to the new entry points. +2. Regenerate the checked-in `NativeSimdOps.java` bindings with jextract 22 when + the public header changes. Follow [Generating Java bindings for native code](#generating-java-bindings-for-native-code). +3. Rebuild and package the native library along with the Java changes. A Java-only + rebuild cannot add new exports to an existing `.so`; build each shipped architecture. +4. Run `TestVectorizationProviderSelection` against the rebuilt library. It covers + the real supported set, noncontiguous test sets, unsupported and missing queries, + missing kernels, automatic fallback, and forced-native startup failure. Update + the fixtures and assertions when the required ABI or bound functions change. +5. For release packaging, run `jvector-native/verify_native_lib.sh` from the + repository root against the cross-architecture JAR. It requires both shipped + libraries and checks their architecture, dependencies, and required exports, + including `jvector_supports_abi`, where the inspection tools are available. + Export inspection complements the runtime tests; it does not establish ABI + support by itself. diff --git a/jvector-native/src/main/native/src/jvector_simd.cpp b/jvector-native/src/main/native/src/jvector_simd.cpp index f12746e64..ec69d5cfa 100644 --- a/jvector-native/src/main/native/src/jvector_simd.cpp +++ b/jvector-native/src/main/native/src/jvector_simd.cpp @@ -249,6 +249,17 @@ static const char *max_isa_env = dispatch.max_isa_env; } // namespace // ---- Public API ------------------------------------------------------------ +int32_t jvector_supports_abi(int32_t abi) +{ + // Advertise only contracts whose entry points and calling conventions are + // still implemented. Compatible releases retain the same ABI ID. + static constexpr std::array supported_abis = {1}; + for (int32_t supported : supported_abis) { + if (abi == supported) return 1; + } + return 0; +} + // Each function is a thin wrapper that forwards to the pre-resolved function // pointer in `kernels`. // Auto-generated from jvector_simd_kernel_list.h diff --git a/jvector-native/src/main/native/src/jvector_simd.h b/jvector-native/src/main/native/src/jvector_simd.h index e3d540662..e1a12f414 100644 --- a/jvector-native/src/main/native/src/jvector_simd.h +++ b/jvector-native/src/main/native/src/jvector_simd.h @@ -31,6 +31,11 @@ extern "C" { #endif +// Stable bootstrap contract: return 1 if this library implements the requested +// ABI, otherwise 0 (including unknown IDs). IDs are positive and never reused. +// Keep this signature unchanged across all future ABIs. +JVECTOR_SIMD_API int32_t jvector_supports_abi(int32_t abi); + #define KERNEL_ENTRY(ret_type, name, params, names) \ JVECTOR_SIMD_API ret_type name params; @@ -56,4 +61,3 @@ JVECTOR_SIMD_API const char *jvector_simd_get_max_isa_env(void); } #endif // extern "C" #endif // VECTOR_SIMD_DOT_H - diff --git a/jvector-native/src/test/java/io/github/jbellis/jvector/vector/MemorySegmentVectorProviderTest.java b/jvector-native/src/test/java/io/github/jbellis/jvector/vector/MemorySegmentVectorProviderTest.java index 5afa4b2bf..539f21cfc 100644 --- a/jvector-native/src/test/java/io/github/jbellis/jvector/vector/MemorySegmentVectorProviderTest.java +++ b/jvector-native/src/test/java/io/github/jbellis/jvector/vector/MemorySegmentVectorProviderTest.java @@ -31,6 +31,36 @@ public class MemorySegmentVectorProviderTest { + @Test + void testCopyBetweenArrayAndMemorySegmentStorage() { + var arraySource = new ArrayVectorFloat(new float[] {1, 2, 3, 4}); + var segmentDest = new MemorySegmentVectorFloat(new float[] {9, 9, 9, 9}); + segmentDest.copyFrom(arraySource, 1, 2, 2); + Assert.assertArrayEquals(new float[] {9, 9, 2, 3}, segmentDest.array(), 0); + + var arrayDest = new ArrayVectorFloat(new float[] {8, 8, 8, 8}); + arrayDest.copyFrom(segmentDest, 2, 1, 2); + Assert.assertArrayEquals(new float[] {8, 2, 3, 8}, arrayDest.array(), 0); + + segmentDest.copyFrom(arraySource, arraySource.length(), segmentDest.length(), 0); + arrayDest.copyFrom(segmentDest, segmentDest.length(), arrayDest.length(), 0); + org.junit.jupiter.api.Assertions.assertThrows(IndexOutOfBoundsException.class, + () -> segmentDest.copyFrom(arraySource, 3, 0, 2)); + org.junit.jupiter.api.Assertions.assertThrows(IndexOutOfBoundsException.class, + () -> arrayDest.copyFrom(segmentDest, 0, 3, 2)); + } + + @Test + void testCrossStorageCopyWithOverlappingBackingArray() { + float[] data = {1, 2, 3, 4}; + var array = new ArrayVectorFloat(data); + var segment = new MemorySegmentVectorFloat(data); + segment.copyFrom(array, 0, 1, 3); + Assert.assertArrayEquals(new float[] {1, 1, 2, 3}, data, 0); + array.copyFrom(segment, 1, 0, 3); + Assert.assertArrayEquals(new float[] {1, 2, 3, 3}, data, 0); + } + @Test void testWriteByteSequenceSlice() throws IOException { MemorySegmentVectorProvider provider = new MemorySegmentVectorProvider(); diff --git a/jvector-native/verify_native_lib.sh b/jvector-native/verify_native_lib.sh index ecf98369b..4b8202fa5 100755 --- a/jvector-native/verify_native_lib.sh +++ b/jvector-native/verify_native_lib.sh @@ -42,6 +42,7 @@ if [ -z "${JAR}" ] || [ ! -f "${JAR}" ]; then fi REQUIRED_SYMBOLS=( + jvector_supports_abi jvector_simd_get_active_isa jvector_simd_get_max_isa_env dot_product_f32 diff --git a/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProvider.java b/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProvider.java index 81a99aafc..b8d0b5c3e 100644 --- a/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProvider.java +++ b/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProvider.java @@ -98,13 +98,13 @@ public void testAssembleAndSum() { * -ea * --add-modules jdk.incubator.vector * --enable-native-access=ALL-UNNAMED - * -Djvector.experimental.enable_native_vectorization=true * } * * To run with panama support, use *
{@code
-     *   --ea
+     *   -ea
      *   --add-modules jdk.incubator.vector
+     *   -Djvector.vectorization_provider=panama
      * }
* * If
{@code -DTest_RequireSpecificVectorizationProvider=}
diff --git a/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProviderSelection.java b/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProviderSelection.java new file mode 100644 index 000000000..ed25566d6 --- /dev/null +++ b/jvector-tests/src/test/java/io/github/jbellis/jvector/vector/TestVectorizationProviderSelection.java @@ -0,0 +1,436 @@ +/* + * Copyright DataStax, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package io.github.jbellis.jvector.vector; + +import org.junit.Assume; +import org.junit.Test; + +import java.io.File; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.security.Permission; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Comparator; +import java.util.concurrent.TimeUnit; +import java.util.stream.Collectors; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertTrue; + +/** Fresh JVMs exercise the real singleton without leaking provider properties between tests. */ +public class TestVectorizationProviderSelection { + private static final String ENABLE_NATIVE = "-Djvector.enable_native_vectorization="; + private static final String LEGACY_ENABLE_NATIVE = "-Djvector.experimental.enable_native_vectorization="; + private static final boolean HAS_VECTOR_MODULE = VectorizationProvider.vectorModulePresentAndReadable(); + + private static String classpath() { + return System.getProperty("surefire.test.class.path", System.getProperty("java.class.path")); + } + + private static String withoutNativeArtifact() { + return Arrays.stream(classpath().split(java.util.regex.Pattern.quote(File.pathSeparator))) + .filter(entry -> !entry.replace('\\', '/').contains("/jvector-native/")) + .filter(entry -> !(Path.of(entry).getFileName().toString().startsWith("jvector-native-") + && entry.endsWith(".jar"))) + .collect(Collectors.joining(File.pathSeparator)); + } + + private static boolean nativeAvailable() throws Exception { + if (Runtime.version().feature() < 22 || !HAS_VECTOR_MODULE + || !System.getProperty("os.name").startsWith("Linux")) return false; + String arch = System.getProperty("os.arch"); + String resource = arch.equals("aarch64") ? "/libjvector-aarch64.so" + : (arch.equals("amd64") || arch.equals("x86_64")) ? "/libjvector-x86_64.so" : null; + if (resource == null) return false; + try { + return Class.forName("io.github.jbellis.jvector.vector.NativeVectorizationProvider", false, + TestVectorizationProviderSelection.class.getClassLoader()).getResource(resource) != null; + } catch (ClassNotFoundException e) { return false; } + } + + private static String javaProvider() { + return HAS_VECTOR_MODULE ? "PanamaVectorizationProvider" : "DefaultVectorizationProvider"; + } + + private static String probe(String cp, boolean vectorModule, boolean success, String... properties) throws Exception { + return probe(cp, vectorModule, true, success, properties); + } + + /// Run [Probe] in a fresh JVM. `nativeAccess` controls whether the child gets + /// `--enable-native-access=ALL-UNNAMED` (Java 22+ only). + private static String probe(String cp, boolean vectorModule, boolean nativeAccess, boolean success, + String... properties) throws Exception { + var tmp = Files.createTempDirectory("provider-selection-"); + Process child = null; + try { + var command = new ArrayList(); + command.add(Path.of(System.getProperty("java.home"), "bin", "java").toString()); + if (vectorModule) command.add("--add-modules=jdk.incubator.vector"); + if (nativeAccess && Runtime.version().feature() >= 22) command.add("--enable-native-access=ALL-UNNAMED"); + command.add("-Djava.io.tmpdir=" + tmp); + command.addAll(Arrays.asList(properties)); + command.add("-cp"); command.add(cp); command.add(Probe.class.getName()); + var process = new ProcessBuilder(command).redirectErrorStream(true).start(); + child = process; + assertTrue("Provider probe timed out", process.waitFor(30, TimeUnit.SECONDS)); + String output = new String(process.getInputStream().readAllBytes(), StandardCharsets.UTF_8); + if (success) assertEquals(output, 0, process.exitValue()); + else assertTrue(output, process.exitValue() != 0); + try (var files = Files.list(tmp)) { + assertFalse("Extracted native library was not removed: " + output, + files.anyMatch(file -> file.getFileName().toString().startsWith("libjvector-"))); + } + return output; + } finally { + if (child != null && child.isAlive()) { child.destroyForcibly(); child.waitFor(); } + try (var files = Files.walk(tmp)) { + for (var file : files.sorted(Comparator.reverseOrder()).collect(Collectors.toList())) + Files.delete(file); + } + } + } + + private static void assertProvider(String expected, String cp, String... properties) throws Exception { + String output = probe(cp, HAS_VECTOR_MODULE, true, properties); + assertTrue(output, output.contains("PROVIDER=" + expected)); + // The native distance and inherited sum must both work with the selected vector storage. + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + } + + @Test public void automaticSelectionPrefersAvailableNative() throws Exception { + assertProvider(nativeAvailable() ? "NativeVectorizationProvider" : javaProvider(), classpath()); + } + + @Test public void explicitFalseDisablesAutomaticNative() throws Exception { + assertProvider(javaProvider(), classpath(), ENABLE_NATIVE + "false"); + } + + @Test public void explicitTrueStillEnablesNative() throws Exception { + assertProvider(nativeAvailable() ? "NativeVectorizationProvider" : javaProvider(), classpath(), ENABLE_NATIVE + "true"); + } + + @Test public void deprecatedFalseAliasStillDisablesNative() throws Exception { + assertProvider(javaProvider(), classpath(), LEGACY_ENABLE_NATIVE + "false"); + } + + @Test public void deprecatedTrueAliasStillEnablesNative() throws Exception { + assertProvider(nativeAvailable() ? "NativeVectorizationProvider" : javaProvider(), classpath(), + LEGACY_ENABLE_NATIVE + "true"); + } + + @Test public void newFalseOverridesDeprecatedTrue() throws Exception { + assertProvider(javaProvider(), classpath(), ENABLE_NATIVE + "false", LEGACY_ENABLE_NATIVE + "true"); + } + + @Test public void newTrueOverridesDeprecatedFalse() throws Exception { + assertProvider(nativeAvailable() ? "NativeVectorizationProvider" : javaProvider(), classpath(), + LEGACY_ENABLE_NATIVE + "false", ENABLE_NATIVE + "true"); + } + + @Test public void explicitScalarProviderTakesPrecedence() throws Exception { + assertProvider("DefaultVectorizationProvider", classpath(), ENABLE_NATIVE + "true", LEGACY_ENABLE_NATIVE + "false", + "-Djvector.vectorization_provider=default"); + } + + @Test public void explicitPanamaProviderTakesPrecedence() throws Exception { + Assume.assumeTrue(HAS_VECTOR_MODULE); + assertProvider("PanamaVectorizationProvider", classpath(), ENABLE_NATIVE + "true", LEGACY_ENABLE_NATIVE + "false", + "-Djvector.vectorization_provider=panama"); + } + + @Test public void explicitNativeProviderTakesPrecedenceOverFalse() throws Exception { + Assume.assumeTrue(nativeAvailable()); + assertProvider("NativeVectorizationProvider", classpath(), ENABLE_NATIVE + "false", LEGACY_ENABLE_NATIVE + "true", + "-Djvector.vectorization_provider=native"); + } + + @Test public void absentNativeArtifactUsesJavaProvider() throws Exception { + assertProvider(javaProvider(), withoutNativeArtifact()); + } + + @Test public void unsupportedArchitectureUsesJavaProvider() throws Exception { + assertProvider(javaProvider(), classpath(), "-Dos.arch=unsupported-provider-test"); + } + + @Test public void explicitNativeFailureDoesNotSilentlyFallBack() throws Exception { + String output = probe(classpath(), HAS_VECTOR_MODULE, false, + "-Djvector.vectorization_provider=native", "-Dos.arch=unsupported-provider-test"); + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + } + + @Test public void absentVectorModuleUsesScalar() throws Exception { + String output = probe(classpath(), false, true); + assertTrue(output, output.contains("PROVIDER=DefaultVectorizationProvider")); + } + + /// Repro for #751: when native access is denied, provider selection fails instead of falling + /// back to the Java provider. + /// + /// ## The bug + /// + /// Automatic selection now constructs `NativeVectorizationProvider` by default on Java 22+. + /// Its constructor calls `LibraryLoader.loadJvector()`, whose first `try` wraps + /// `System.loadLibrary("jvector")` and catches only `UnsatisfiedLinkError`. When native + /// access is denied (`--illegal-native-access=deny` on Java 24+, and the announced future + /// default), `loadLibrary` throws `IllegalCallerException`, which escapes the loader. The + /// constructor's `InvocationTargetException` then has a cause other than + /// `UnsupportedOperationException`, so `VectorizationProvider.lookup` throws + /// `AssertionError`. The provider holder fails to initialize, and no vector operation in the + /// JVM can run. Before #751 native was opt-in, so the same JVM used Panama, and still does + /// with `-Djvector.enable_native_vectorization=false`. + /// + /// ## The test + /// + /// Runs the probe without `--enable-native-access` and with `--illegal-native-access=deny`, + /// and expects the Java provider to be selected and to compute correctly. At `599478a1` + /// the child exits with `AssertionError` caused by `IllegalCallerException`. + @Test public void deniedNativeAccessFallsBackToJavaProvider() throws Exception { + Assume.assumeTrue("--illegal-native-access needs Java 24+", Runtime.version().feature() >= 24); + Assume.assumeTrue(nativeAvailable()); + String output = probe(classpath(), HAS_VECTOR_MODULE, false, true, "--illegal-native-access=deny"); + assertTrue(output, output.contains("PROVIDER=" + javaProvider())); + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + } + + @Test public void deniedNativeAccessFailsDuringForcedInitialization() throws Exception { + Assume.assumeTrue("--illegal-native-access needs Java 24+", Runtime.version().feature() >= 24); + Assume.assumeTrue(nativeAvailable()); + String output = probe(classpath(), HAS_VECTOR_MODULE, false, false, + "--illegal-native-access=deny", "-Djvector.vectorization_provider=native"); + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + assertFalse(output, output.contains("PROVIDER=")); + } + + // Java 22/23 rejects FFM access when native access is enabled only for another module. + @Test public void nativeAccessForWrongModuleFallsBack() throws Exception { + Assume.assumeTrue(Runtime.version().feature() == 22 || Runtime.version().feature() == 23); + Assume.assumeTrue(nativeAvailable()); + String output = probe(classpath(), HAS_VECTOR_MODULE, false, true, "--enable-native-access=java.base"); + assertTrue(output, output.contains("PROVIDER=" + javaProvider())); + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + } + + @Test public void nativeAccessForWrongModuleFailsDuringForcedInitialization() throws Exception { + Assume.assumeTrue(Runtime.version().feature() == 22 || Runtime.version().feature() == 23); + Assume.assumeTrue(nativeAvailable()); + String output = probe(classpath(), HAS_VECTOR_MODULE, false, false, + "--enable-native-access=java.base", "-Djvector.vectorization_provider=native"); + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + assertFalse(output, output.contains("PROVIDER=")); + } + + @Test public void securityPolicyDenialFallsBackUnlessNativeIsForced() throws Exception { + Assume.assumeTrue("SecurityManager is disabled in Java 24+", Runtime.version().feature() < 24); + Assume.assumeTrue(nativeAvailable()); + String output = probe(classpath(), HAS_VECTOR_MODULE, true, + "-Djava.security.manager=allow", "-Dprovider.test.deny_library_loading=true"); + assertTrue(output, output.contains("TEST_LIBRARY_LOADING_DENIED")); + assertTrue(output, output.contains("PROVIDER=" + javaProvider())); + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + output = probe(classpath(), HAS_VECTOR_MODULE, false, + "-Djava.security.manager=allow", "-Dprovider.test.deny_library_loading=true", + "-Djvector.vectorization_provider=native"); + assertTrue(output, output.contains("TEST_LIBRARY_LOADING_DENIED")); + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + assertFalse(output, output.contains("PROVIDER=")); + } + + /// Repro for #751: a `libjvector` without jvector's kernels is selected, and fails at first use. + /// + /// ## The bug + /// + /// `LibraryLoader.loadJvector()` first tries `System.loadLibrary("jvector")`, so any + /// `libjvector.so` on `java.library.path` (an older jvector build, or an unrelated library + /// with that name) wins over the bundled one. Nothing then checks that the loaded library + /// exports the kernels. The jextract bindings resolve each symbol lazily, in a per-function + /// holder class. Selection calls only `jvector_simd_get_active_isa` through reflection, and + /// `VectorizationProvider.lookup` swallows that failure as "Unable to determine Highway ISA + /// tier" and returns the native provider. The first kernel call, such as `dotProduct` on the + /// query path, then throws `UnsatisfiedLinkError: unresolved symbol`, and every later call to + /// that kernel fails. With native selected by default, this now reaches every Java 22+ JVM + /// whose library path has such a file, not only those that opted in. + /// + /// ## The test + /// + /// Copies the JDK's own `libsyslookup.so`, a valid shared library with none of jvector's + /// symbols, to a temporary directory as `libjvector.so`, puts that directory on the probe's + /// `java.library.path`, and expects automatic selection to fall back to the Java provider and + /// compute correctly. At `599478a1` the probe selects `NativeVectorizationProvider` and exits + /// with `UnsatisfiedLinkError: unresolved symbol: dot_product_f32`. + @Test public void libraryWithoutKernelsFallsBackToJavaProvider() throws Exception { + Assume.assumeTrue(nativeAvailable()); + Path foreign = Path.of(System.getProperty("java.home"), "lib", "libsyslookup.so"); + Assume.assumeTrue("needs the JDK's libsyslookup.so", Files.isRegularFile(foreign)); + var libraryPath = Files.createTempDirectory("provider-selection-libpath-"); + var stale = libraryPath.resolve("libjvector.so"); + try { + Files.copy(foreign, stale); + assertProvider(javaProvider(), classpath(), "-Djava.library.path=" + libraryPath); + String output = probe(classpath(), HAS_VECTOR_MODULE, false, + "-Djava.library.path=" + libraryPath, "-Djvector.vectorization_provider=native"); + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + assertTrue(output, output.contains("does not export")); + assertFalse(output, output.contains("PROVIDER=")); + } finally { + Files.deleteIfExists(stale); + Files.delete(libraryPath); + } + } + + @Test public void libraryWithIsaButMissingLaterKernelIsRejected() throws Exception { + Assume.assumeTrue(nativeAvailable()); + var libraryPath = Files.createTempDirectory("provider-selection-partial-"); + try { + compileSelectionLibrary(libraryPath, "missing-kernel.c"); + String output = probe(classpath(), HAS_VECTOR_MODULE, true, "-Djava.library.path=" + libraryPath); + assertTrue(output, output.contains("PROVIDER=" + javaProvider())); + assertTrue(output, output.contains("does not export nvq_loss")); + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + output = probe(classpath(), HAS_VECTOR_MODULE, false, + "-Djava.library.path=" + libraryPath, "-Djvector.vectorization_provider=native"); + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + assertTrue(output, output.contains("does not export nvq_loss")); + assertFalse(output, output.contains("PROVIDER=")); + } finally { + deleteSelectionLibrary(libraryPath); + } + } + + @Test public void nativeLibraryReportsInitialAbiSet() throws Exception { + Assume.assumeTrue(nativeAvailable()); + String output = probe(classpath(), HAS_VECTOR_MODULE, true, "-Dprovider.test.query_abis=true"); + assertTrue(output, output.contains("PROVIDER=NativeVectorizationProvider")); + assertAbiQueries(output, 1); + } + + @Test public void librarySupportingRequiredAbiAmongOtherAbisIsAccepted() throws Exception { + assertAbiLibrary(null, "-DABI_FIRST=1", "-DABI_SECOND=3"); + } + + @Test public void librarySupportingOnlyNewerAbisIsRejected() throws Exception { + assertAbiLibrary("does not support required ABI 1", "-DABI_FIRST=2", "-DABI_SECOND=3"); + } + + @Test public void librarySupportingNoAbisIsRejected() throws Exception { + assertAbiLibrary("does not support required ABI 1", "-DABI_FIRST=0", "-DABI_SECOND=0"); + } + + @Test public void libraryWithoutAbiQueryIsRejected() throws Exception { + assertAbiLibrary("does not export jvector_supports_abi", "-DOMIT_ABI_QUERY"); + } + + private static void assertAbiLibrary(String rejection, String... definitions) throws Exception { + Assume.assumeTrue(nativeAvailable()); + var libraryPath = Files.createTempDirectory("provider-selection-abi-"); + try { + compileSelectionLibrary(libraryPath, "abi-library.c", definitions); + String pathOption = "-Djava.library.path=" + libraryPath; + String output = probe(classpath(), HAS_VECTOR_MODULE, true, pathOption, + "-Dprovider.test.query_abis=" + (rejection == null)); + assertTrue(output, output.contains("PROVIDER=" + + (rejection == null ? "NativeVectorizationProvider" : javaProvider()))); + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + if (rejection == null) assertAbiQueries(output, 1, 3); + else assertTrue(output, output.contains(rejection)); + + output = probe(classpath(), HAS_VECTOR_MODULE, rejection == null, pathOption, + "-Djvector.vectorization_provider=native"); + if (rejection == null) { + assertTrue(output, output.contains("PROVIDER=NativeVectorizationProvider")); + assertTrue(output, output.contains("DOT=14.0 SUM=6.0")); + } else { + assertTrue(output, output.contains("Failed to load forced NativeVectorizationProvider")); + assertTrue(output, output.contains(rejection)); + assertFalse(output, output.contains("PROVIDER=")); + } + } finally { + deleteSelectionLibrary(libraryPath); + } + } + + private static void assertAbiQueries(String output, int... supported) { + for (int abi : new int[] {Integer.MIN_VALUE, -1, 0, 1, 2, 3, Integer.MAX_VALUE}) { + int expected = Arrays.stream(supported).anyMatch(value -> value == abi) ? 1 : 0; + assertTrue(output, output.contains("ABI[" + abi + "]=" + expected)); + } + } + + private static void compileSelectionLibrary(Path directory, String fixture, String... definitions) throws Exception { + var source = directory.resolve(fixture); + try (var in = TestVectorizationProviderSelection.class.getResourceAsStream("/provider-selection/" + fixture)) { + assertTrue("Missing native selection fixture: " + fixture, in != null); + Files.copy(in, source); + } + var command = new ArrayList(); + command.add("cc"); command.add("-shared"); command.add("-fPIC"); + command.addAll(Arrays.asList(definitions)); + command.add(source.toString()); command.add("-o"); + command.add(directory.resolve("libjvector.so").toString()); + var log = directory.resolve("compiler.log"); + Process compiler; + try { + compiler = new ProcessBuilder(command).redirectErrorStream(true).redirectOutput(log.toFile()).start(); + } catch (java.io.IOException e) { + Assume.assumeNoException("Needs a C compiler", e); + return; + } + try { + assertTrue("Fixture compilation timed out", compiler.waitFor(30, TimeUnit.SECONDS)); + assertEquals(Files.readString(log), 0, compiler.exitValue()); + } finally { + if (compiler.isAlive()) { compiler.destroyForcibly(); compiler.waitFor(); } + } + } + + private static void deleteSelectionLibrary(Path directory) throws Exception { + try (var files = Files.walk(directory)) { + for (var file : files.sorted(Comparator.reverseOrder()).collect(Collectors.toList())) + Files.delete(file); + } + } + + public static class Probe { + @SuppressWarnings("removal") + public static void main(String[] args) throws ReflectiveOperationException { + if (Boolean.getBoolean("provider.test.deny_library_loading")) { + System.setSecurityManager(new SecurityManager() { + @Override public void checkPermission(Permission permission) {} + @Override public void checkLink(String library) { + if (library.equals("jvector")) { + System.err.println("TEST_LIBRARY_LOADING_DENIED"); + throw new SecurityException("Native library loading denied by test policy"); + } + } + }); + } + var provider = VectorizationProvider.getInstance(); + var vector = provider.getVectorTypeSupport().createFloatVector(new float[] {1f, 2f, 3f}); + var support = provider.getVectorUtilSupport(); + System.out.println("PROVIDER=" + provider.getClass().getSimpleName()); + System.out.println("DOT=" + support.dotProduct(vector, vector) + " SUM=" + support.sum(vector)); + if (Boolean.getBoolean("provider.test.query_abis")) { + var query = Class.forName("io.github.jbellis.jvector.vector.cnative.NativeSimdOps") + .getMethod("jvector_supports_abi", int.class); + for (int abi : new int[] {Integer.MIN_VALUE, -1, 0, 1, 2, 3, Integer.MAX_VALUE}) + System.out.println("ABI[" + abi + "]=" + query.invoke(null, abi)); + } + } + } +} diff --git a/jvector-tests/src/test/resources/provider-selection/abi-library.c b/jvector-tests/src/test/resources/provider-selection/abi-library.c new file mode 100644 index 000000000..653c1a6a8 --- /dev/null +++ b/jvector-tests/src/test/resources/provider-selection/abi-library.c @@ -0,0 +1,68 @@ +/* + * Copyright DataStax, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// Complete symbol table for selection probes; dot product and sum are exercised. +// Compile with ABI_FIRST/ABI_SECOND to simulate a supported set, or OMIT_ABI_QUERY. +#include +#include +#ifndef OMIT_ABI_QUERY +#ifndef ABI_FIRST +#define ABI_FIRST 1 +#endif +#ifndef ABI_SECOND +#define ABI_SECOND 3 +#endif +int32_t jvector_supports_abi(int32_t abi) { + return abi > 0 && (abi == ABI_FIRST || abi == ABI_SECOND); +} +#endif +const char *jvector_simd_get_active_isa(void) { return "selection-test"; } +const char *jvector_simd_get_max_isa_env(void) { return NULL; } +float dot_product_f32(const float *a, size_t ao, const float *b, size_t bo, size_t n) { + float sum = 0; + for (size_t i = 0; i < n; i++) sum += a[ao + i] * b[bo + i]; + return sum; +} +void cosine_f32(void) {} +void euclidean_f32(void) {} +void add_in_place_f32(void) {} +void add_scalar_in_place_f32(void) {} +void sub_in_place_f32(void) {} +void sub_scalar_in_place_f32(void) {} +void sub_f32(void) {} +void sub_scalar_f32(void) {} +void scale_f32(void) {} +void max_f32(void) {} +void min_f32(void) {} +void min_in_place_f32(void) {} +float sum_f32(const float *v, size_t n) { + float sum = 0; + for (size_t i = 0; i < n; i++) sum += v[i]; + return sum; +} +void assemble_and_sum_f32(void) {} +void assemble_and_sum_pq_f32(void) {} +void pq_decoded_cosine_similarity_f32(void) {} +void calculate_partial_sums_dot_f32(void) {} +void calculate_partial_sums_euclidean_f32(void) {} +void calculate_partial_sums_self_magnitude_f32(void) {} +void nvq_quantize_8bit(void) {} +void nvq_uniform_loss(void) {} +void nvq_square_l2_distance_8bit(void) {} +void nvq_dot_product_8bit(void) {} +void nvq_cosine_8bit_packed(void) {} +void nvq_shuffle_query_in_place_8bit(void) {} +void nvq_loss(void) {} diff --git a/jvector-tests/src/test/resources/provider-selection/missing-kernel.c b/jvector-tests/src/test/resources/provider-selection/missing-kernel.c new file mode 100644 index 000000000..50fe9b097 --- /dev/null +++ b/jvector-tests/src/test/resources/provider-selection/missing-kernel.c @@ -0,0 +1,57 @@ +/* + * Copyright DataStax, Inc. + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +// Selection must reject this library before any placeholder kernel is called. +// It exports both ISA accessors and dot product, but deliberately omits nvq_loss. +#include +#include +int32_t jvector_supports_abi(int32_t abi) { return abi == 1; } +const char *jvector_simd_get_active_isa(void) { return "selection-test"; } +const char *jvector_simd_get_max_isa_env(void) { return NULL; } +float dot_product_f32(const float *a, size_t ao, const float *b, size_t bo, size_t n) { + float sum = 0; + for (size_t i = 0; i < n; i++) sum += a[ao + i] * b[bo + i]; + return sum; +} +void cosine_f32(void) {} +void euclidean_f32(void) {} +void add_in_place_f32(void) {} +void add_scalar_in_place_f32(void) {} +void sub_in_place_f32(void) {} +void sub_scalar_in_place_f32(void) {} +void sub_f32(void) {} +void sub_scalar_f32(void) {} +void scale_f32(void) {} +void max_f32(void) {} +void min_f32(void) {} +void min_in_place_f32(void) {} +float sum_f32(const float *v, size_t n) { + float sum = 0; + for (size_t i = 0; i < n; i++) sum += v[i]; + return sum; +} +void assemble_and_sum_f32(void) {} +void assemble_and_sum_pq_f32(void) {} +void pq_decoded_cosine_similarity_f32(void) {} +void calculate_partial_sums_dot_f32(void) {} +void calculate_partial_sums_euclidean_f32(void) {} +void calculate_partial_sums_self_magnitude_f32(void) {} +void nvq_quantize_8bit(void) {} +void nvq_uniform_loss(void) {} +void nvq_square_l2_distance_8bit(void) {} +void nvq_dot_product_8bit(void) {} +void nvq_cosine_8bit_packed(void) {} +void nvq_shuffle_query_in_place_8bit(void) {}