Skip to content

Native nightly red on macOS and Windows: diag() picks jvector's provider before the engine does #155

Closed

Description

@tae898

The nightly native run on main cd6f2e3 (run 41165481827, 2026-10-31) fails one test on macos-arm64 and windows-x64, Python 3.12 and 3.14: test_native_vector_provider.py::test_the_default_provider_follows_the_shipped_library expects DefaultVectorizationProvider and gets PanamaVectorizationProvider. Pull requests build only linux-x86_64, so #154 merged green.

Cause. ArcadeNative.diag reads the provider with VectorizationProvider.getInstance(). jvector resolves its provider once per process, on first use. The engine forces the scalar provider in a native image through VectorApiSupport.configureJVectorProvider() (ArcadeData/arcadedb#10077), which runs when LocalDatabase loads or through VectorApiSupport.jvectorProvider(). The test's child calls diag() before any database exists, so jvector chooses on its own, and on macOS and Windows its choice is Panama. In a native image Panama is the uncompiled Vector API (ArcadeData/arcadedb#10072: 178 s against 6.5 s for a 20k x 384-d graph build), and the choice stays for the life of the process.

The order, reproduced on Linux x86-64 (the dev native wheel 26.11.1.dev0 sha256 88bd63d53368, engine e353330c5c, from before #154 so nothing sets ARCADEDB_VECTOR_PROVIDER, libjvector-x86_64.so removed from the package directory, 2,000 vectors of 32 dimensions, create_vector_index + find_nearest, a fresh process per row):

first call in the process provider before the database provider after the database and index find_nearest hits
diag() NativeVectorizationProvider NativeVectorizationProvider 5
the database n/a DefaultVectorizationProvider 5

So diag() first takes the choice away from the engine for the whole process, and the database first gets the engine's scalar provider. On Linux jvector's own choice is its NativeVectorizationProvider (Highway) even with the library file removed from the package directory; on macOS and Windows it is Panama, per the CI logs above.

Who it affects. diag() is in the private arcadedb._native.lib module and nothing in the package calls it, so a user's process only meets this if their code calls it before opening a database. The test does; our benchmark scripts call it after opening a database, so their provider readings stand.

Fix. diag() reads the provider through VectorApiSupport.jvectorProvider(), which runs the engine's choice first. The existing test then checks exactly this (its child calls diag() first). PR to follow, verified with a dispatched native run on all four platforms before merge.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions