From 16fecdc6b53075cc42b10eeff2e87af9eb3325a2 Mon Sep 17 00:00:00 2001 From: Taewoon Kim Date: Sat, 31 Oct 2026 17:05:18 +0900 Subject: [PATCH] docs: a native image build takes about 5 to 30 minutes per platform, not 1 to 2.5 hours (#157) The Image and wheel jobs of the last four native runs (41097491149, 41108390123, 41165481827, 41179658884) took 5.5 to 27.2 minutes, macos-arm64 the slowest. The runner limit shared with upstream ArcadeDB still justifies the selective matrix; only the number changes, in the six places that state it. Closes #157 Co-Authored-By: Claude Opus 5.5 --- .github/workflows/release-native.yml | 2 +- .github/workflows/test-native.yml | 2 +- CLAUDE.md | 2 +- README.md | 2 +- docs/modules/ROOT/pages/development/ci.adoc | 2 +- scripts/select_native_matrix.py | 2 +- 6 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.github/workflows/release-native.yml b/.github/workflows/release-native.yml index 7d24ad5..4fee617 100644 --- a/.github/workflows/release-native.yml +++ b/.github/workflows/release-native.yml @@ -5,7 +5,7 @@ name: Release native (arcadedb) # starts release-jvm.yml, which publishes `arcadedb-jvm`; the two workflows do not wait for each other. # # The build and the tests are test-native.yml, called with `release: true` (the full matrix: linux-x86_64, linux-aarch64, -# macos-arm64 and windows-x64, each on Python 3.11 to 3.14). A native image build takes about 1 to 2.5 hours per platform; this workflow adds +# macos-arm64 and windows-x64, each on Python 3.11 to 3.14). A native image build takes about 5 to 30 minutes per platform; this workflow adds # no build of its own. The way the JVM release calls test-jvm.yml is the model. # # PyPI keys a trusted publisher on this FILENAME and on the job's environment, so do not rename either without changing the publisher diff --git a/.github/workflows/test-native.yml b/.github/workflows/test-native.yml index 295ccdf..a52fa06 100644 --- a/.github/workflows/test-native.yml +++ b/.github/workflows/test-native.yml @@ -2,7 +2,7 @@ # wheels and runs the parity suite against them; release-native.yml calls it on a version tag and publishes. The JVM build, # `arcadedb-jvm`, is test-jvm.yml / release-jvm.yml. # -# A native image build takes about 1 to 2.5 hours per platform and the ArcadeData organisation shares a runner cap with +# A native image build takes about 5 to 30 minutes per platform and the ArcadeData organisation shares a runner cap with # upstream ArcadeDB, so the load is selective (scripts/select_native_matrix.py holds the legs, tests/test_select_native_matrix.py pins them): # pull request only when a native path changed (the `paths` filter below); linux-x86_64, Python 3.12; not for a draft # push to main only when a native path changed; all four platforms, Python 3.12 and 3.14 diff --git a/CLAUDE.md b/CLAUDE.md index f1cfade..67efcda 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -75,4 +75,4 @@ Each package has one test and one release workflow: `test-jvm.yml` and `release- `.github/workflows/release-jvm.yml` publishes `arcadedb-jvm` on a bare-version tag (`26.11.1.dev0`, `26.11.1`, `26.11.1.post1`; no `v`, never `aN`/`bN`/`rcN`). Its first job runs `scripts/check_version.py` (the devN/postN counting rule, read-only, fails closed); `scripts/after_tag_verify.sh` checks the finished release. The rule is stated once, on the docs page `development/release.adoc`. -`test-native.yml` builds the GraalVM native image and runs the parity suite against the installed wheel (selective: scripts/select_native_matrix.py, 1 to 2.5 hours per platform, do not add runs lightly). `.github/workflows/release-native.yml` publishes `arcadedb` on the same tag: it calls `test-native.yml` with `release: true` (4 platforms x Python 3.11 to 3.14), then the readiness gate (the repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET`, for a final or post version), then one publish job through the environment `pypi`. +`test-native.yml` builds the GraalVM native image and runs the parity suite against the installed wheel (selective: scripts/select_native_matrix.py, 5 to 30 minutes per platform, do not add runs lightly). `.github/workflows/release-native.yml` publishes `arcadedb` on the same tag: it calls `test-native.yml` with `release: true` (4 platforms x Python 3.11 to 3.14), then the readiness gate (the repository variable `ARCADEDB_NATIVE_READINESS_BAR_MET`, for a final or post version), then one publish job through the environment `pypi`. diff --git a/README.md b/README.md index e3f4318..e9c89c1 100644 --- a/README.md +++ b/README.md @@ -96,7 +96,7 @@ Releases from this repository start with 26.11.1. The 26.10.x wheels were built ./scripts/build.sh # an arcadedb-jvm wheel for your platform in dist/ ``` -The `arcadedb` (native) wheel is built from `native/` with GraalVM Community Edition; [`CLAUDE.md`](https://github.com/ArcadeData/arcadedb-python/blob/main/CLAUDE.md) has the commands. A native image build takes about 1 to 2.5 hours per platform. +The `arcadedb` (native) wheel is built from `native/` with GraalVM Community Edition; [`CLAUDE.md`](https://github.com/ArcadeData/arcadedb-python/blob/main/CLAUDE.md) has the commands. A native image build takes about 5 to 30 minutes per platform (macOS is the slowest). The JVM build runs in Docker on Linux. macOS and Windows need a JDK 25 or later. The jars come from the `arcadedata/arcadedb` Docker image: `latest` by default, or the tag in `ARCADEDB_TAG` (for example `ARCADEDB_TAG=26.10.2 ./scripts/build.sh`). The package version is read from the engine jar in that image, so a `26.11.1-SNAPSHOT` image builds `26.11.1.devN`. diff --git a/docs/modules/ROOT/pages/development/ci.adoc b/docs/modules/ROOT/pages/development/ci.adoc index 1002fab..4653a95 100644 --- a/docs/modules/ROOT/pages/development/ci.adoc +++ b/docs/modules/ROOT/pages/development/ci.adoc @@ -67,7 +67,7 @@ The workflow `test-native.yml` builds the native image of the `arcadedb` package * A pull request runs it only when native paths change: `native/**`, `src/arcadedb/**`, `tests/**`, `scripts/select_native_matrix.py` or `test-native.yml` itself. Changes under `tests/` and `src/arcadedb/` are included because both backends share those files, so a shared test or source can break native parity without touching a native file; docs outside those paths do not run it. It then runs on `linux-x86_64` with Python 3.12 only. * A push to `main`, the nightly run, a manual run (dispatch) run all four platforms: `linux-x86_64`, `linux-aarch64`, `macos-arm64` and `windows-x64` (Python 3.12 and 3.14). A release (`release-native.yml` calls the workflow with `release: true`) runs the same four platforms on all four Python versions, 16 legs. * The `pyver` job installs the already built `linux-x86_64` wheel (no rebuild) into the Pythons that the full parity runs of the event leave out (3.11, 3.13 and 3.14 on a pull request; 3.11 and 3.13 on a push to `main`, the nightly run and a dispatch; none on a release) into 3.15 and into the free-threaded 3.14t, and runs 13 test files there (14 where polars installs; 3.15 has no jpype1 wheel, so it leaves out the two files that need jpype1), about 20 seconds of tests per Python. It is informational: `release-native.yml` reads only the `native-passed-*` markers and the `native-wheel-*` wheels, so a red `pyver` leg never blocks a release. The 3.14t leg has no polars (no free-threaded wheel exists), so it skips `test_dataframe_ingest.py`. `scripts/select_native_matrix.py` decides its legs. -* A native image build takes about 1 to 2.5 hours for each platform. The organisation shares one runner limit with upstream ArcadeDB, so do not add runs lightly. Change the paths or the matrix of `test-native.yml` only for a reason that you can write down. +* A native image build takes about 5 to 30 minutes for each platform (macOS is the slowest). The organisation shares one runner limit with upstream ArcadeDB, so do not add runs lightly. Change the paths or the matrix of `test-native.yml` only for a reason that you can write down. == Change the selection diff --git a/scripts/select_native_matrix.py b/scripts/select_native_matrix.py index e4c1f05..af785e6 100644 --- a/scripts/select_native_matrix.py +++ b/scripts/select_native_matrix.py @@ -1,7 +1,7 @@ #!/usr/bin/env python3 """Choose the legs of the native test workflow (.github/workflows/test-native.yml) for one event. -A native image build takes about 1 to 2.5 hours per platform and the ArcadeData organisation shares a runner cap with +A native image build takes about 5 to 30 minutes per platform and the ArcadeData organisation shares a runner cap with upstream ArcadeDB, so a pull request builds one platform (linux-x86_64) and tests one Python (3.12), and only when native paths changed (the workflow's `paths` filter). Push to main, the nightly run and a dispatch build all four platforms and test Python 3.12 and 3.14 on each. A release (release-native.yml calls the workflow with `release: true`) tests every