Repository navigation
Publish a daily GPU Operator chart and OLM bundle with main-branch operand images - #3087
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Priority: ➖ Normal Merge Risk: ⚪ Minimal · up to
|
|
Instead of |
dca1d9c to
101e1ce
Compare
fdeec34 to
4c11283
Compare
|
@rahulait Thanks for the suggestion. Updated the version format to Note that: The workflow now skips publishing if a build already exists for that date. An existing build can be replaced by manually running the workflow with |
4c11283 to
44447f4
Compare
rahulait
left a comment
rahulait
left a comment
There was a problem hiding this comment.
LGTM, thanks @abrarshivani
44447f4 to
56840b2
Compare
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| main_build_tags=$(regctl tag ls "${main_build_repository}") | ||
| main_version=$(grep -E "${MAIN_VERSION_PATTERN}" <<< "${main_build_tags}" | sort | tail -n 1 || true) | ||
|
|
||
| if [[ -z "${main_version}" ]]; then | ||
| echo "Error: ${main_build_repository} has no main build" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| [[ "${main_version}" =~ ${MAIN_VERSION_PATTERN} ]] | ||
| published_date="${BASH_REMATCH[1]}" | ||
|
|
||
| oldest_accepted_date=$(date -u -d "${max_age_days} days ago" +%Y%m%d) | ||
| if (( published_date < oldest_accepted_date )); then | ||
| echo "Error: newest main build ${main_version} is older than ${max_age_days} day(s); the scheduled publish did not run or failed" >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| main_build_record=$(regctl artifact get "${main_build_repository}:${main_version}") | ||
| operator_version=$(jq -er '.operatorVersion' <<< "${main_build_record}") | ||
| recorded_chart_digest=$(jq -r '.chart.digest' <<< "${main_build_record}") | ||
| current_chart_digest=$(regctl image digest "${chart_repository}:${main_version}") | ||
| if [[ "${current_chart_digest}" != "${recorded_chart_digest}" ]]; then | ||
| echo "Error: ${chart_repository}:${main_version} is ${current_chart_digest}, but main build ${main_version} recorded ${recorded_chart_digest}" >&2 | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
🟡 Main build selection lacks regression tests
Changes to get-latest-main-build.sh have no committed tests for stale records or mismatched chart digests. Regressions can select the wrong chart for forward-compatibility tests.
Learn more
Repository guidance requires appropriate coverage for new behavior. The main build lookup is new and determines which published chart the weekly GPU tests install. Its failure branches depend on registry output, artifact contents, UTC dates and digest comparisons, yet the PR adds no repeatable test cases for them. Local harnesses described in the PR are not checked into the repository.
Example: A tag listing contains a current build whose recorded chart digest differs from the registry digest. A test must assert that selection fails rather than emitting a chart version.
Recommended fix: Add committed shell tests using stubbed regctl and a controlled date, covering missing tags, freshness boundaries, invalid records, digest mismatches and a successful lookup. Wire them into an existing CI test target or workflow.
Was this helpful? React with 👍 or 👎 to provide feedback.
56840b2 to
d689cd5
Compare
d689cd5 to
557f604
Compare
Add publish-main-chart.yaml, which runs daily and can be started by hand on the default branch only. It resolves the newest main image of every operand that publishes per-commit images, prepares the chart values once, and publishes the chart, the OLM bundle and a build record, all versioned 0.0.0-main-<UTC date>, for example 0.0.0-main-20261016. There is one build per date: the run stops when that date's record already exists, unless it was started by hand with overwrite. - get-latest-images.sh covers driver-manager, vgpu-device-manager, the kata and kubevirt sandbox device plugins and cc-manager, takes several components and resolves them concurrently. - generate-values-overrides.sh takes VALUES_PATH=IMAGE pairs, and apply-values-overrides.sh merges them into values.yaml with merge-yaml.py, rejects paths that are not image values and checks that every image renders unchanged into the ClusterPolicy. The operator, validator and node status exporter use the operator image of the same commit. dcgm, dcgm-exporter, gds, gdrcopy, the driver and the DRA driver stay on their released versions. - The chart goes into the existing charts/gpu-operator package through a new main_chart_version input of publish-helm-oci-chart.yaml. - The bundle CSV is updated with update-csv-images.py and validated and pushed through two composite actions, setup-olm-bundle-tools and validate-and-push-olm-bundle, which release-rc-assets.yaml now uses too. The operator-sdk version comes from OPERATOR_SDK_VERSION in versions.mk rather than the builder label in the bundle annotations, which only records the SDK that generated the bundle. The CSV name and version are unchanged, so a main bundle is installed on its own. - push-main-build-record.sh pushes a record of the chart, bundle and image digests to gpu-operator/main-builds only when both were published. forward-compatibility.yaml now finds the newest build record with get-latest-main-build.sh and runs e2e against that build's chart, with the operator version taken from the record. It fails if the newest build is dated before yesterday (UTC) or the chart's digest no longer matches the record. e2e-tests.yaml gets chart_reference and chart_version inputs to install a published chart instead of publishing one from the current commit; callers that pass neither behave as before. Signed-off-by: Abrar Shivani <ashivani@nvidia.com>
557f604 to
4601891
Compare
Description
Publish a daily Helm chart and OLM bundle with main-branch operand images for CI and QA, and run forward compatibility against them.
publish-main-chart.yaml(daily) resolves the newest main image of each operand, prepares the chart values once, and publishes the chart, the bundle and a build record (chart, bundle and image digests), all tagged0.0.0-main-<UTC date>, for example0.0.0-main-20261016. There is one build per date: a run stops if that date's record exists, unless started by hand withoverwrite. The record is pushed only when both chart and bundle were published. DCGM, DCGM exporter, GDS, GDRCopy, the driver and the DRA driver stay on released versions.merge-yaml.pyfor the chart values,update-csv-images.pyfor the bundle CSV, and two new composite actions, shared by both workflows, that set up the bundle tools and validate and push the bundle. The CSV name and version are unchanged, so a main bundle is installed on its own, not through a catalog.forward-compatibility.yaml(Mondays) installs the chart of the newest recorded build, and fails if it is dated before yesterday (UTC) or its digest changed.e2e-tests.yamlcan now install a published chart viachart_referenceandchart_version.Using a main build
Checklist
make lint): no Go changes; shellcheck and actionlint are clean on the changed scripts and workflows apart from findings that already exist on mainmake validate-generated-assets): no generated assets touchedmake validate-modules): no module changesTesting
curl,regctl,helmandsleep.operator-sdk bundle validateandmake validate-csvpass.0.0.0-main-20261020, and the record was read back as forward compatibility reads it. Re-running the whole run found that date published and skipped publishing, leaving the chart unchanged.closes: https://github.com/NVIDIA/cloud-native-team/issues/586