[lokewate]

Summary

Replicas load their artifact sets at different times. A flow started by a replica that loaded manifest_v4.json can run its next step on a replica that started before v4 was published. This PR makes Set.Registry load a version above the highest it has, instead of failing that step.

Stack

Manifest versioning, in five PRs to review and merge in order. Each one builds and passes its tests on its own.

1. feat(artifact)!: load versioned manifests as a set #305 artifact: load versioned manifests as a set
2. feat(artifact)!: resolve explicitly versioned rows across manifests #346 artifact: resolve explicitly versioned rows across manifests
3. feat(artifact): load newer manifests on demand #319 artifact: load newer manifests on demand ← this PR
4. feat(workflow)!: carry the manifest version through every workflow #322 workflow: carry the manifest version through every workflow
5. feat(taskflow)!: resolve a task's templates through its manifest version #306 taskflow: resolve a task's templates through its manifest version

#320 (validate every artifact a manifest lists) and #321 (manifestcheck) are parked: pinning doesn't need them, and a release job can bring validation back later.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Performance improvement
• Other (please describe):

Changes Made

• Set.Registry loads a missing newer version on demand. Current does not move: a new manifest becomes current when the process reloads the set.
• A probe fetches only up to the version asked for, so a broken later manifest cannot fail it.
• It holds no lock that lookups wait on. A one-slot channel serializes probes across the loader round trips, and a caller waiting for it gives up when its own context ends; mu is taken only to read state and to install what a probe fetched.
• Back-off. After a probe that fails or finds nothing, the next one waits 10s, so retries of a flow pinned to an unpublished version do not each cost a loader round trip. A probe cut short by its caller's context does not start the wait, since it says nothing about what is published.
• GetExplicit probes the same way when no loaded manifest lists the version asked for, and reports a cancelled request as cancelled, not as ErrNotFound.

Testing

• I have tested this change locally
• I have added tests that prove my fix is effective or that my feature works
• I have tested edge cases
• All existing tests pass

Run with go test -race ./artifact/. The tests cover loading a manifest published after the set was loaded, a bad manifest found on demand (reported, not cached), a probe stopping at the version asked for, back-off after both empty and failed probes, a probe not blocking lookups of loaded versions, concurrent Registry calls, a pin still loading newer versions on demand, GetExplicit finding a version only a newer manifest lists, a caller waiting behind a stuck probe giving up at its deadline, and a cancelled probe not starting the back-off. The last two fail without the fix (the first hangs).

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have checked that there are no merge conflicts

Related Issues

Part of OpenNSW/one-trade-artifacts#243. Host: OpenNSW/nsw-srilanka#660.

Screenshots/Demo

N/A

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 4f95df01-7f42-4839-87d1-33ff5068c098

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mushrafmim]

LGTM!