Skip to content

feat(deps): agent-driven Dependabot PR verifier (phase 1) - #849

Draft
nieblara wants to merge 58 commits into
mainfrom
cursor/dependency-pr-verifier-9b50
Draft

nieblara wants to merge 58 commits into
mainfrom
cursor/dependency-pr-verifier-9b50

Conversation

@nieblara

@nieblara nieblara commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Requirements

  • I have added test coverage for new or changed functionality (scripts/dependency-pr/test/run.sh: 154 unit tests. scripts/dependency-pr/test/replay.sh: 16 replays of past PRs.)
  • I have followed the repository's pull request submission guidelines
  • I have validated my changes against all supported platform versions (Linux only, with Docker and arm64 qemu. The release targets were cross-compiled, but not executed.)

Related issues

Dependabot PRs stay open for a long time, and a green CI run does not mean that a PR is safe. #720 merged green, but its generated code was never regenerated, and the regenerated code does not compile.

Describe the solution you've provided

This PR adds a toolkit that verifies a Dependabot PR to the standard of a diligent reviewer. After a run, a person does not have to verify anything again. A person only answers the decisions that the comment asks for. The toolkit does not approve or merge.

  • scripts/dependency-pr/verify.sh --pr N (or --branch B) runs without prompts. It makes a base worktree and a pr worktree (the PR merged into the base tip), classifies the updates, and runs the baseline checks. It writes result.json, comment.md, and logs.
  • The verdict is block (exit 1, with the fix), incomplete (exit 2, a check or review did not run, with the fix for each item), needs-human (exit 1, one yes-or-no question for each decision, with evidence), or safe-to-merge (exit 0), in that order of strength. A question waits until nothing is incomplete.
  • A gate is a check that must pass. A failure that also happens on base is "pre-existing" and does not count against the PR, but it never satisfies a gate. For a check that lists findings (actionlint, golangci-lint, prettier, npm ls), a PR is also pre-existing if each of its findings occurs on base. A tool or environment failure is incomplete, never fail.
  • The baseline covers what CI does not:
    • Go: tidiness, go generate drift with a build of the regenerated code, mock regeneration for go.uber.org/mock updates, and the go directive.
    • UI: dist/ drift, ERESOLVE, unused dependencies, and new advisories.
    • All: transitive changes, new npm install scripts, licenses, upstream notes with compare links, and direct updates that Dependabot's description does not name (pr-disclosure). Bot summaries and quoted release notes do not count as disclosure. A Go update that a named update requires is a note, not a decision.
    • Smoke and release: a binary smoke test (help for all commands and a dev-server start), and a goreleaser-cross snapshot of all release targets.
    • Docker: a release image built with the static CGO musl binary, with HTTPS and the dev server tested inside it.
    • Actions: SHA pins, actionlint, interface comparison against every old ref that the PR replaces (inputs, outputs, defaults, new inputs that default to on, runtime), the nested actions and run steps of a composite action, and permission changes. Floating tags resolve to their release (v4 (v4.4.0)). If an exact pin is older than the release of a floating tag, the check asks a person.
    • CI: the workflows whose triggers match the PR must have a green run on the head commit. A missing, queued, startup-failure, or 0-job run is incomplete. A result for another commit counts as not run. A failure only in steps that read none of the changed ecosystems (a flaky Go test on an npm PR) is incomplete with a rerun as the fix. CI that ran before the base branch changed a workflow is also incomplete.
  • Upstream notes come from release notes or CHANGELOGs. For npm monorepo packages (react-router, @launchpad-ui/*, vite), they come from the package CHANGELOG.md. For golang.org/x/*, they come from the commit messages on the GitHub mirror. The summary says whether the notes cover the full range, part of it, or none of it.
  • The release snapshot uses the private ghcr.io/launchdarkly/goreleaser-cross image when GHCR credentials can pull it. Otherwise, it uses the public goreleaser/goreleaser-cross:v1.24.2 image, pinned by digest, with sha256-pinned musl.cc toolchains at /musl. If neither image can be pulled, the check is incomplete. The check details state the fidelity gaps: the public image and musl.cc replace the LaunchDarkly image, the binaries are not executed, and the Docker and Homebrew steps do not run.
  • The agent writes an impact review (agent/impact.json) and generated checks. A discriminating check counts only if it fails on the old version and passes on the new version. Generated checks run with a private HOME and no global git configuration. The runner removes tokens from all check logs.
  • Each failing check with a mechanical fix records a recipe (command plus expected paths) in result.json .fixes[]. A later apply-fixes.sh can use the recipes.
  • post-comment.sh is the only script that writes to GitHub. The caller runs it. It edits one comment and does not post if the PR head moved.
  • The skill is in .agents/skills/verify-dependency-pr/, with playbooks for gomod, the UI, the npm wrapper, GitHub Actions, and Docker. AGENTS.md points to it and corrects the pre-commit section.

Validation (local, no comments posted)

The runs merged each PR into main @ 9197f92, on a VM with Docker and no GHCR credentials. The 2026-10-06 rows come after the fixes for the bugs that two runs on all 16 open Dependabot PRs found.

PR Verdict Reason
#829 go-sqlite3 safe-to-merge All 8 release targets build with the public goreleaser-cross image and musl.cc. golangci-lint is clean. The generated check is proven.
#830 alpine 3.24 safe-to-merge The release image builds with the static CGO musl binary. ldcli, HTTPS, and the dev server work in it.
#717 setup-go v4/v5 → v6 incomplete (Oct 13) CI did not run on the new head 348e5d8. Go and dev-server UI failed with 0 jobs, and Dependency Scan is queued. Re-run CI. On Oct 6 it was safe-to-merge.
#718 release-secrets needs-human (Oct 20) Compared with the real old ref, v1.0.1. The composite steps move configure-aws-credentials from v1-node16 to v4 in release-only workflows.
#831 react-window 2 needs-human "Remove @types/react-window and react-window from package.json instead of updating them?" (The replay has no impact review, so it is incomplete.)
#719 checkout v6 needs-human "Accept actions/checkout v6.0.3 in workflows that PR CI does not run?" The exact pin is a separate question.
#721 setup-python v6 needs-human (Oct 13) "Accept that pre-commit in go.yml will use the runner's Python 3.12 instead of the newest 3.x in the tool cache?"
#722 release-please v5 needs-human "Accept release-please-action v5.0.0 in workflows that PR CI does not run?" The action sets its outputs at run time, so they are not counted as removed.
#729 react-router 8 block Conflicts with main. npm ci fails: react-router 8.0.1 needs react >= 19.2.7.
#779 npm security group block npm ci fails, and new GHSA advisories appear. The description does not name 4 of the direct updates (react-router 8, vite 8, and two others). License question: "Accept MPL-2.0 for lightningcss 1.33.0 and 11 lightningcss-* package(s) (dev-only build tools)?"
#723 @launchpad-ui/core 0.59 block Conflicts with main, and npm ci fails on the react 19.2.6 peer. No new advisory ID, so no audit question.
#621 mock 0.6.0 block Conflicts with main. mock 0.6.0 requires the x/term update, so no disclosure question. All 8 release targets build on the old tree.
#737 x/net security group block Conflicts with main. x/net 0.55.0 requires the x/term update. It fixes nine Go advisories, and GO-2026-5970 and GO-2026-5320 stay reachable.

The #717 and #721 rows come from reruns on main @ b818ae4 on Oct 13. The replays cover each row above except #719, #729, and #830. Each fixture added for a bug fails on the code before the fix. They also cover #720 (block), #639 stale dist (block, with a rebuild recipe), a low-risk patch #635 (safe to merge), and the #829 false safe case with no pullable image (incomplete).

Verifier fixes, 2026-10-20

The Oct 13 to Oct 20 runs found 22 verifier bugs. This round fixes all 22 and adds the mocks-regenerate baseline check. Each lenient or verdict-changing fix has a unit test or replay that fails on 2b24b7a.

  • Lenient: actions-coverage now compares the steps of a composite action. chore(deps): bump launchdarkly/gh-actions from release-secrets-v1.2.0 to 1.2.1 #863 (release-secrets v1.2.1) hid a nested configure-aws-credentials v4 → v6 update and a replaced SSM loader.
  • Lenient: a UI tool crash is incomplete, not a format failure. ui-prettier runs node_modules/.bin/prettier, not npx.
  • Ranking, questions, and fixes: the verdict ranks incomplete above needs-human. Each choice is its own question. Each incomplete item names its fix, such as a rerun on GitHub for CI.
  • Notes: upstream-changes resolves floating action tags, lists the commits that a diverged ref lacks, and counts "version bump only" entries as no coverage.
  • Other fixes: a test-only Go module is low tier. Release tooling is high tier. Licenses compare at the same lockfile path. 0.x minor updates have one label. Comments show both sides of a generated check, the tier reasons, and the guard results next to each question.
  • Comment text: bare @ names and raw <...> text are in code, and upstream #N refs become owner/repo#N, so comments do not mention accounts or link the wrong PRs.

Reruns with the fixes (no comments posted): #863 asks about the composite change, and it is incomplete because the base branch changed go.yml after its CI ran. #870 with a crashing npx first on PATH runs prettier and is safe to merge. With prettier itself crashing, it is incomplete. #869 is incomplete with "re-run the failed job (Go: build)" instead of block.

Describe alternatives you've considered

  • A Go tool for the JSON logic. I kept bash, jq, and small Python helpers, so that a GitHub workflow can call the same files.
  • A cross-compile with CGO off. It cannot build internal/dev_server/db/backup, so the toolkit uses goreleaser-cross.
  • zig cc for the cross-compile without an image. It cannot link the darwin targets without a macOS SDK, so the toolkit uses the public goreleaser-cross image.

Additional context

  • For automation, run verify.sh in a pull_request job with a read-only token. Post from a separate workflow_run job. Do not use pull_request_target with a PR checkout.
  • The go-generate-drift check reports the chore(deps): bump github.com/oapi-codegen/oapi-codegen/v2 from 2.4.1 to 2.7.1 #720 drift on main as pre-existing. This PR does not fix it.
  • The public goreleaser-cross image is 10.9 GB on disk. The check pulls it on demand and caches the Go modules and builds in Docker volumes.
  • govulncheck lists the advisories that a Go update fixes (reachable, imported package, or required module). Only a new reachable advisory fails.
Open in Web Open in Cursor 

cursoragent and others added 30 commits October 5, 2026 19:56
verify.sh runs a deterministic, non-interactive baseline for a Dependabot PR
(tidy, codegen drift, UI dist drift, binary and dev-server smoke, actions
pinning/coverage, docker base image, and more), compares failures against the
base branch to separate pre-existing issues, runs agent-generated per-PR checks
on both versions, and writes result.json and comment.md. post-comment.sh is the
only script that writes to GitHub and only upserts one comment.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…otes

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
- run/go_tool keep command echoes and tool downloads out of captured output
- pass gh's config dir through the hermetic env so action.yml comparison works
- require the goreleaser-cross snapshot for cgo-tagged updates (required_for_tags)
- replace the CGO-off cross-compile (cannot build internal/dev_server/db/backup)
  with an opt-in goreleaser-cross snapshot
- dev-server smoke passes a placeholder access token instead of relying on env
- report go test cache hits

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…ample

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…m and a scratch prefix

go-npm falls back to $npm_config_prefix/bin because newer npm has no 'npm bin',
so a plain local npm ci tries to write into the global prefix.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…semantics

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…verifier-9b50

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
Verdicts are now block, needs-human, incomplete, or safe-to-merge.
Checks report fail, decide (one exact question with evidence), incomplete,
or info. A person is asked only for a real decision. Tool and environment
gaps are reported as incomplete. A gate must pass: a pre-existing failure
never satisfies it. High tier alone no longer asks a person.

New checks collect upstream release notes and diffs, compare transitive
dependencies and licenses, verify go directive changes, and compare action
outputs, input defaults, runtime, and workflow permissions. Each failing check
can carry a machine-applicable fix recipe in result.json.

release-snapshot pulls the private goreleaser-cross image first and reports
incomplete if the pull fails. docker-image builds the static CGO musl binary
that the release ships and runs the dev server in the image.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
Replays #720 codegen drift, #639 stale UI dist, #723 npm ERESOLVE, #831
unused dependency, a low-risk patch (#635) that must be safe to merge, and
the #829 false-safe case where an unpullable image satisfied the cgo gate.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…wer standard

The skill and the playbooks now say what a thorough reviewer must make sure
of for each ecosystem and risk tier, what the baseline checks cover, what the
agent must do, and when to ask a person. An impact review can state that no
local check can prove a reachable change, which turns it into a decision.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…the same change

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…e-snapshot

If the private ghcr.io/launchdarkly/goreleaser-cross image cannot be
pulled, release-snapshot uses the public goreleaser/goreleaser-cross
v1.24.2 image, pinned by digest, with sha256-pinned musl.cc toolchains
mounted at /musl. If neither image can be pulled, the check is
incomplete. The summary names the image. The details list each built
target and the fidelity gaps: the public image and musl.cc replace the
LaunchDarkly image, the binaries are not executed, and goreleaser build
does not run the Docker and Homebrew steps.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
The pr-829-release-targets-build fixture expects safe-to-merge and a
release-snapshot pass for all 8 targets. pr-829-gate-not-satisfied
now makes both images unpullable, so it still proves that a missing
image gives incomplete. replay.sh can match text in a check summary.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
- verdict.jq: if each PR finding of a list check (actionlint,
  golangci-lint, prettier, actions-pinning, npm ls) also occurs on
  base, the failure is pre-existing. Before, a PR that removed one
  actionlint finding got a block (#717, #721).
- golangci-lint: run with --allow-parallel-runners. A non-zero exit
  with no finding is incomplete, not a failure with 0 findings.
- ui-npm-ci: take the conflict from npm error lines only. An npm warn
  line named the wrong peer on #729.
- deps.py: a package that already had an install script on base adds
  no new script (esbuild on #779).
- ui-dep-usage: join package names with "and".
- pr-state: for a conflicting PR, do not say that the checks ran on
  the PR merged into main.
- verify.sh: save the baseline profile. --phase generated and
  --phase render use it instead of the default profile.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
- classify.sh: when workflows pin an action at several old refs, record
  each ref that the PR replaces. The old version is the lowest of them.
  Before, #718 compared release-secrets v1.2.0 with itself, and #717
  compared only setup-go v5 with v6.
- actions_analysis.py: compare each replaced ref with the new ref. An
  output or input counts as removed only if an old action.yml declared
  it. release-please-action declares no outputs and sets them at run
  time, which gave a false block on #722.
- actions-coverage.sh: read fields with the unit separator. Tab is IFS
  whitespace, so empty fields collapsed and shifted the runtime into
  the wrong field (#717, #721). Show a missing default as "none".

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
- pr-disclosure: for a grouped update, or a PR with more than one
  direct update, list the direct updates that the PR title and body do
  not name, and ask one decision. #779 names dompurify and uuid, but
  also moves @launchpad-ui/components, launchdarkly-js-client-sdk,
  react-router, and vite.
- upstream.py: use the from_ref and to_ref from classify. Map
  golang.org/x/* to the github.com/golang mirror, and use the commit
  messages as notes when a Go module has no release notes. For npm
  monorepo packages (react-router, @launchpad-ui/*), read the package
  CHANGELOG.md and use <name>@<version> tags.
- verify.sh --pr-meta FILE: read the PR title and body from a file,
  for replays that need them without GitHub.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
New fixtures: #717 safe-to-merge (setup-go from v4 and v5), #718 compares
release-secrets v1.0.1 with v1.2.0, #721 needs-human with the
python-version question, #722 needs-human (not block), and #779 block
with the undisclosed updates and no esbuild question. The #831 fixture
expects the new wording. The impact-review copies drop notes about the
old verifier bugs.

replay.sh supports pr_meta, decisions_exclude, and updates expectations.
run.sh adds 16 unit tests with stub tools: the findings subset rule,
golangci-lint lock contention, npm warn lines, install scripts on base,
pr-disclosure, pr-state wording, classify with mixed refs, and
actions-coverage with a stub gh.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…e sources

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
The disclosure evidence says "none of them" instead of "0 of them".
The ERESOLVE summary drops the node_modules path that npm prints on the
line after the conflict.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
Try <name>@<version>, v<version>, and <version>, and use the first tag
that exists. vite tags v8.2.1 and launchdarkly-js-client-sdk tags 3.9.5,
so #779 got no compare link or notes for them.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…ced updates

- pr-disclosure strips bot blocks (<!-- NAME --> … <!-- /NAME -->, such
  as the Cursor Bugbot summary) and the <details> sections that quote
  upstream notes. On #737, golang.org/x/term was named only in the
  Bugbot summary, and the check passed.
- A direct Go update that a named update requires at the new version
  or higher is forced (deps.py forced walks go mod graph). It is a note,
  not a decision. On #621, mock 0.6.0 needs x/term 0.34.0 through
  x/tools and x/net.
- The check calls an update grouped only if the title says so.
- deps.py licenses also reports the package name, version, and dev flag.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
npm audit flags each package that depends on a vulnerable package. On
#723 it flagged @launchpad-ui/core only through navigation and
react-router, whose advisories base has too, and the check asked about
a new advisory that does not exist. The check now compares GHSA IDs.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
- A CHANGELOG section ends at the first release heading at or below the
  old version, not after 400 lines. If no such heading exists, the
  coverage is partial, with the lowest version that the notes reach.
- If a version has no tag (@launchpad-ui/core 0.59.17), read the
  CHANGELOG on the default branch and skip the compare link.
- Scan up to 10 pages of releases, and stop at the old version.
- The summary gives full, partial, and missing coverage instead of
  "notes for N of N". On a low-tier update, it does not ask for an
  impact review.
- The breaking keyword also matches removed packages, components,
  exports, props, inputs, and outputs.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
…old image tags

- actions_analysis: evaluate defaults as github.com does. A GHES
  fallback (github.server_url == 'https://github.com' && X || Y) is X,
  so the setup-python token default is not a change (#721). The
  question names the workflows that use the action.
- license-changes: one item per package family, which says dev-only
  build tools: "Accept MPL-2.0 for lightningcss 1.33.0 and 11
  lightningcss-* package(s) (dev-only build tools)?" (#779).
- verdict.jq: a guard regression or a failing gate already shows that a
  change reaches ldcli, so the missing discriminating proof is not an
  incomplete item (#723, #729, #779).
- release-snapshot: accept a tagged release image. A conflicting PR is
  tested at its old head, and trees from before #629 use
  goreleaser-cross:v1.24.2 instead of a digest (#621).

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
govulncheck -format json reports each advisory as reachable, in an
imported package, or in a required module. Only a new reachable
advisory fails, as before. The summary now lists the fixed advisories
by level and the reachable advisories that stay (#737 fixes nine and
leaves GO-2026-5970 and GO-2026-5320).

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
New fixtures: #621 (forced x/term update, not grouped, release snapshot
on the old tree) and #737 (x/term named only in the Bugbot summary).
The #723 fixture expects no audit question and full note coverage. The
#779 fixture expects full note coverage, GHSA IDs, and the short license
question. The #721 fixture expects no token default in the question.

run.sh adds 12 unit tests with stub tools: bot text and <details> in
pr-disclosure, forced Go updates, npm audit advisory IDs, note coverage
(partial CHANGELOG, missing tag, release scan past 3 pages), the
license family question, impact proof with a guard regression, and the
govulncheck levels. Each one fails on the code before this change.

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
… govulncheck levels

Co-authored-by: Ramon Niebla <nieblara@users.noreply.github.com>
cursoragent and others added 12 commits October 13, 2026 06:05
… head

The status-check rollup does not show a workflow run that failed before
any job started, or a run that waits in the queue. On #717, ci-status
counted only Bugbot, Semgrep, and the bot-oversight check, and passed.
The Go and dev-server UI runs had failed with 0 jobs, and Dependency Scan
was still queued.

ci-status now finds the workflows whose triggers match the PR (event,
branch, activity types, and path filters). It compares them with the runs
on the head commit. A missing, queued, in-progress, startup-failure, or
0-job run is incomplete. A result for another commit counts as not run.
ci-status is now a required check, so incomplete CI gives an incomplete
verdict.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
A generated check inherited the global git configuration. An insteadOf
rule with an access token put the token into a remote URL, and the check
wrote the URL to its log.

Generated checks now run with a private HOME, GIT_CONFIG_GLOBAL=/dev/null,
and GIT_CONFIG_NOSYSTEM=1. The tool caches stay shared. After each check,
the runner removes tokens from the log and from the evidence files that
result.json reads. It removes known token formats, credentials in URLs,
Authorization headers, and the values of token variables, the gh token,
and credentials in the global git configuration.

run_one moves to lib/runner.sh, so the unit tests can run it.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
actions-coverage showed #719 as "v4 → v6.0.3". The floating @v4 is
v4.4.0 today, and @v6 is v6.1.0, so the exact pin v6.0.3 misses a release.

The check now resolves each floating tag to the exact release on the same
commit and shows both, for example "v4 (v4.4.0) → v6.0.3". If an exact
pin is older than the release of its floating major, or of the floating
ref that it replaces, the check asks a person and recommends a newer pin.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
The #721 question named the input default: "Accept the changed input
defaults of actions/setup-python in go.yml (python-version: 3.x → none)?"
A person needs the effect instead.

actions-coverage now finds the step after the action and the runner image.
For a dropped python-version default, it asks: "Accept that pre-commit in
go.yml will use the runner's Python 3.12 instead of the newest 3.x in the
tool cache?" If .python-version exists, the question names that version.
Other changed defaults keep the old question.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
Co-authored-by: rniebla <rniebla@launchdarkly.com>
A composite action can change its nested actions in a patch release.
release-secrets v1.2.1 moved configure-aws-credentials from v4 to v6 and
replaced the SSM loader, and the check said that the interfaces match.
The check now compares the nested uses refs and run steps. A nested major
update, or a nested action that is added or removed, counts as breaking.
An action.yml that the check cannot fetch is now incomplete.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
On the cloud VM, /exec-daemon/npx crashed before prettier started. The
check recorded "prettier would reformat 0 file(s)", base crashed the same
way, and the result counted as pre-existing, so #870 was safe to merge
without a prettier run. ui-prettier now runs node_modules/.bin/prettier.
If prettier, eslint, vitest, the build, or npm ls exits non-zero and
reports no finding, the check is incomplete.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
SKILL.md says that needs-human means verification is complete, but the
verdict ranked needs-human above incomplete. On #719 the headline and the
exit code hid an open CI item. The verdict is now incomplete while any
item is open, and the comment still lists the questions.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
… change

On #869, an npm-only PR, the flaky TestStreamClientV2Handler failed the Go
job, and ci-status made the verdict block. verify.sh now records the failed
jobs and steps of each run. If every failed step is a run step that reads
only ecosystems that the PR does not change, the result is incomplete with
a rerun of the failed job as the fix.

ci-status also gives the age of the oldest run, and it is incomplete when
the base branch changed a workflow after the run (#719).

Co-authored-by: rniebla <rniebla@launchdarkly.com>
The comment told the reader to run verify.sh again for every incomplete
item. On #719 the open item was CI, which only a rerun on GitHub closes.
Each incomplete item now has a fix: the recommendation of its check (a
GitHub rerun for ci-status), the impact review, the generated check, or a
verify.sh rerun.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
The #621 run used a generated guard for this. go-generate-drift already
fails on main because of the stale server.gen.go (#720), so it cannot show
whether the mocks drift. The new check runs only for go.uber.org/mock
updates. It runs each mockgen directive, requires no diff, and requires one
directive for each committed MockGen file.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
…erage

On #719 one question asked about checkout v6 in release workflows and
about the exact pin v6.0.3. SKILL.md asks for one yes-or-no question per
decision. A check can now call ask for each choice and decide_asked at the
end. Each question becomes its own decision with its own evidence.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
cursoragent and others added 16 commits October 20, 2026 03:09
Runners already run node20 actions on node24, so actions-coverage now
says that node20 → node24 changes nothing (#719). It also reports a new
input that the repo does not set and that defaults to true, such as
setup-node v5 package-manager-cache (#865).

Co-authored-by: rniebla <rniebla@launchdarkly.com>
…eam-changes

- A floating ref such as @v7 now resolves to its release (v7.0.1) before
  the notes and the compare link are read. The old code read v7 as 7.0.0,
  dropped checkout v7.0.1 and setup-node v7.1.0, and kept releases that
  main already had.
- A CHANGELOG entry that says only "version bump only" is not a note, so
  the coverage is partial. For a meta package such as typescript-eslint,
  the check also reads the CHANGELOG of each re-exported package.
- When GitHub reports the refs as diverged, the details list the commits
  of the old ref that the new ref lacks (#719).

Co-authored-by: rniebla <rniebla@launchdarkly.com>
SKILL.md puts release tooling in the high tier, but the risk map gave
launchdarkly/gh-actions, docker/*, alpine, and actions/attest,
upload-artifact, and setup-node the medium tier. #863 came out medium.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
…cision

The playbook asks for the guard results next to the question. On #867 and
#865 the evidence under the question was only the actions-coverage
summary.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
…dence

Co-authored-by: rniebla <rniebla@launchdarkly.com>
SKILL.md asks the agent to give the reason when it raises the tier, but
result.json dropped tier_reasons from agent/impact.json. On #863 the
comment showed the high tier without the reason. result.json now has
tier_reasons, and the comment says why the impact review raised the tier.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
render-comment.sh printed the PR details, or the base details when the PR
had none. On #873 the comment showed the 3011 ms timeout but not the 8 ms
failure on base that made the check proven. When the two sides differ, the
details now show both.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
license-changes took the first base license for each package name. A
package can have copies with different licenses (minimatch 3 is ISC,
minimatch 10 is BlueOak-1.0.0), so an update of one copy could read as a
license change. Each new copy is now compared with the copy at the same
path. A copy at a new path is compared with the other copies of the
package, and the question names the package that the copy is for (#868).

Co-authored-by: rniebla <rniebla@launchdarkly.com>
samber/lo is used only in tests and is not in the binary, but it got the
medium tier of a runtime dependency (#874). classify.sh now reads go list
-deps . on base and PR. A direct module that neither binary contains is
test-only, which the tier rules treat like a dev dependency.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
transitive-changes called golang.org/x/sys v0.41.0 → v0.45.0 major, and
the comment called it minor (#873, #621). deps.py, the transitive list in
the comment, and the nested-action diff now all say 0.x minor.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
The details printed the output of file -b as it was, so text such as
flags:<NOUNDEFS|DYLDLINK|TWOLEVEL|PIE> was outside code, and each post of
#725, #828, #871, #874, and #621 needed a hand edit.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
… ldcli PRs

Bare npm scopes such as @launchpad-ui/core and @types/react-window became
GitHub mentions (#723), and an upstream #258 linked to ldcli PR #258
(#621). render-comment.sh now sends its Markdown through comment_safe.py,
which puts bare @-tokens and raw <...> text in backticks outside code. A
bare #N in an impact review changelog entry becomes owner/repo#N, with
the repository from the entry URL or the upstream-changes report. The
upstream commit subjects get the same prefix.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
The verify.sh help said that the full profile adds govulncheck. The
registry runs govulncheck in both profiles. The full profile adds
release-snapshot and golangci-lint.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
A base lockfile entry without a license field became "unknown", so the
#635 replay asked to accept a change from unknown to MIT. A missing base
license now gives no license to compare, as before. The #718 fixture now
expects the composite question: v1.2.0 moves the nested
configure-aws-credentials from v1-node16 to v4 in release-only workflows.

Co-authored-by: rniebla <rniebla@launchdarkly.com>
…update

Co-authored-by: rniebla <rniebla@launchdarkly.com>
Co-authored-by: rniebla <rniebla@launchdarkly.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants