Skip to content

chore: adopt the AI-native SDLC playbook and repository policies - #1193

Draft
goanpeca wants to merge 8 commits into
masterfrom
chore/ai-native-sdlc
Draft

goanpeca wants to merge 8 commits into
masterfrom
chore/ai-native-sdlc

Conversation

@goanpeca

@goanpeca goanpeca commented Oct 28, 2026 •

Copy link
Copy Markdown

Summary: adopt the AI-native SDLC playbook in the B2 CLI, adapted from Backblaze/blazer#151. towncrier stays as the changelog tool.

Motivation

Closes #1192.

Reviewers keep raising the same issues across PRs: missing or mistyped changelog.d fragments, tests that do not cover every input branch, unannounced exit-code and stderr changes, developer knobs in the main --help, and live-B2 integration flakiness blamed on the wrong PR. This PR records those lessons where agents and contributors read them, adds one local gate that mirrors CI, and adds the repo policy files that were missing.

Changes

  • Playbook: AGENTS.md (commands, architecture, ApiVer and output conventions, 15 recurring review findings), CLAUDE.md pointing to it, REVIEW.md (CLI-specific failure modes, Important / Nit / Pre-existing severities, file:line evidence, capped review rounds), intent/README.md with intent/spec/plan templates, the playbook skills in .claude/skills/, and a PostToolUse hook (.claude/settings.json, .claude/hooks/ruff-format.sh) that rejects Claude Code edits leaving a Python file unformatted.
  • Review lessons: doc/review-lessons.md groups what reviewers asked for across all 692 reviewer comments on the 571 merged PRs (inline, review and conversation comments, excluding authors' own replies and bots; Copilot had not reviewed any merged PR yet), by theme with PR links: compatibility and defaults, data safety, tests, help text, output and errors, B2 semantics, Python style, shell/CI/release, and PR scope. The strongest ones are items 12-15 in AGENTS.md, and REVIEW.md and CONTRIBUTING.md link to it.
  • Code review practices from How the Claude Code team does code review: .github/workflows/ai-review.yml runs an advisory AI code review (anthropics/claude-code-action, pinned SHA) and security review (anthropics/claude-code-security-review, pinned SHA, with repo-specific instructions in .github/security-review-instructions.txt) on every non-draft PR. Both skip Dependabot and are no-ops until ANTHROPIC_API_KEY is set and the Claude GitHub App is installed. REVIEW.md adds review depth by path (Quick / Standard / Deep), paths to skip, a root-cause and whole-repo check, verifying findings before posting, fixing repeated findings at the source, and what people still review. AGENTS.md asks for an end-to-end run plus /code-review, /simplify and /security-review before a PR, and RELEASE.md covers staged pre-release tags, one-step rollback and incident reviews. doc/reviewer-evals.md seeds a reviewer eval set with six real bugs, plus clean-PR and noise measures. nox -s review_metrics reports human approvers, comments and wait per merged PR, with reverts as the guardrail. The review-process intent adds a CODEOWNERS draft, circuit breakers, nightly review of existing code and staged releases.
  • noxfile.py: nox -s changelog_check [base] runs the CI "Validate new changelog entries" logic locally, also counting uncommitted and untracked fragments. nox -s ci runs codespell on the files the branch changes, then lint, towncrier_check, build, changelog_check, unit-3.12 and the offline integration tests, plus the live-B2 ones when B2_TEST_APPLICATION_KEY* are set (reported as NOT RUN otherwise). NOX_CI_PYTHON, NOX_CI_BASE and NOX_CI_SKIP_CHANGELOG adjust it. No existing session changes.
  • Repo policies: RELEASE.md replaces README.release.md (same two steps, plus prerequisites, what cd.yml publishes, and recovery), SECURITY.md, CODE_OF_CONDUCT.md, issue forms (bug, feature, task, plus a security contact link), a PR template with Risk and How to revert, .editorconfig, and new CONTRIBUTING sections on agents, local CI, releases and security.
  • intent/review-process/intent.md: a draft proposal for the maintainers covering what needs repo settings: risk tiers by path, re-approval after new commits, required conversation resolution, automated AI and security review, a merge queue, reviewer evals built from past bugs, and metrics.

No CLI code or CI workflow changes.

Testing

  • nox -s ci on Python 3.12, macOS arm64, without B2 credentials: codespell (changed files), lint, towncrier_check, build, changelog_check, unit-3.12 (509 / 513 / 512 passed for b2v3 / b2v4 / b2v5) and offline integration-3.12 (4 passed per ApiVer) all passed.
  • Integration tests with secrets: NOT RUN (no live-B2 credentials locally).
  • changelog_check: fails against HEAD with no new fragment, passes once an untracked fragment exists, and stops with a clear message for an unknown base ref.
  • ruff hook: exit 2 with a fix message on an unformatted file in the repo, exit 0 on a formatted file, a non-Python file, or a file outside the repo.
  • A fresh-context review subagent reported 4 Important and 7 Nit findings, all fixed in 466262e: the wrong b2_binary_name rule (commands use self.console_tool.b2_binary_name), nox -s lint described as checking formatting (it reformats in place), ci failing on non-master bases, -changelog PRs and uncommitted fragments, missing-ref tracebacks, and doc accuracy fixes.

Risk

Docs and tooling only. The riskiest part is the new ci and changelog_check nox sessions, which run only when someone calls them. README.release.md is renamed to RELEASE.md, so external links to it break.

How to revert

Reverting the commits is enough; nothing published depends on these files.

Follow-ups needing maintainers or repo settings

  • Fix the codespell step in ci.yml: ignore_words_list: datas re-use is space-separated, so the action runs codespell --ignore-words-list datas re-use, which checks only a path named re-use. CI has not been spell-checking anything. Using datas,re-use will surface about five existing typos to fix in the same PR.
  • nox -s lint runs ruff format without --check, so CI never fails on formatting. Consider ruff format --check in lint.
  • Enable private vulnerability reporting (the SECURITY.md and issue-form links depend on it).
  • Decide whether to add .github/CODEOWNERS.
  • Decide whether Dependabot PRs should keep running the full matrix with live-B2 secrets.
  • Add the ANTHROPIC_API_KEY secret and install the Claude GitHub App to turn on ai-review.yml (advisory at first).
  • Fill in owners for the CODEOWNERS draft in intent/review-process/intent.md.
  • Review and accept or reshape intent/review-process/intent.md.

Checklist

  • nox -s ci passes locally.
  • A fresh-context review ran on the diff; findings are fixed (see Testing).
  • changelog.d fragments exist (+ai-native-sdlc.infrastructure.md, +review-lessons.doc.md, +ai-review.infrastructure.md).
  • Documentation is updated.
  • The description matches the final code.

🤖 Generated with Claude Code

Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The CI gate can silently skip spelling checks for filenames with spaces or non-ASCII characters and fails to propagate the selected Python version to lint, validation, and build sessions.

2 open issues
What changed in this PR

Adopts the AI-native SDLC playbook for the B2 CLI, addressing #1192 with contributor guidance, local verification tooling, and repository policies. No CLI behavior or CI workflow changes are introduced.

Changes:

  • Adds agent guidance, review procedures, planning templates, reusable skills, and a Python formatting hook.
  • Adds local ci and changelog_check Nox sessions while retaining towncrier.
  • Documents release and security processes and adds contribution templates and community policies.

Review: 2 Important, 0 Nit, 0 Pre-existing. Checked the supplied changes against #1192 and inspected Nox interpreter selection. Tests and live-B2 checks were not run.

File Description
test/​unit/​console_tool/​test_upload_file.py Formatting-only change to test arguments.
SECURITY.md Defines vulnerability reporting and security guidance.
REVIEW.md Defines review passes, evidence, and resolution rules.
RELEASE.md Expands release steps and recovery guidance.
README.release.md Removes instructions replaced by RELEASE.md.
noxfile.py Adds local CI and changelog checks.
intent/​review-process/​intent.md Proposes risk-based review and repository settings.
intent/​README.md Provides intent, spec, and plan templates.
CONTRIBUTING.md Documents agent workflows and local verification.
CODE_OF_CONDUCT.md Adds community standards and reporting guidance.
CLAUDE.md Points Claude Code to AGENTS.md.
changelog.d/​+ai-native-sdlc.infrastructure.md Records the tooling and policy additions.
AGENTS.md Centralizes commands, architecture, and conventions.
.github/​pull_request_template.md Prompts for testing, risk, and rollback details.
.github/​ISSUE_TEMPLATE/​task.yml Adds a maintenance task form.
.github/​ISSUE_TEMPLATE/​feature_request.yml Adds a feature request form.
.github/​ISSUE_TEMPLATE/​config.yml Routes security and service reports.
.github/​ISSUE_TEMPLATE/​bug_report.yml Collects reproduction and environment details.
.editorconfig Defines shared editor formatting defaults.
.claude/​skills/​sdlc-subagents/​SKILL.md Guides parallel sessions and scoped helpers.
.claude/​skills/​sdlc-spec/​SKILL.md Guides requirements and design specifications.
.claude/​skills/​sdlc-pr-review/​SKILL.md Guides structured review and comment resolution.
.claude/​skills/​sdlc-policy-skill/​SKILL.md Guides policy skills and automated enforcement.
.claude/​skills/​sdlc-plan/​SKILL.md Guides implementation planning and verification.
.claude/​skills/​sdlc-intent/​SKILL.md Guides problem and outcome capture.
.claude/​skills/​sdlc-feedback-loop/​SKILL.md Guides test-first fixes and verification evidence.
.claude/​skills/​sdlc-claude-md/​SKILL.md Guides agent-context documentation maintenance.
.claude/​skills/​ai-native-sdlc/​SKILL.md Routes work through the playbook stages.
.claude/​settings.json Registers the post-edit formatting hook.
.claude/​hooks/​ruff-format.sh Checks Python formatting after agent edits.

Review effort: Balanced

Comment thread noxfile.py Outdated
Comment thread noxfile.py Outdated
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The blanket ban on printing credentials conflicts with existing command contracts and needs an explicit exception for documented credential output.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Low severity Allow documented credential output in security checks

REVIEW.md:38

Nit — Security: The unconditional ban on printing credentials conflicts with existing command contracts: AccountGetBase deliberately prints the application key and auth token (b2/_internal/console_tool.py:2077-2088), and key creation prints the new key at line 1782. Scope this check to unintended disclosure, with an explicit exception for documented credential output, so following the review policy does not require breaking those commands.

2 resolved since last review

Review effort: Balanced

Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
@goanpeca

Copy link
Copy Markdown
Author

Re Copilot review "Allow documented credential output in security checks" (REVIEW.md:38): fixed in 472dab7. The Security pass now bans credentials in logs, error messages and stderr, and on stdout only where a command does not document it, with account get and key create named as the documented exceptions; a new command or flag that prints them must say so in its help. The same exception is in .github/security-review-instructions.txt for the AI security review.

🤖 Addressed by Claude Code

Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adopt the AI-native SDLC playbook and repository policies

2 participants