Repository navigation
feat: actionability gate round 2 — prefix tier + deferral vocabulary, wired into all four consumers (#316) - #318
Conversation
Verify Report — PR #318Enginepai-ensemble 2.20.0 (canonical #207) — 4 IDD lenses + DA + Codex (gpt-5.6-sol), model: opus AggregateFAIL — 2 CRITICAL, 21 HIGH blocking; 28 MEDIUM / 11 LOW follow-up Frozen snapshot Scope coveragePR refs: #37 #84 #136 #292 #298 #306 #309 #310 #316 #317 — 8 已 CLOSED/MERGED(敘述脈絡)、#317 為獨立 follow-up(PR body 自述不在本次修復範圍) #316 — bug: #298 的修正只落在 idd-listRequirements coverage: 部分達成,但兩個 CRITICAL 使核心宣稱未成立。 CRITICAL(blocking,兩者互相獨立)C1. 三訊號 gate 從未接進四個 consumer — parking-lot / Blocking 仍被繞過lens: Codex 的首要發現,也是它給出「建議修改後再合併」的理由: 問題:
對照 spec( 觸發情境: 後果:parser 回 exit 0,consumer 照 tier routing 直接執行。fixture 自己描述的 建議:(1) 每個 consumer 讀 labels 與 body 的 Blocking section;(2) 一律呼叫 C2. 封閉值域與本 repo 真實語料不符:90 筆真實 diagnosis 有 38 筆域外,其中 22 筆是舊 parser 路由正確、新契約改成 hard abort 的正常 issuelens: 四位評審都只讀 diff,沒有人拿真實資料跑過這個 helper。我跑了。 方法: 結果:
真實值長這樣(逐字,非我編的): 這推翻了本 change 的因果診斷。design/proposal 認定「截斷是 bug」,但在真實語料上截斷 22 次是對的、1 次是錯的(#200 那一類才是錯的)。把截斷整個拿掉、改成「任何後綴文字=域外=停止」,是用一個 24%(22/90)的 false-stop 去換一個 10%(9/90)的 false-go —— 而且 false-stop 是硬停,比原本的靜默誤路由更容易讓整條 pipeline 停擺。design 明文否決的替代方案 (b)「只加寬 parser」,才是符合資料的那一個。 proposal 的 BREAKING 估計也是錯的: 而且流量端沒有任何強制力:新的 producer 規則只是 重現: . plugins/issue-driven-dev/scripts/lib/actionability.sh
gh issue view 264 --json comments --jq '[.comments[]|select(.body|test("^## Diagnosis";"m"))]|last|.body' \
| python3 -c 'import sys,json;print(json.load(sys.stdin))' >/tmp/b # 或直接傳 body
idd_parse_complexity "$(cat /tmp/b)"; echo $? # → 3, unparseable-complexity: Spectra(opt-out → 直接 propose)HIGH(blocking)
MEDIUM / LOW(follow-up 候選)MEDIUM ×28
LOW ×11
INFO ×7含 4 份獨立的 prompt-injection 掃描結果(皆 clean)。 Scope Check有 scope creep:M44 指出一個 routing-parser 變更同時改了 IC_R011 sister-sweep 的 label 慣例( Lens 統計
|
`### Complexity` 從沒宣告過封閉值域,也沒有 unparseable 契約 —— 而結構相同的姊妹欄位 `### Conflict Class` 兩者都有。於是 producer 寫出 `Simple when triggered` 不違反任何明文 規則,三個 consumer 各自發明互不相容的窄化:idd-list 靜默截成 `Simple`、idd-all 與 idd-implement 得到非法字串。2026-08-10 實測本 repo 的 22-issue backlog,11 個 diagnosed 裡 8 個 parked/deferred 被判成「Actionable now」,其中兩個帶著使用者親自下的 defer 裁決。 根因不是 parser 太窄,是**把會變的狀態存進不可變的 artifact**。Complexity 活在 append-only 的 Diagnosis comment,parked 卻會變(trigger 成立就該解除)。#136 的 comment 寫 bare `Spectra`、body 寫 `Spectra when triggered (parking lot)` —— 狀態改不動就自己漂到別處。 IDD 早就知道正確做法:`### Blocking` 正因為會變才放在 body 由 idd-update 維護。 - tier 值域收斂為封閉四值(可帶既有 ` via <來源>` 後綴),延期狀態遷至 parking-lot label - 三訊號 gate(域外值 / label / Blocking 區塊),放行需三者皆不成立 - 解析與判定抽成 scripts/lib/actionability.sh,四個 consumer 全部改為引用 + fail-loud guard —— 沒有 guard 的話,「找不到就自己想辦法」的 consumer 會把三方分歧原封帶回來 三個刻意的取捨: Strategy 的 `- [~]` **不進 gate**。它的既有 consumer 是 idd-close 的 checklist gate,語意是 per-item 的 close-time disposition;拿它回答 per-issue 的 routing 問題會撞壞既有語意。 migration **只貼 label、不改寫歷史**。回填既有 Diagnosis comment 會 modify-in-place 一個 append-only artifact;而且不必要 —— legacy 值走 unparseable 路徑得到的正是「not-actionable + surface 原值」,對那批 parked issue 就是正確結果。實測當前 open backlog 域外值 0 筆。 producer **不自動貼 label**。限定詞是 diagnose 當下的判斷,label 是人的裁決,兩者會分岔 (11 筆實測只有 5 筆一致)。由前者推導後者,等於取消「人可以事後 park 一個 tier 明確的 issue」這條路徑。 顯示層仍分兩組:reason 僅為 blocking 者維持 #84 既有的 Blocked 分組(標題/banner/footer 逐字不變),其餘進 Parked 組。gate 統一不等於顯示統一。 ic-r011 的 `blocker:infeasible` / `blocker:waiting` 一併退役 —— 0 個 issue 在用、無任何 consumer 讀,(b)/(c) 的區分保留在 audit 字串(那是說明性資訊,不是機械判定用)。 47/47 suites 綠(新增 actionability-gate 66 條,fixture 含 2026-08-10 事故快照, 斷言 9 筆中只有 #37 為 actionable)。 Refs #298, #310, #317
… 定形 (#316) 第 1 輪的封閉值域被 /idd-verify --pr 318 CRITICAL-2 證偽:90 筆真實 diagnosis 有 38 筆 落在值域外,其中只有 9 筆是延期修飾語,22 筆是「tier + 同行理由」的正常寫法。分界畫錯了 位置 —— 不在「後面有沒有字」,而在「那些字是否表達延期」。 /idd-reorganize #316 裁定:design D1/D3/D5、spec R1/R3/R7 redo;D2/D4/D6/D7/D8 still-valid。 /idd-diagnose #316 第 2 輪以完整 corpus(225 issue 中 159 筆有 Diagnosis)重新定規則: 剝裝飾 → 取 tier prefix → 對整個值掃延期語彙。158/158,0 false positive。 本 commit 只動 Spectra artifact(ingest 產物),不動 code: - design.md:三個決策重寫並留「前版為何被推翻」註記;新增「風險姿態:label 為主、 語彙為輔」——延期語彙是開放列舉,漏抓有 label 兜底、誤抓會硬停正常工作,所以取高精度 - spec.md:R1/R3/R7 重寫,延期偵測獨立成 requirement(7 → 8 條);reason 值域 4 → 5, 新增 complexity-deferral-marker —— 與 unparseable 分離,因為人的處置不同(前者確認 label、後者修 diagnosis) - proposal.md:BREAKING 敘述移除。「9 筆需 migration」是從有偏樣本推的;新規則對既有 全部語料都對,migration 為零 - tasks.md:保留 10 [x] + 2 [~],新增第 6–9 組共 10 個 task。7.1 明寫「只換 parser 不 呼叫 verdict 等於沒修」—— 第 1 輪 gate 完整實作、66 測試全綠、零 consumer 呼叫 Refs #298, #310
a2e7de0 to
dbadd38
Compare
…inate (#316) rebase onto main 3.0.0 後重看 pending task:main 於 2026-08-15 post-merge audit 為 idd-list 記下 `--json comments` 硬截最舊 100 則的問題,並明寫「不得只在其中一個 consumer 修」。 本 change 的三個 consumer snippet 有同一個洞 —— latest Diagnosis 正是 >100 則時被丟掉的 那則,helper 會誤回 complexity-missing。修法沿用 check-closed-without-summary.sh 的現成實作。 Refs #335
…gate wired into all four consumers Refs #316 (round-2 re-baseline of PR #318) Round 1 declared a closed value domain for `### Complexity` and shipped a three-signal gate that no consumer called; /idd-verify --pr 318 returned FAIL (CRITICAL-1 unwired, CRITICAL-2 domain refuted by the 159-diagnosis corpus: 66 real values wrongly refused). Helper (scripts/lib/actionability.sh) - tier = leading whole word among Simple|Plan|Spectra|SDD-warranted after decoration strip; rationale / parenthetical / ` via` suffix are legal - whole first line scanned for deferral vocabulary -> exit 5, stderr `deferral-marker: <raw>`, no stdout (tier prefix never routable) - reason vocabulary 4 -> 5 (`complexity-deferral-marker`, group = parked) - new idd_blocking_section (idd-update's `- (none)` placeholder is empty) - shared fence-aware section extractor (``` / ~~~; same-or-higher heading ends) - value-less flag -> named exit 2 (was an infinite loop) Consumers (idd-list / idd-all / idd-implement / idd-plan) - actually call idd_actionability_verdict with all three signals - paginate the Diagnosis fetch (gh issue view --json comments = oldest 100) - set -euo pipefail-safe conditional capture; exit 2 is FATAL, not withheld - idd-list: Parked group added; #84 blocked heading/banner/footer verbatim Tests (225 assertions) - fixture rows now verbatim Diagnosis comment values; corpus-sampled shapes >=3 each; #128 pinned as the designed prose-deferral miss - frozen 159-issue corpus regression: 149 / 9 / 1 / 0 - strict-mode integration test; consumer-wiring drift guard Docs: references/actionability-gate.md rewritten (risk posture, corpus table, canonical call shape); idd-diagnose producer declaration rewritten; CHANGELOG 3.1.0; version 3.0.0 -> 3.1.0.
Verify Report — PR #318 (round 2)Enginemanual fan-out (4 lens Agents + sequenced DA, model: opus, file-based output) + Codex (gpt-5.6-sol) — Codex leg FAILED (codex-call HTTP 429 AggregateFAIL — 6 blocking, 10 in-scope fixes, 5 follow-ups Round 1's two CRITICALs are genuinely repaired and every lens mutation-tested it (renaming the verdict call in each SKILL.md turns the drift guard red; removing the deferral scan / leaking the tier on exit 5 / dropping fence tracking turn 73 / 22 / 6 assertions red). The DA independently re-implemented the reference's prose rule in Python and got 159/159 on the frozen corpus, so the corpus is not a tautology. The 21 round-1 HIGH findings check out on the named paths. But round 2 repeated CRITICAL-2's failure shape on the third signal: Scope coveragePR refs: #316 Process Gaps
#316 — bug: #298 的修正只落在 idd-list — 另三個 Complexity consumer 未動,且 idd-list 自身 Step 5 與 Step 3.7 互相矛盾Requirements coverage: spec R1 / R3 / R5 / R7 FULLY; R2 / R4 / R6 / R8 PARTIALLY (details in rows 6, 5, 2, 13 below). Issue Expected 6/6 FULLY; Acceptance criteria 6/9 FULLY, 3 knowingly superseded by the round-2 re-baseline but not annotated in the issue body (row 22). Blocking
In-scope fix (same PR, non-blocking)
Follow-up (beyond #316)
Scope CheckNo scope creep. CHANGELOG 3.1.0, the version bumps, the marketplace description and the PR-body rewrite are in-scope round-2 accompaniments; the ic-r011 label convergence is an acceptance criterion of #316 (and is what row 4 finds half-done). External-writes block for #316 read successfully by all five reviewers; it contains no instruction-shaped text and its two IC_R011 exemption claims match the diff — with one honesty gap: the sister sweep covered files changed, not contracts affected (rows 2 and 4 are divergences this round created). Verified as sound (so round 3 need not re-check)
NextRound-3 fixes in this PR (rows 1–18), then re-verify with Codex once its usage limit resets. Follow-up Findings Filed (v2.72.0+ #148)Filed by default per IC_R011 §1.1 from the Follow-up (beyond #316) rows above:
Rows 1–18 are being addressed in this PR (round 3). Codex re-run pending its usage-limit reset. |
…nosed group, gate before egress Refs #316 (round-3 repairs for the /idd-verify --pr 318 round-2 FAIL: 6 blocking + 10 in-scope) Helper (scripts/lib/actionability.sh) - idd_blocking_section reads the section PER BULLET (any non-placeholder bullet is a blocker; continuation lines belong to the bullet above) and recognises a placeholder by its LEADING TOKEN (none / n/a / 無, optionally bulleted, decorated, parenthesised, then EOL / ')' / separator). Round 2 anchored the whole line and withheld 31 of 47 empty sections, including #316's own `- (none — 可動)`. - trailing CR stripped before either section reader judges a line (GitHub web textarea submits CRLF) - idd_actionability_group gains `undiagnosed` (complexity-missing alone) - new frozen fixture corpus-blocking.json: all 55 `### Blocking` sections in this repo's issue bodies, hand-reviewed (47 empty / 8 non-empty), 0 FP / 0 FN; two rival rules measured there and rejected (documented in the helper) Consumers - idd-implement: gate moved to Step 0.35, ahead of tree-lock, branch creation and the Implementation Plan comment; test pins the line order - idd-list: Needs-diagnosis group keeps `→ /idd-diagnose #N`; Parked is now exactly the --parked set; labels/body/comments taken from the Step 2 bulk fetch (paginate only at >= 100 comments); open-state guard; per-row fetch failure marks the row instead of aborting; REASONS reset; C0 stripped - all four: issue number digit-checked before the REST path; Diagnosis comments trusted only from OWNER/MEMBER/COLLABORATOR; jq/python3 in allowed-tools; table header VEXIT · CEXIT · TIER - idd-all: Layer V sub-issue scan and Phase 3b.1 fallback paginate Contract / producer - references/actionability-gate.md: signal-3 risk posture, undiagnosed group, corpus denominator (66/225 never diagnosed), scoped producer prohibition, hardened canonical snippet, "gate SHALL precede any egress or branch" - idd-diagnose: prohibition scoped to the issue under diagnosis - idd-issue + spec delta idd-ic-r011-checkpoint: blocker:* retired for parking-lot - rules/sdd-integration.md: parallel bare-token parse narrative removed Tests (285 assertions): blocking corpus 55/55; CRLF both readers; per-bullet cases incl. `- none of the reviewers replied yet`; row 905 really emits a `- [~]` Strategy item; drift guard pins verdict capture shape, REASONS branch, FATAL branch, digit check, author filter, allowed-tools, gate-before-branch and gate-before-egress order, no `=$(gh issue view … --json comments)` left. Spec: R6 three groups, R7 scoped, new R9 (blocking signal per bullet, frozen corpus); design round-3 decisions + open question (#336); tasks 10.x. Follow-ups filed from the verify: #336 #337 #338 #339.
Round 3 pushed —
|
| Row | Change |
|---|---|
| 1 (CRITICAL) | idd_blocking_section reads per bullet with a leading-token placeholder rule; frozen corpus-blocking.json (55 real sections, hand-reviewed) is 0 FP / 0 FN. #316's own - (none — 可動) now reads empty; - (none) followed by - 等 … reports the blocker. The two rejected candidate rules are recorded in the helper. |
| 2 (CRITICAL) | undiagnosed group: idd-list shows Needs diagnosis (N): with → /idd-diagnose #N; Parked is now exactly the --parked set. Spec R6 amended. |
| 3 (HIGH) | idd-implement gate moved to Step 0.35 (before tree-lock, git checkout -b, and the Implementation Plan comment); contract clause + line-order guard. |
| 4 (HIGH) | Producer prohibition scoped to the issue under diagnosis; blocker:* retired in idd-issue and, via a spec delta, in the live idd-ic-r011-checkpoint spec. |
| 5 (HIGH) | Trailing \r stripped in the shared extractor; CRLF fixtures for both readers. |
| 6 (HIGH) | Drift guard pins the verdict conditional-capture shape, the REASONS= branch and the FATAL … misuse branch in all four consumers. |
| 7–18 | idd-all Layer V scan + 3b.1 fallback paginate; idd-list reuses its bulk fetch, open-state guard, per-row fetch failure; sdd-integration parallel narrative removed; digit check + author filter in all four snippets and the reference; allowed-tools jq/python3; row 905 emits a real - [~]; signal-3 risk posture, corpus denominator and mention-class recorded; fixture rows #160/#136 verbatim; ≥3-per-shape counts real rows only; table headers; REASONS reset; C0 strip; 158/158 → 159/159; plugin.json description synced. |
Suite: actionability-gate 285/285; 53 suites green except plan-routing-consistency (#335, environment-dependent). Follow-ups filed: #336 #337 #338 #339; acceptance-criteria errata posted on #316.
Not yet re-verified: Codex is under its usage limit until ~2026-09-08 14:00 (+08:00). A round-3 /idd-verify --pr 318 should run with the cross-model leg present.
Verify Report — PR #318 (round 3)Enginemanual fan-out (4 lens Agents + sequenced DA, model: opus, file-based output) + Codex (gpt-5.6-sol) — Codex leg absent again (HTTP 429 AggregateFAIL — 6 blocking, 11 in-scope fixes, 5 follow-ups Round 2's six blocking rows were re-measured by every lens (live Scope coveragePR refs: #316 · Verified scope: #316 Process Gaps
#316 — bug: #298 的修正只落在 idd-list — 另三個 Complexity consumer 未動,且 idd-list 自身 Step 5 與 Step 3.7 互相矛盾Requirements coverage: spec R3 / R4 / R7 / R8 FULLY; R1 / R2 / R5 / R6 / R9 PARTIALLY (rows 2, 3, 5, 10, 11 below). tasks §10: 10.5 FULLY, 10.1–10.4 / 10.6 PARTIALLY. Blocking (round 4, this PR)
In-scope fix (same PR)
Follow-up (beyond round 4)
Scope CheckNo scope creep: Verified as soundRound-2 rows 2, 6, 7, 9, 10, 11, 12, 13, 14, 18 FULLY on re-measurement; DA's premise ruling (recorded verbatim in spirit)Round 3 is in the regex rabbit hole (three rounds, each adding a character class, each growing a new hole). Round 4 should fix correctness and honesty, not coverage: repair the regressions (rows 1, 3), close the one-line fail-opens (rows 4, 6), make the verdict observable (row 2), bring every claim down to the evidence (rows 5, 7, 8), and hand #336 a corrected acceptance bar. Neither "tune the regex again" nor "demote signal 3" — the latter would be a mid-verify scope change against #84, and with a live effect of n = 1 the decision can be made calmly in #336. NextRound-4 fixes in this PR (rows 1–15), then a fresh verify — with Codex once its limit resets. Follow-up Findings Filed (v2.72.0+ #148)
Rows 1–15 addressed in round 4 (this PR). |
…the verdict, claims down to evidence Refs #316 (round-4 repairs for the /idd-verify --pr 318 round-3 FAIL: 6 blocking + 11 in-scope) Regressions repaired - idd-implement Step 2.5 consumed gate variables across Bash calls and forbade re-running; now checks ${VEXIT:-} and re-runs the Step 0.35 block with the same helper (the shape idd-all 3b.1 already had) - idd-list state guard used the listing --state flag (--state all skipped every issue, Suggested-next vanished); now per-issue .state == "OPEN", and `skipped` rows keep the phase x PR matrix - placeholder rule was locale-dependent (multibyte separators in a bracket expression flipped both directions under LC_ALL=C; the suite itself failed 3 assertions); rewritten as alternations, suite also run under LC_ALL=C - an unclosed code fence swallowed every later section (live #290); unbalanced fences now disable fence tracking for that body Observability / hygiene - the gate block prints `gate #N: VEXIT=… TIER=… REASONS=… | …` in all four consumers and the canonical shape (a silent exit 0 was indistinguishable from actionable to the executing model) - C0/DEL scrubbed at the helper's outputs (round 3 only described it, and the described set omitted \r); idd_actionability_group "" is exit 2 - REASONS reset on the actionable path; routing table rows carry all three keys; idd-plan gains allowed-tools and drops the bare --json comments fetch; idd-all Layer-V sub-issue scan digit-checks + author-filters; _idd_section_first_line no longer pipes into head (SIGPIPE under pipefail) Claims brought down to the evidence - corpus-blocking.json rows carry the original body so the extractor is exercised; provenance: 54/55 agree with hand review, #1 an accepted FP, 54/55 CLOSED issues the gate never evaluates (signal 3's live effect on 2026-09-07 was one issue) - accepted misses stated as a rule in both directions; the two rejected placeholder regexes recorded with FP/FN counts; "parked group == --parked set" claim deleted from spec R6 and the reference; cluster-path coverage gap (first issue only, pre-existing) stated and tracked in #340 - spec delta: `respectively` residue, nested backticks and a mis-attributed @trace removed; design/proposal/tasks/CHANGELOG synced; #336 acceptance bar is the semantic truth, not the fixture Tests: 318 assertions (LC_ALL=C probes, unclosed-fence probes, scrub probes, extractor-through-corpus, verdict-printed / REASONS-reset / re-run-fallback / per-issue-state / skipped-rule / sub-issue-hygiene drift guards). Follow-ups: #340 filed; reader-semantics items handed to #336.
Round 4 pushed —
|
| Row | Change |
|---|---|
| 1 (CRITICAL) | idd-implement Step 2.5 checks ${VEXIT:-} and re-runs the Step 0.35 block with the same helper when the variables did not survive the Bash-call boundary; the prohibition now reads "no private regex", not "no re-run". |
| 2 (HIGH) | The gate block prints gate #N: VEXIT=… TIER=… REASONS=… | … in all four consumers and the canonical shape; drift guard pins it. |
| 3 (HIGH) | idd-list guards on per-issue .state == "OPEN"; skipped rows keep the phase × PR matrix; the listing-flag guard is refuted by test. |
| 4 (HIGH) | Placeholder rule rewritten with multibyte separators as alternations; the suite runs the probes under LC_ALL=C (and the whole suite passes under LC_ALL=C). |
| 5 (HIGH) | Claims restated everywhere: 54/55 agree with hand review, #1 accepted FP, 54/55 CLOSED, extractor covered via original bodies; #336's bar is the semantic truth (48/7); the two rejected regexes are recorded with FP/FN. |
| 6 (HIGH) | Unbalanced fences disable fence tracking for that body; #290's section is found; balanced fenced examples still skipped. |
| 7–15 | C0/DEL scrub moved into the helper's outputs (CR included); accepted misses stated as a rule in both directions; cluster gap stated in the contract → #340; "parked group == --parked set" deleted; spec delta cleaned (respectively, nested backticks, @trace); idd-plan allowed-tools + Step 1 fetch; idd-all sub-issue digit check + author filter; table rows three-keyed; REASONS reset; idd_actionability_group "" → exit 2; no head in the first-line reader; 159/159 wording; proposal Modified Capabilities / Impact. |
Suite: actionability-gate 318/318 (also under LC_ALL=C); 53 suites green except plan-routing-consistency (#335). Follow-ups: #340 (cluster-path gate); reader-semantics items commented on #336.
Not yet re-verified: Codex is under its usage limit until ≈ 2026-09-08 13:30 (+08:00). A round-4 /idd-verify --pr 318 should run with the cross-model leg present — three consecutive 5-of-6 runs is one too many.
Verify Report — PR #318 (round 4)Enginemanual fan-out (4 lens Agents + sequenced DA, model: opus, file-based output) + Codex (gpt-5.6-sol via pai AggregateFAIL — 9 blocking (1 CRITICAL / 2 HIGH / 6 MEDIUM), 12 in-scope fixes, 2 follow-ups Round 3's fifteen rows were re-measured by every lens (live Scope coveragePR refs: #316 · Verified scope: #316 Process Gaps
#316 — bug: #298 的修正只落在 idd-list — 另三個 Complexity consumer 未動,且 idd-list 自身 Step 5 與 Step 3.7 互相矛盾Requirements coverage: spec R1–R8 FULLY; R9 (The blocking signal is read per bullet against a frozen corpus, Round-3 rows 1–15, re-measured
Blocking (round 5, this PR)
In-scope fix (same PR)
Added by the cross-model leg (Codex, in-scope)
Follow-up
Refuted (no action)
Round 5 prescription (DA, adopted)Two of the three helper rule changes this round had no measured need. Applied to a worktree, the two reverts below are a 4-insertion / 7-deletion drop-in: suite goes 313/5, and the five failures are exactly the five round-4 assertions (
Gate for any future reader-rule change to Baseline measurements (all re-run this round, none taken from the green suite)
External writes (#315)
Cross-model leg (Codex gpt-5.6-sol, effort high, round-4 diff minus the fixture file) — corroboration mapCodex reviewed blind (no access to the five Claude findings files) and returned FAIL with 2 HIGH / 3 MEDIUM / 3 LOW. Every Codex finding lands on a Claude row or adds one; no Claude blocking row was contradicted.
Codex's round-4 claim table agrees with the Claude re-measurement above on all 13 claims (成立 / 部分成立 / 未完全成立 line up with real / half). On out-of-diff writes it found #340 / #336 / #316 consistent with the diff, flagged that a "Round 4 pushed" comment claiming all six items fully repaired is partly refuted by rows 1–3 and 23, and read the Follow-up Findings Filed (v2.72.0+ #148)
Rows 1–9 (blocking) and 10–19, 23–24 (in-scope) → round 5, this PR. Refuted with no action: regression R4-M3. Verified snapshot |
…er widenings, delimit the printed verdict (#316) Change: add-actionability-gate Tasks: 28/28 complete (3 skipped with reason) Refs #316 (round-5 repairs for the /idd-verify --pr 318 round-4 FAIL: 9 blocking + 12 in-scope) Reverted (no measured need; both opened the fail-open direction) - odd-count fence heuristic: a closed ``` block containing a ~~~ line, or a fenced example followed by one stray opener, exposed the fenced template as the real section (Plan when triggered routed as Simple). Its motivating body got the same gate verdict either way; CommonMark renders its section as code. Fences now follow CommonMark; the EOF-probe variant was measured and rejected. - `.` / `。` as placeholder terminators: `- None. Waiting on X` read as empty. - the five round-4 assertions that pinned both, replaced by fence shapes A/B/C and "a full stop is not a terminator". Repaired - the gate prints a machine line of closed-vocabulary fields, then the raw third-party text inside raw<<< ... >>>raw, every line indented (4 consumers + canonical shape) - idd-implement Step 2.5 repeats the Step 0.35 block verbatim instead of echoing - reader greps pin LC_ALL=C; two-locale comparison on one input - group misuse message scrubbed; scrub comment states what it does not do - corpus claims test-locked: semantic_empty per row, 48/7, disagreement {1}, 54 CLOSED, frozen section == extractor output - idd-list: group call guarded against exit 2, `skipped` in Step 5's closed enumeration, every error exit prints why, body read under its own guard - idd-all digit check before the first gh call; idd-plan Simple row 3 keys - comments drift guard: one pattern, any --json field position, self-tested - honesty: bullet-class "measured" restated as a design judgment; rejected regexes on one denominator; 31 of the 48; 11/14 dated; @trace kept and updated - change gate recorded: a reader-rule change must flip a measured row or revert
…the change gate as closed lists (#316) Change: add-actionability-gate Tasks: 28/28 complete (3 skipped with reason) Refs #316 (self-review before freezing the round-5 verify) Two contradictions round 5 introduced, caught before verify: - The change gate allowed only "flip a measured row" or "revert"; the same round's LC_ALL=C pin is neither, so the gate as written refused its own commit. Now a closed list of three forms: (a) flip a measured corpus row, named in the commit; (b) revert; (c) remove an execution-environment dependence a pinned test demonstrates, every corpus row unchanged. - "Fences SHALL be interpreted as CommonMark interprets them" was false. The line-based reader follows CommonMark's two visibility rules (fenced example is never a section; unclosed fence runs to the end) and departs in three measured ways: fence length not compared, indentation not checked (fail-open), a closer with trailing text accepted. Each is now listed with its direction and pinned as a DOCUMENTED DIVERGENCE; on the live snapshot (243 bodies, 164 Diagnosis comments) reader and CommonMark disagree on section presence zero times, so the gate keeps the behaviour.
Verify Report — PR #318 (round 5)Enginemanual fan-out (4 lens Agents + sequenced DA, model: opus, coordinator-persisted findings) + Codex (gpt-6.1-sol, effort medium, via pai AggregateFAIL — 5 blocking (MEDIUM), 9 in-scope fixes, 3 follow-ups, plus one constraint on round 6 rated HIGH Round 5's reverts are correct and complete, and every lens re-measured that independently: the odd-count fence heuristic and the Scope coveragePR refs: #316 · Verified scope: #316 Process Gaps
#316 — bug: #298 的修正只落在 idd-list — 另三個 Complexity consumer 未動,且 idd-list 自身 Step 5 與 Step 3.7 互相矛盾Requirements coverage: spec R1–R8 FULLY. R9 (The blocking signal is read per bullet against a frozen corpus) PARTIALLY — three of its new SHALLs are not met: "SHALL NOT depend on the process locale", "every remaining way the reader departs from CommonMark fences SHALL be listed … and pinned by test", and "control characters SHALL be removed from every output the helper writes". tasks 10.9: the reverts, the print, the corpus pins, the consumer fixes and Round-4 rows, re-measured
Blocking (round 6, this PR)
Constraint on round 6 (Devil's Advocate, HIGH — a prescription risk, not a defect at
|
| # | Severity | Finding | Source | Action |
|---|---|---|---|---|
| 1 | LOW | idd-list :593 calls blocked / parked / error / skipped "a closed list of four" and omits undiagnosed, contradicting :589 ("only group=actionable uses the table"); a diagnosed-phase issue whose Diagnosis lacks ### Complexity falls through to 推不出 → /idd-implement #N. idd-implement's own gate still stops it before any side effect, and Step 5 has no error group either |
regression M-4 + requirements LOW-5; DA lowered | List five values with undiagnosed → /idd-diagnose #N and a display rule for error |
| 2 | LOW | Numbers: "31 of the 48" is 30 when the round-2 reader reads the bodies (31 only with the pre-round-5 synthetic #290 section); :304 still says 20 FP while :364 says 21; the rejected-regex counts are UTF-8 counts (22 / 21 under C); "55 sections" is 54 under CommonMark; test.sh:341-342 labels the rule's 47 / 8 as the hand review; "11 of 14" undated at reference :83 and idd-list :505 |
requirements LOW-3 + logic L-4 + regression INFO | Correct each; state "55 rows (54 sections under CommonMark)" and "measured under a UTF-8 locale" |
| 3 | LOW | The five idd_actionability_verdict misuse messages (:230, 241, 252, 256, 260) are not scrubbed (ESC survives), so "every output … misuse messages included" is false; inputs are the consumer's own, so no live impact |
requirements LOW-2 + logic L-3 + codex | Route each through _idd_scrub, or narrow the claim |
| 4 | LOW | idd-list :337 says the skill strips label names itself; no code does (labels need triage rights, so low impact) |
security S7 + requirements LOW-4 + logic | Add the one line, or delete the claim |
| 5 | LOW | COMMENTS_GUARD_RE misses --json=title,comments, a \-continued --json and "#$N" before --json; 0 such lines today |
logic L-2 + requirements LOW-7 + regression LOW-5 | Accept --json([[:space:]]+|=) and join continuation lines before matching |
| 6 | LOW | CHANGELOG :138-140 (round 4's "re-runs the Step 0.35 block") and :162 ("rows carry all three keys") need errata markers |
logic L-5 | Append markers |
| 7 | LOW | The spec scenario "A fenced example is never a section … nothing inside the fenced copy is read" is universal and contradicted by D1 / D3, which the suite pins | logic L-6 + requirements INFO-1 + codex | Add "listed divergences excepted" |
| 8 | LOW | idd-all :1011: the digit check now runs before the Clarity Surface scan, so a #-prefixed value in SPAWNED_ISSUES (model-filled, unnormalised) silently skips both scans |
regression LOW-4 | sub_n=${sub_n#\#} before the case, and print ⚠ when skipping |
| 9 | LOW | (optional) The repeated gate block fetches comments with an unguarded gh … | jq -s | python3 pipe; at Step 2.5 a transient failure reads as complexity-missing after the branch and the public comment, with the wrong reason. idd-list fixed this exact pipe in round 4 |
regression LOW-2 | Fetch-then-fold guard in the canonical block (all copies inherit it under B1) |
Follow-up
| # | Severity | Finding | Source | Action |
|---|---|---|---|---|
| 10 | — | Reader convergence on CommonMark top-level fences (security's measured fix: opener ^ {0,3} with no backtick in the info string, closer ^ {0,3}; corpus 55 / 55 and live verdicts unchanged); per-construct choice of the canonical environment for NBSP / U+3000; detect-and-surface for an unclosed fence (logic I-2: an unclosed-fence line on stderr, no verdict change); the B3 environment dependencies with their directions |
security S1 / S2, logic I-2, DA | Comment on #336, together with a correction of the round-4 handover there (it states a two-form gate, "reader fence semantics must equal the CommonMark render", and detect-and-surface, none of which round 5 ships) |
| 11 | LOW | The gate block now exists in five copies (idd-implement ×2, idd-plan, idd-all, reference); B1 pins their equality but the structural fix is one canonical script the consumers call, per deep-integration-over-hardcode. The same script can bound idd-list's output, which grew from one to five lines per open issue (about 235 lines for today's 47) |
DA-I3, DA prescription | New issue |
| 12 | LOW | U+2028, U+2029 and NEL pass _idd_scrub and the indentation sed; a line splitter that honours them sees a column-0 gate #77: or an early >>>raw, and in idd-list one issue's body can forge a line for another issue. The helper comment already discloses that the scrub does not handle them; the model reads the characters, not a visual break |
security S6 + requirements INFO-6 + codex | New issue |
Refuted or lowered (no action)
- logic I-1,
sedaborting on invalid UTF-8: not reachable — the DA checked that nothing in the helper truncates bytes, and jq output is valid UTF-8. - codex "≥ 7 messages is only a count": requirements and logic enumerated all eleven error exits; each prints.
- security S1's adversarial severity: lowered to an honesty finding (B2), for the reason given there.
Round 6 prescription (Devil's Advocate, adopted)
Principle: every sentence is backed by a test or a measurement, or it is deleted. No new universal or closed empirical claim — common-spec-prose-enumeration governs normative membership (what the author decides counts), not empirical statements about the outside world (where the line reader differs from CommonMark, which environments differ); calling those "closed" is a claim of exhaustiveness that has to be proven. Apart from the two command prefixes in B3, no change to reader semantics.
- B1–B5 as written above.
- In-scope rows 1–9.
- Do not: blanket-pin
:197,:375or the awk; normalise NBSP / U+3000; converge the reader on CommonMark; introduce any new closed list. - Re-verify scope: the B1–B5 and in-scope diff only. Required runs: the corpus and the 407 documents in zsh + ugrep and in bash; the B1 assertion against the security M1–M5 mutations; B4's new text against the two counter-examples in the constraint (both must be refused).
Baseline measurements (re-run this round, none taken from the green suite)
| Measurement | Result |
|---|---|
actionability-gate/test.sh at a80134e |
368 / 368 under C.UTF-8, C, en_US.UTF-8 and zh_TW.UTF-8 |
run-all-tests.sh in a clean worktree |
53 suites, 0 failed, in both locales |
| 55-row corpus with bodies, r3 / r4 / r5 | identical verdicts in UTF-8; under C round 3 flips #295 and rounds 4 / 5 do not; the C pin of round 5 flips no row |
243 bodies + 164 Diagnosis comments vs markdown_it |
section presence, first-heading line, section content and verdict: 0 disagreements (217 sections), in bash UTF-8, bash C and zsh |
| 47 issues open today, full three-signal gate, r4 vs r5 | byte-identical in both locales and in zsh: 5 actionable (#331, #332, #351, #353, #366), 1 blocked (#316), 41 undiagnosed |
| production grep | whence -w grep → function (Claude Code shell snapshot, ugrep); test.sh runs BSD grep in a bash child |
spectra archive on a tmp copy |
valid; added 9, modified 1; live idd-ic-r011-checkpoint keeps an updated @trace |
| gate block copies | idd-implement ×2 and idd-plan byte-identical; idd-all differs only by $N; reference only by comments |
External writes (#315)
ew-block.md was present and well-formed. The text between its markers is an older #316 comment (Sister Concerns Filed (mid-diagnose)), pure data with no instruction-shaped text; none of the five coordinator-disclosed writes since round 4 is under an audit-trail heading, so none was in the block. Checked with read-only gh: #341 exists, open, title matches; #316's Current Status matches; the #336 handover (comment 5581667971) no longer matches what round 5 ships (two-form gate, "reader fence semantics must equal the CommonMark render", detect-and-surface — round 5 ships three forms, listed divergences and a silent "no blocker" under an unclosed fence) → follow-up 10. The round-4 master report, the #316 pointer and the dashboard were not re-reviewed: UNKNOWN.
Cross-model leg (Codex gpt-6.1-sol, effort medium) — corroboration map
Codex reviewed blind and returned FAIL with 1 HIGH / 4 MEDIUM. Every finding lands on a Claude row; no Claude blocking row was contradicted.
| Codex finding | Its severity | Lands on |
|---|---|---|
| D1–D3 is not a complete divergence list (backtick in an info string; containers) | HIGH | B2 (DA lowered to MEDIUM) |
LC_ALL=C pins only the placeholder greps; the section reader follows the process locale |
MEDIUM | B3 |
| the verbatim re-run is real but its drift guard counts occurrences | MEDIUM | B1 |
| change-gate form (c) can admit a corpus-insensitive widening | MEDIUM | B4 (the DA measured it) |
| the U+3000 test asserts equality, not the blocker value | MEDIUM | B3 |
Codex's claim table agrees with the Claude re-measurement on every claim it could check; it confirmed that a forged >>>raw or gate #77: is impossible at column 0 under LF line breaks and noted the U+2028 / C1 residue (follow-up 12).
Follow-up Findings Filed (v2.72.0+ #148)
| Row | Disposition |
|---|---|
| 10 — CommonMark convergence, unclosed-fence detect-and-surface, per-construct locale directions; correction of the round-4 handover | comment on #336 |
| 11 — the gate block in five hand-maintained copies → one script; bounded idd-list output | #370 |
12 — U+2028 / U+2029 / NEL past _idd_scrub; quote raw text in idd-list Step 5 |
#371 |
Blocking B1–B5 and in-scope rows 1–9 → round 6, this PR. Verified snapshot a80134e: FAIL, no idd-316-verified tag.
… test or a measurement (#316) Round-5 verify on PR #318 returned FAIL (5 blocking, all MEDIUM). Round 5's reverts held; what failed was six closed or universal claims round 5 wrote about itself, and the change gate's premise. - B1: the gate block's copies (idd-implement Step 0.35 / Step 2.5, idd-plan, idd-all after $N -> $NUMBER) are compared byte for byte, replacing two occurrence counts. Six Step 2.5-only mutations plus a dropped fetch guard are now caught. - B2: fence divergences are "known, not exhaustive"; tab indent, backtick in the info string, indented closer, list-item fence and HTML-comment section added and pinned against markdown_it; a blockquote fence is pinned as not a divergence. - B3: only the placeholder rule claims environment independence: its two greps run as LC_ALL=C command grep (Claude Code's zsh shadows grep with a ugrep function that ignores LC_ALL). Value assertions, a UTF-8 precondition and a shadow-grep test replace the equality check. Direction pins refuse the deferral-grep and bullet-detector C pins. - B4: the change gate is keyed on direction plus an external oracle: vocabulary only toward withholding; structure only toward the markdown_it render and never toward clearing on the corpus, the live snapshot or a direction pin. The mutation test showed the awk C pin converges on CommonMark rather than failing open; the docs say so. - B5: plugin.json and marketplace.json descriptions rewritten for rounds 2 to 6. - In-scope: guarded comment and issue fetches in every gate copy; idd-list Step 5 five-value enumeration with display rules; label-scrub claim replaced by what is true; verdict misuse messages scrubbed; drift guard accepts --json=, continuation lines and "#$N"; idd-all strips a leading # from sub-issues; numbers corrected (30 of 48, 21 FP, 55 rows / 54 sections); CHANGELOG errata and a Round 6 section. Verified: suite 417/417 under bash default and LC_ALL=C; run-all-tests 53 suites, 0 failed; 440 live documents give identical round-5 and round-6 output in bash, LC_ALL=C and zsh+ugrep. Refs #316 #370 #371 #336
Verify Report — PR #318 (round 6)Enginemanual fan-out (4 lens Agents + sequenced DA, model: opus, coordinator-persisted findings) + Codex (gpt-6.1-sol, effort medium, via pai AggregateFAIL — 3 blocking (MEDIUM), 12 in-scope fixes, 2 follow-ups The code is right; the prose around it is not, again. Every lens re-measured the code and found nothing wrong with what ships. The suite passes 417/417 in five locales and What fails is what round 6 wrote about itself. Its stated principle was "every sentence backed by a test or a measurement", and it broke that principle three times:
Scope coveragePR refs (every Process Gaps
#316 — bug: #298 的修正只落在 idd-list — 另三個 Complexity consumer 未動,且 idd-list 自身 Step 5 與 Step 3.7 互相矛盾Requirements coverage: round-5 B1, B2, B3, B5 and in-scope rows 1–9 FULLY addressed; B4 PARTIALLY. The do-not list is honoured: no pin on the bullet detector, the deferral grep or the awk, and no NBSP normalisation. Spec R9 is PARTIALLY met: "SHALL be listed" (environment dependencies) and "control characters SHALL be removed" are not met as written, and property (ii) is indeterminate (row B1 below). Blocking (round 7, this PR)
In-scope fix (same PR)
Follow-up
Refuted or lowered (no action)
Round 7 prescription (Devil's Advocate, adopted)Principle unchanged: no change to reader semantics. The coordinator's round-6 awk correction broke the round's own principle: it generalised from two shapes. Round 7 therefore states measured directions only, shape by shape, each pinned.
Gate applied to each change (DA)
Baseline measurements (re-run this round, none taken from the green suite)
External writes (#315)
The round-5 master report, the round-5 pointer, the dashboard and the #366 note were not re-reviewed: UNKNOWN. Cross-model leg (Codex gpt-6.1-sol, effort medium) — corroboration mapCodex reviewed blind and returned FAIL: 3 MEDIUM, 2 LOW. It judged B1, B2, B3 and B5 FULLY addressed and B4 PARTIALLY. Every finding lands on a Claude row; no Claude blocking row was contradicted.
Codex also noted "55 real sections" in the descriptions (B3 (6)), the untested Follow-up Findings Filed (v2.72.0+ #148)
Blocking B1–B3 and in-scope rows 1–10 → round 7, this PR (rows 11–12 record-only). Verified snapshot |
…tructural rules frozen (#316) Round-6 verify on PR #318 returned FAIL (3 blocking, all MEDIUM) on prose alone; the code was correct. No reader semantics change in this round: the helper differs from c006c8b in comments only. - B1: change-gate property (ii) withdrawn. It gave no determinate answer (universal reading refused every divergence's convergence; narrow reading depended on which direction pins existed). Spec R9 now freezes structural rules (headings, fences, bullets, section boundaries, Complexity trim, and the locale of their matchers) until a requirement names the oracle they converge on (#336). The vocabulary rule's exception is narrowed: every added element must be required by a named corpus row. The gate is written once, in the spec. - B2: the round-6 awk claim is replaced by a nine-shape measurement against markdown-it and pandoc's CommonMark reader (the two disagree on trailing NBSP / U+3000 in heading text). Known divergences D8-D12 added and pinned under C.UTF-8 with a precondition; the awk C pin now fails four pins. - B3: six claims narrowed: corpus edge shapes (#290), the Complexity trim added to the environment list, two spec scenarios replaced by concrete ones, the author-can-delete framing scoped (#372), "C0 control characters and DEL" (#371), plugin/marketplace descriptions. - In-scope: always-matching grep shadow and whitespace-only value pins for the second placeholder grep; the gate-block compare checks file-wide verdict calls, exactly one block in idd-plan and idd-all, no gate assignments outside blocks, no $NUMBER in idd-all, and runs the extracted block against a stub gh; the comments guard checks raw and joined lines; errata markers in design, tasks and the CHANGELOG are test-checked; idd-all's skip line uses printf + C0/DEL strip; idd-list has one display rule for group=error; label sentence narrowed to what jq -c escapes. Verified: suite 502/502 in five locales; run-all-tests 53 suites, 0 failed; 440 live documents give identical round-6 and round-7 output in bash UTF-8, bash C and zsh+ugrep; 12/12 mutations caught. Refs #316 #336 #370 #371 #372
…and glibc (#316) CI (ubuntu-latest) had failed on every commit since c006c8b: round 6 pinned NBSP forms as if macOS behaviour were universal. glibc does not classify NBSP as blank, so on Linux `Plan when<NBSP>triggered` already routes and a `-<NBSP>` blocker after a placeholder is not read — pre-existing since round 3, not a regression. The round-6 verify ran on macOS only. - Direction pins use U+3000, blank to grep on both platforms; they still catch the deferral-grep and bullet-detector C pins on macOS and on Linux. - The NBSP forms become environment records with the platform's measured value. - D8-D10 take their expected value from the platform awk's measured classification (macOS awk: NBSP blank, U+3000 not; gawk: the reverse; mawk: neither); the awk C pin is caught on macOS (4 pins) and under gawk (1 pin). - The Complexity-trim record follows bash's classification. - Helper, reference, spec, CHANGELOG, design and tasks name the platform dimension. Measured: Ubuntu 24.04 container (GNU grep 3.11, gawk, mawk), full runner as a non-root user 53 suites 0 failed; macOS 502/502 under C.UTF-8 and C, full runner 53/0. Refs #316 #336
這個 PR 做什麼(第 2 輪,2026-09-07)
讓四個 routing consumer(
idd-list/idd-all/idd-implement/idd-plan)共用同一份### Complexity抽取與 actionability 判定,並且真的呼叫它。追蹤 issue #316。根因(兩層)
### Complexity活在 append-only 的 Diagnosis comment,但 parked 會變。IDD 早就知道正確做法:
### Blocking正因為會變才放在 body 由idd-update維護。規則(第 2 輪,corpus 159/159)
### Complexity下第一個非空行(code fence 不是區段;同級或更高 heading 結束區段)Simple/Plan/Spectra/SDD-warranted之一開頭(整詞、最長匹配優先);開頭即 tier,其後的理由/括號/
via <來源>一律合法when triggered/parking lot/deferred/暫緩→ exit 5,stderrdeferral-marker: <原值>,stdout 不印 tier(合法的前綴正是不能路由的理由)移入 discussion list)complexity-unparseable(資料錯誤,要修)complexity-missing(先 diagnose)complexity-deferral-marker(合法狀態,不是要修的東西)風險姿態:
parking-lotlabel 是主訊號,語彙是高精度低召回的安全網 —— 漏抓退回 #298 前行為(label 兜底)、誤抓是對可動 issue 的 hard stop,所以語彙保守、加詞要有 corpus 零誤報證據。
只掃第一行:掃整段會在 corpus 上多 2 個誤報(#154「remove deferred caveat」、#137「
deferredenum」)。主要改動
scripts/lib/actionability.shidd_blocking_section(- (none)placeholder 算空);共用 fence-aware 區段抽取器;value-less flag → exit 2(原本無限迴圈)skills/{idd-list,idd-all,idd-implement,idd-plan}/SKILL.mdidd_actionability_verdict(三訊號齊);Diagnosis 改用gh api … --paginate(--json comments只回最舊 100 則);set -euo pipefail安全的條件式捕捉;exit 2 是 consumer 自己的 FATALskills/idd-list/SKILL.mdskills/idd-diagnose/SKILL.mdreferences/actionability-gate.mdscripts/tests/actionability-gate/CHANGELOG.md/plugin.json/marketplace.json誠實的殘餘
Plan。fixture 把它 pin 為 actionable,不偽造 marker;要擋它由人貼 label。快照斷言因此從「只有 feature: /idd-all (無參數) bulk-solve mode — AI 自動 sequence + bundle 所有 open issue #37」改為「feature: /idd-all (無參數) bulk-solve mode — AI 自動 sequence + bundle 所有 open issue #37 與 feat: auto-@mention when idd-issue/idd-comment poses open question to a person (diagnose excluded) #128」。
Spectra—— 那是截斷 regex 自己說的話;真值是**Spectra**(Layer 2 + Layer 3 if/when triggered):…,延期藏在括號裡(這反而是「掃整行」的證據)。plan-routing-consistency在有 gitignored.spectra/snapshots/的開發機上會失敗、乾淨 worktree 通過 —— bug: plan-routing-consistency 測試掃到 gitignored .spectra/ snapshot — 乾淨 checkout 綠、跑過 spectra 的本機必紅 #335,本 PR 不動。驗證
plan-routing-consistency(bug: plan-routing-consistency 測試掃到 gitignored .spectra/ snapshot — 乾淨 checkout 綠、跑過 spectra 的本機必紅 #335,pre-existing、與本 PR 無關)actionability-gate225/225,含 159 筆 corpus 回歸 159/159#37 #128(後者為記錄在案的 designed miss)Spectra
openspec/changes/add-actionability-gate/—— 22/22 tasks(10 第 1 輪、2 skipped、10 第 2 輪)。