You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
LINBEE-30104 | fix: don't grade tasks that change no code (v1.4.6) - #22
A task that changes no code now prints a 'skipped (not a code task)' line instead of a LOW grade: no calls, nothing reported, normal care; LOW stays for trivial code edits, and the hook treats the skipped line as answered.
Purpose: Update agentic-advisor to skip grading for non-code tasks while maintaining effort holds for actual file edits.
Main changes:
Added VERDICT_RE and GRADED_RE to distinguish between effort grades and skipped tasks
Updated verdict_state and guidance to support skipped (not a code task) verdict signals
Modified agentic-advisor-trigger.sh to require a real grade before allowing file edits
Generated by LinearB AI and added by gitStream. AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Description using GuidelinesLearn how
The reason will be displayed to describe this comment to others. Learn more.
✨ PR Review
The change adds a "skipped (not a code task)" verdict and updates the hook regex, guidance and skill docs to match. The regex and the new reminder text are consistent with each other. The new classification rules in SKILL.md overlap, and the skip verdict lets later code edits pass the hook unenforced.
2 issues detected:
🧹 Maintainability - Two adjacent rules both claim comment/doc edits and give different outputs (skipped vs LOW). 🛠️
Details:"Not a code task" lists "writing a comment" as skipped with no grade, while "Trivial code edit" lists "comments, docs" as LOW. Editing a code comment or a doc file fits both rules. The README says "writing a comment" is not graded, and "docs" was dropped from the trivial list, so doc edits now match neither rule clearly. The agent can pick either outcome (a skipped line or a LOW grade) for the same task, which makes telemetry and behavior inconsistent.
🛠️ A suggested code correction is included in the review comments.
🐞 Bug - Treating "skipped" as a final answer lets code edits proceed without an effort grade, because only the latest Skill call's verdict is checked.
Details:The verdict regex now counts "skipped (not a code task)" as a completed verdict. The SKILL text says that if the task later turns into a code change, the sweep should run before the first edit. After a skip, verdict_state returns "ok" and the done marker is already set, so the hook never reminds the agent or enforces a grade before the first edit. A task that starts as a question and then edits code is therefore never graded or reported, and the "MANDATORY FIRST STEP" gate is bypassed. The guidance also says "do not skip it even if the task seems obvious" and then allows a skip, which invites over-skipping.
Generated by LinearB AI and added by gitStream. AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using GuidelinesLearn how
- **Not a code task → no grade at all.** If the task will not change any code (reading or explaining code, answering a question, reviewing a doc, ticket or screenshot, writing a comment or message), don't grade it: print `> LinearB: <repo> — skipped (not a code task)`, call nothing, and work with your normal care. A grade measures the risk of a code change; a LOW here would wrongly tell you to go light on accuracy. If the task later turns into a code change, run the sweep then, before the first edit.
- **Trivial code edit → LOW, skip everything.** If the task edits code but changes **no code behavior** (comments, docs, formatting/whitespace, a pure rename), skip signal gathering entirely: effort is **LOW**, add no defensive guards (there's no logic to protect), and don't call LinearB at all — this holds **even on a fragile repo**. Anything that changes behavior is **not** trivial and warrants the sweep below — note that a **constant-value change counts as behavioral** (e.g. a timeout, retry count, or feature-flag default), as does any conditional or logic edit.
The reason will be displayed to describe this comment to others. Learn more.
🧹 Maintainability - Overlapping skip/LOW rules: Make the categories mutually exclusive. For example, say that any task that edits a file, including a comment or a doc, is a trivial code edit graded LOW. Reserve "skipped" for tasks that modify no files (read, explain, review, answer). Update the README wording to match.
Suggested change
-**Not a code task → no grade at all.** If the task will not change any code (reading or explaining code, answering a question, reviewing a doc, ticket or screenshot, writing a comment or message), don't grade it: print `> LinearB: <repo> — skipped (not a code task)`, call nothing, and work with your normal care. A grade measures the risk of a code change; a LOW here would wrongly tell you to go light on accuracy. If the task later turns into a code change, run the sweep then, before the first edit.
-**Trivial code edit → LOW, skip everything.** If the task edits code but changes **no code behavior** (comments, docs, formatting/whitespace, a pure rename), skip signal gathering entirely: effort is **LOW**, add no defensive guards (there's no logic to protect), and don't call LinearB at all — this holds **even on a fragile repo**. Anything that changes behavior is **not** trivial and warrants the sweep below — note that a **constant-value change counts as behavioral** (e.g. a timeout, retry count, or feature-flag default), as does any conditional or logic edit.
-**Not a code task → no grade at all.** If the task will not modify any file (reading or explaining code, answering a question, reviewing a doc, ticket or screenshot, drafting a message in chat), don't grade it: print `> LinearB: <repo> — skipped (not a code task)`, call nothing, and work with your normal care. A grade measures the risk of a code change; a LOW here would wrongly tell you to go light on accuracy. If the task later turns into a file edit, run the sweep then, before the first edit.
-**Trivial code edit → LOW, skip everything.** If the task edits any file but changes **no code behavior** (code comments, doc files, formatting/whitespace, a pure rename), skip signal gathering entirely: effort is **LOW**, add no defensive guards (there's no logic to protect), and don't call LinearB at all — this holds **even on a fragile repo**. Editing a comment or a doc is always this case, never "skipped". Anything that changes behavior is **not** trivial and warrants the sweep below — note that a **constant-value change counts as behavioral** (e.g. a timeout, retry count, or feature-flag default), as does any conditional or logic edit.
Is this review accurate? Use 👍 or 👎 to rate it
If you want to tell us more, use /gs feedback e.g. /gs feedback this review doesn't make sense, I disagree, and it keeps repeating over and over
The reason will be displayed to describe this comment to others. Learn more.
Fixed in ed815e9: "skipped" now means the task modifies no file; any file edit with no behavior change (code comments, doc files, formatting) is LOW. README updated to match. Same in lb-claude-plugins#147.
The reason will be displayed to describe this comment to others. Learn more.
✨ PR Review
The PR adds a "skipped (not a code task)" verdict and tightens the edit hold so a skip alone doesn't satisfy it. The tightened check in the PreToolUse edit hold can be defeated by the skill's own example text in the transcript.
1 issues detected:
🐞 Bug - A whole-transcript grep for the verdict pattern is satisfied by the skill's own example text, so it can't tell a real grade from a skip. 🛠️
Details:The edit hold now requires LinearB: .+ (LOW|MEDIUM|HIGH) effort anywhere in the whole transcript, which is meant to rule out a skip-only run. The transcript also contains the loaded SKILL.md content, which includes the literal example > LinearB: api-service — HIGH effort (...). Once the skill has been invoked, the grep therefore matches the example even if the agent only printed the skipped line or nothing. The hold is then bypassed and the skipped-then-edit case is not enforced. The check also isn't limited to assistant text, unlike the verdict_state jq filter.
🛠️ A suggested code correction is included in the review comments.
Generated by LinearB AI and added by gitStream. AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using GuidelinesLearn how
The reason will be displayed to describe this comment to others. Learn more.
🐞 Bug - Grade check matches skill example: Restrict the match to assistant-authored text or Bash printf tool_use input, for example by reusing the jq logic from verdict_state or by grepping only lines containing "role":"assistant". Alternatively, make the pattern exclude the SKILL.md example.
Suggested change
if skill_ran && grep -qE 'LinearB: .+ (LOW|MEDIUM|HIGH) effort'"$transcript"2>/dev/null;then
The reason will be displayed to describe this comment to others. Learn more.
Fixed: the edit hold now uses the same parser as the verdict reminder (assistant text and printf calls after the latest skill invocation) and requires a LOW/MEDIUM/HIGH grade, so the loaded SKILL.md example no longer counts. The skip-then-edit test now includes the skill text with its example line, and fails on the previous hook. (fe76b15)
The reason will be displayed to describe this comment to others. Learn more.
✨ PR Review
LGTM
Generated by LinearB AI and added by gitStream. AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Review using GuidelinesLearn how
This pull request has used all 3 of its automatic AI Reviews.
✨ Comment /gs review to run one now. No arguments needed. It uses the same settings as your automatic reviews: gitStream Settings in Managed Mode, your CM files in Self-Managed.
Skipped work is still counted in grading telemetry when a later task in the same session receives a grade. In agentic-advisor-report.sh, parse_tokens() keeps the first Skill timestamp and waits for the first LOW/MEDIUM/HIGH verdict, without recognizing skipped outcomes. It therefore counts the skipped run and intervening answer tokens and elapsed time as grading overhead. Exclude skipped runs from grading intervals and add a mixed-session regression; the new skipped-only test does not exercise this path.
if [ -n "$state" ] && : > "$vdone" 2>/dev/null && [ "$state" = "pending" ]; then
dbg "PreToolUse: reminded the agent to print the verdict line"
note "LinearB: your effort grade isn't recorded yet. It's only recorded from the verdict line, not from thinking, so print it now: > LinearB: <repo> — <LOW|MEDIUM|HIGH> effort (<evidence>) — <plan>. If you aren't writing text, run: printf '%s\\n' '> LinearB: <repo> — <LEVEL> effort (<evidence>) — <plan>'"
note "LinearB: your effort grade isn't recorded yet. It's only recorded from the verdict line, not from thinking, so print it now: > LinearB: <repo> — <LOW|MEDIUM|HIGH> effort (<evidence>) — <plan>. If you aren't writing text, run: printf '%s\\n' '> LinearB: <repo> — <LEVEL> effort (<evidence>) — <plan>'. If no code will change, print > LinearB: <repo> — skipped (not a code task) instead."
This check also handles Task/Agent, not just edits. After the skill prints a valid skipped verdict, delegating read-only work to a general-purpose or custom agent is denied if the session/repo hold has not already been used. The denial then requires a LOW/MEDIUM/HIGH grade for work that should remain ungraded. Accept the skipped verdict for delegation while retaining the real-grade check for source edits. Add a skipped-then-read-only-delegation regression test alongside the existing skipped-then-edit test.
Review effort: Balanced
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A task that changes no code now prints a 'skipped (not a code task)' line instead of a LOW grade: no calls, nothing reported, normal care; LOW stays for trivial code edits, and the hook treats the skipped line as answered.
🤖 Generated with Claude Code
✨ PR Description
Purpose: Update agentic-advisor to skip grading for non-code tasks while maintaining effort holds for actual file edits.
Main changes:
VERDICT_REandGRADED_REto distinguish between effort grades and skipped tasksverdict_stateandguidanceto supportskipped (not a code task)verdict signalsagentic-advisor-trigger.shto require a real grade before allowing file editsGenerated by LinearB AI and added by gitStream.
AI-generated content may contain inaccuracies. Please verify before using.
💡 Tip: You can customize your AI Description using Guidelines Learn how