Repository navigation
fix(llm): rate governance votes on what they execute - #444
Merged
Merged
Conversation
Strategist Multisig nonce 3376 only called Voter.vote_yea for proposals 2-5, moving no funds and enacting nothing (execution is a separate Voting.execute), yet was tagged MEDIUM because the proposal contents were unverified. - risk anchors: vote_yea/vote_nay (Yearn Voter) and Governor castVote/castVoteWithReason anchor at LOW - system prompt: missing context about effects the transaction does not execute goes under "Not verified" and does not raise the tag; unverified targets and undecoded payloads still do Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Generalize the vote anchors beyond Yearn's Voter: OpenZeppelin Governor /
Compound Bravo castVote* (incl. BySig), GovernorAlpha castVote(uint256,bool),
Aave v2 submitVote and Curve-style vote_for_gauge_weights all anchor LOW.
Aragon-style vote(uint256,bool,bool) / votePct(...,bool) run the vote's
script in the same tx when `_executesIfDecided` is set and the vote decides
it, so they are LOW only with the flag off and MEDIUM ("judge that script")
with it on. delegate(address) stays unanchored: it hands voting power to
someone else.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hors Per-signature vote anchors only cover the implementations we list. Drop them and state the rule once in the system prompt: a call that only records a governance vote, in any implementation, is LOW. Exceptions are a vote that itself executes the proposal (Aragon-style `_executesIfDecided`, auto-execution on quorum) and delegation, which is not a vote. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Strategist Multisig nonce 3376 (gist) only calls
Voter.vote_yeafor proposals 2–5 on Voting0x543e8871562a8C53E8B6a26835aeEcB3a5A13070. It was still tagged MEDIUM. The model's own analysis found no transfers, approvals or upgrades. It raised the tag only because "proposal contents and effective voting weight are unverified".Per the verified
Voter.vy/Voting.vy,vote_yeaonly records vote weight, plus YBC/delegated-staking tallies when the voter is a YBC member. A passed proposal still needs its ownVoting.execute(uint256,bytes)call, which our monitor judges on its actions when it is queued.Changes
System prompt (
utils/llm/ai_explainer.py), two general rules:_executesIfDecidedflag set to true, or auto-execution on quorum). The model judges what executes.An earlier revision of this PR added per-signature vote anchors to
utils/risk_anchors.py. They were dropped in favour of this single rule, which also covers voting contracts we haven't listed.Testing
tests/ai_explainer/test_prompt.py: the system prompt carries the vote rule and its exceptions.ruffclean.🤖 Generated with Claude Code