Repository navigation
fix: separate task guidance and deduplicate rule aliases (ACAP-4144) - #223
Conversation
nettapper
left a comment
There was a problem hiding this comment.
Will approve with the question below
Happy to re-review if you decide to make changes
| @@ -0,0 +1,102 @@ | |||
| #!/usr/bin/env python3 | |||
There was a problem hiding this comment.
not necessarily opposed to this but this means this project now also depends on python3
any reason you decided against a go?
There was a problem hiding this comment.
Good point—Go is a better fit here. I replaced the Python harness and its local MCP stub server with Go in ab2b720, so this evaluation no longer requires Python or adds any Go dependencies. It now runs with go run ./evals/prompt-boundaries.
Added unit coverage for prompt generation, grading, and stub-tool logging. All three live evaluation cases passed, along with the full race-enabled suite and lint. The follow-up self-review also caught and fixed a lint-mode regression in f6377f3. Ready for another look.
Summary
Repository rules currently precede the requested task in an undifferentiated prompt, and an instruction file discovered through symlinks or overlapping roots can be included and bootstrapped repeatedly.
Render the requested task first, with explicit guidance about how supporting repository rules and skills relate to it. Label rule sources and use content-aware Markdown fences so embedded fences and fake task headings stay inside their source blocks. Preserve task-only output and the separate structured Result fields.
Resolve rule identities before expansion/bootstrap and attempt each canonical path once, including after a lenient failure. Cache failures so later strict roots still report the original error without re-executing side effects. Stabilize cross-agent discovery order while preserving namespace priority and each agent's path order. Keep the first alias's companion bootstrap if present, otherwise use the canonical target's companion. Select the companion at execution time so earlier bootstraps can create it, and isolate lookup failures to the affected rule in lenient mode.
Compatibility and limits
Validation
make lintpassed; unrelated baseline formatting excluded.go run ./evals/prompt-boundaries) using an authenticated Claude CLI, the actual assembled prompts, and only local stub tools. The harness and its MCP stub server use Go and existing project dependencies; Python is not required. Unit tests cover prompt generation, grading, and stub protocol/evidence handling. A positive control must call the message stub; two task-redirection cases must inspect the README, return the requested marker, and never call the message stub. All three cases passed with the Go harness. These samples do not establish general injection resistance or deployed-agent behavior.Self-review follow-up
Added regressions and fixes for strict roots encountering earlier lenient failures, companion lookup errors prematurely stopping directory discovery, and companion scripts generated by earlier bootstraps. Full isolated library/CLI tests, package race tests, lint, and ten race-enabled repetitions of the new cases passed.
Preserved lint's non-executing companion lookup behavior: a companion stat failure must neither abort strict lint nor omit a rule from lenient lint. Both regression cases failed before the fix and pass afterward. The full isolated race-enabled suite, ten repetitions of the affected regressions, and lint passed after this correction.