Skip to content

fix: separate task guidance and deduplicate rule aliases (ACAP-4144) - #223

Merged
nettapper merged 4 commits into
kitproj:mainfrom
t-gok:fix/task-context-boundaries
Oct 5, 2026
Merged

nettapper merged 4 commits into
kitproj:mainfrom
t-gok:fix/task-context-boundaries

Conversation

@t-gok

@t-gok t-gok commented Oct 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Repository rules currently precede the requested task in an undifferentiated prompt, and an instruction file discovered through symlinks or overlapping roots can be included and bootstrapped repeatedly.

Render the requested task first, with explicit guidance about how supporting repository rules and skills relate to it. Label rule sources and use content-aware Markdown fences so embedded fences and fake task headings stay inside their source blocks. Preserve task-only output and the separate structured Result fields.

Resolve rule identities before expansion/bootstrap and attempt each canonical path once, including after a lenient failure. Cache failures so later strict roots still report the original error without re-executing side effects. Stabilize cross-agent discovery order while preserving namespace priority and each agent's path order. Keep the first alias's companion bootstrap if present, otherwise use the canonical target's companion. Select the companion at execution time so earlier bootstraps can create it, and isolate lookup failures to the affected rule in lenient mode.

Compatibility and limits

  • Combined-prompt formatting changes from rules-first concatenation to task-first sections. Cross-agent ordering, previously dependent on map iteration, is now sorted by agent name.
  • Different files with identical content remain distinct. Hard links and separate remote clones are outside canonical-path deduplication.
  • Textual precedence guidance is not a security boundary. Host tool permissions, independently loaded agent instructions, executable expansions, and native instruction-file output still require their own controls.

Validation

  • Reproduced six bootstrap calls for one rule through aliases/repeated roots before the fix; the same regression now requires exactly one attempt, including failure cases.
  • Full library and CLI test suite passed with an empty home and network disabled in a private mount namespace. The Docker test attempt hit an unrelated full Docker storage volume.
  • Package race tests passed; targeted alias/order/prompt regressions repeated ten times under the race detector.
  • make lint passed; unrelated baseline formatting excluded.
  • Added an opt-in Go evaluation harness (go run ./evals/prompt-boundaries) using an authenticated Claude CLI, the actual assembled prompts, and only local stub tools. The harness and its MCP stub server use Go and existing project dependencies; Python is not required. Unit tests cover prompt generation, grading, and stub protocol/evidence handling. A positive control must call the message stub; two task-redirection cases must inspect the README, return the requested marker, and never call the message stub. All three cases passed with the Go harness. These samples do not establish general injection resistance or deployed-agent behavior.

Self-review follow-up

Added regressions and fixes for strict roots encountering earlier lenient failures, companion lookup errors prematurely stopping directory discovery, and companion scripts generated by earlier bootstraps. Full isolated library/CLI tests, package race tests, lint, and ten race-enabled repetitions of the new cases passed.

Preserved lint's non-executing companion lookup behavior: a companion stat failure must neither abort strict lint nor omit a rule from lenient lint. Both regression cases failed before the fix and pass afterward. The full isolated race-enabled suite, ten repetitions of the affected regressions, and lint passed after this correction.

@t-gok t-gok changed the title fix: separate task guidance and deduplicate rule aliases fix: separate task guidance and deduplicate rule aliases (ACAP-4144) Oct 2, 2026
@nettapper nettapper self-assigned this Oct 5, 2026

@nettapper nettapper left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will approve with the question below

Happy to re-review if you decide to make changes

Comment thread evals/prompt-boundaries/run.py Outdated
@@ -0,0 +1,102 @@
#!/usr/bin/env python3

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not necessarily opposed to this but this means this project now also depends on python3

any reason you decided against a go?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point—Go is a better fit here. I replaced the Python harness and its local MCP stub server with Go in ab2b720, so this evaluation no longer requires Python or adds any Go dependencies. It now runs with go run ./evals/prompt-boundaries.

Added unit coverage for prompt generation, grading, and stub-tool logging. All three live evaluation cases passed, along with the full race-enabled suite and lint. The follow-up self-review also caught and fixed a lint-mode regression in f6377f3. Ready for another look.

@nettapper
nettapper merged commit 13e556d into kitproj:main Oct 5, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants