Repository navigation
feat(wire-schema): let a scenario scope wire-schema-valid to its methods - #567
Draft
preston-footprint wants to merge 1 commit into
Draft
preston-footprint wants to merge 1 commit into
preston-footprint wants to merge 1 commit into
Conversation
wire-schema-valid grades every implementation message in every scenario, so an SDK gap outside a suite's area (say a tools/list result missing newer fields) fails an extension suite that never meant to test it. A scenario can now set wireSchemaScope, a list of methods (an entry ending in "/" matches a prefix). wire-schema-valid then grades only the implementation's messages for those methods, and violations elsewhere go to an INFO check, wire-schema-valid-out-of-scope, so they stay visible without failing the run. A violation whose method is unknown stays in scope. Scenarios that don't set it, or set it empty, are graded as before, and harness-origin violations are unaffected. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
preston-footprint
force-pushed
the
feat/wire-schema-scope
branch
from
October 20, 2026 23:46
1c6193a to
711da52
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
wire-schema-valid(#399) grades every implementation message in every scenario. That's the right default, but an extension suite then fails for SDK gaps outside its area. Against an rmcp-based server, for example, the SEP-3415 events suites (#521) failwire-schema-validbecause the server'stools/listresult is missingttlMs/cacheScope, which has nothing to do with events.This adds an optional
wireSchemaScopetoScenarioandClientScenario: a list of methods, where an entry ending in/matches a prefix (an events scenario would use['events/', 'notifications/events/']).wire-schema-validgrades only the implementation's messages for those methods.wire-schema-valid-out-of-scope. They stay visible in the output andchecks.json, but don't fail the run. INFO already counts as neither a failure nor a warning in the runners and the expected-failures baseline.initialize,server/discover,ping) are out of scope unless listed.mainsets it, so nothing changes until a scenario opts in. The events scenarios in feat(sep-3415): Conformance suite for triggers and events #521/feat(events): client conformance scenarios (#540) #541 would be the first users.wire-schema-harness-error) are unaffected.Split out of #521 as suggested in #521 (comment).
Prove it passes and fails
New tests in
src/validation/wire-schema.test.ts, underwireSchemaChecks scope:['events/'], an invalidtools/listresult leaveswire-schema-validSUCCESS and produces the INFO check. This fails onmain.method(so it goes out of scope under['events/']). This fails onmain.How Has This Been Tested?
npm run typecheck,npm run lintandnpm run buildpass.npm test: 48 files, 648 tests passed (the pre-push hook ran it as well).npx vitest run src/validation: 32 passed.Runs against a real SDK
An rmcp-based server (Rust) running the
events-pushscenario from #521, with this branch merged on top locally:Scope unset (today's behaviour):
With
wireSchemaScope = ['events/', 'notifications/events/']onEventsPushScenario:The other five failures are events rows unrelated to this change.
AI disclosure
This PR and its tests were written primarily by Claude Code, under my direction and review.
🤖 Generated with Claude Code