Trace corpus

Fixtures for the golden-corpus tests (crates/mcp-trace-validator/tests/golden.rs):

• good/ — sessions that must validate with verdict pass.
• violations/ — single-issue sessions, each falsifying at least one check; named after the requirement whose check they exist to kill.
• draft/{good,violations}/ — the same two kinds for revision 2026-07-28, judged against that revision’s registry behind the draft-2026-07-28 feature.
• draft/captured/ — sessions recorded off the wire from implementations this repository did not write. No verdict is asserted for them; their whole report is byte-pinned instead. See Captured versus authored below.
• golden/ — the byte-pinned expected report for every trace, with golden/draft/ holding the 2026-07-28 corpus’s. Regenerate only via cargo xtask bless and review the diff like code.
• golden/exclusions/ — one ledger per revision, holding the clauses that revision’s registry declines to judge from a trace, and why.

The revisions keep separate golden directories because both name traces after requirement IDs drawn from their own registry: base-045-… is one clause at 2025-11-25 and a different one at 2026-07-28, so a shared directory would let one revision’s golden answer for the other’s trace.

What a golden pins

A report states two kinds of fact, and they are pinned separately (ADR-0013).

golden/<stem>.json holds what the trace decided: the revision, the whole totals, and every pass / fail / warn / not-observed / not-applicable / unsupported row, in registry order.

golden/exclusions/<revision>.json holds what the registry decided: the excluded rows, whose outcome and prose come from the registry alone and are identical for every trace judged against it — engine::build_row never consults the session for them. One distinct set per revision, and before the split every golden restated its revision’s whole set: at the time that was 59% of the corpus’s lines.

Every cell is verified against the corpus by cargo xtask ci (xtask::coverage::corpus). It is checked rather than trusted because it had already drifted twice: this paragraph is prose, and prose does not recount itself.

Nothing stopped being pinned. Splicing a golden back together with its ledger reproduces the whole report, and assert_reconstructs_the_full_report proves it does on every trace, on every run — so a judged row that went missing, a ledger that drifted from its registry, or rows that interleave in any order but the registry’s all fail the suite. totals.excluded stays in every golden as the per-trace tie back to the ledger: a clause entering or leaving the excluded set moves every golden for that revision, one line each.

Only excluded collapses. not-observed and not-applicable are per-trace evidence, and the table’s third row is the measure of it: near one distinct set per golden, in both revisions. Collapsing those would delete exactly what makes the captures differentiate each other.

Violation naming contract

A violation trace is named area-nnn-<slug>.jsonl and must produce a Fail/Warn row with findings for exactly requirement AREA-NNN — the golden harness enforces the attribution by name (violation_traces_fail_and_match_goldens, and draft::draft_violation_traces_fail_and_match_goldens for draft/), so a defect re-routed to a different requirement cannot re-bless silently. A stem that does not begin with a requirement ID fails the suite loudly.

Captured versus authored

Everything in good/, violations/ and draft/{good,violations}/ is authored: hand-written to exercise one clause. That is what makes them precise, and it is also their limit — an authored fixture can only ever confirm its author’s reading of the specification. A check that is wrong in the same way the author is wrong passes its unit tests and its corpus alike, and neither notices.

draft/captured/ exists to close that gap. Its traces are real recordings of traffic between implementations neither of which was written to satisfy these checks, so they can disagree with us — and when they do, the disagreement is information rather than a test failure. Their goldens are pinned without any pass/fail expectation, because a real implementation is whatever it actually is; what the golden protects is which requirements fire, on which events, with what detail. A check that starts misfiring on real traffic moves it — which is precisely the half of the report that stays in the per-trace file.

Provenance ledger

Every trace’s origin, in one reviewable place that survives history rewrites (the invariant test in golden.rs fails if a trace is added without a ledger row). All current traces share one provenance: hand-authored for this repository as synthetic sessions (no third-party traffic, no recorded production data), written against the 2025-11-25 spec text fetched live from modelcontextprotocol.io on 2026-06-09 (re-verified clause-by-clause against the live text on 2026-06-11) and validated against the embedded registry at the commit that introduced them.

Traces under draft/captured/ are the exception and record their own origin below: the implementations on both ends, the tool that recorded them, and when.

good/

violations/

Each file injects exactly the violation its name states; golden/ shows every row the trace decided, including any intrinsic secondary findings the injected defect causes (a malformed notification also fails lifecycle accounting, for example).

2026-07-28 captured (corpus/draft/captured/)

Two recordings of the same client driving the same scenarios, differing in one variable: which revision the server was serving. That is what makes the pair worth committing — a single capture shows what one implementation does, while a matched pair shows what the revision costs, and every difference between the two reports is attributable to that one change.

Both carry server/discover, tools/list, tools/call, completion/complete, resources/{list,read}, prompts/{list,get} and progress notifications; every request carries the revision’s per-request _meta envelope and its SEP-2243 metadata headers, and neither carries an initialize anywhere. Their tools/list results differ by one entry, which is the point of recording both: test_url_elicitation is listed by the legacy server and not by the stateless one, because URL-mode elicitation is a feature 2026-07-28 removed.

The one finding. CACH-001 — the legacy server’s complete results carry no ttlMs, on resources/list and three resources/reads. That is our 2025-11-25 server being held to a revision it does not implement, which is the correct answer, and the stateless capture is the control that proves the check reports the server rather than the recording.

Two findings that used to be here were ours, not the implementations’. An earlier version of this pair reported TRAN-058 (client POSTs missing Mcp-Method/Mcp-Name) and TRAN-068 (SSE responses missing X-Accel-Buffering: no). Both were artifacts of the tap’s recording allowlist, which predated SEP-2243 and did not record those headers — the client sent them (rmcp’s transport rejects a 2026-07-28 request without Mcp-Method before dispatch, and the suite scored 23/23) and the server sent them (rmcp sets X-Accel-Buffering on every SSE response it builds). The allowlist was extended; both clauses now pass on both captures. The lesson is recorded rather than quietly fixed: a check can only be as honest as the recording it reads, and a capture path that silently drops evidence manufactures findings against conforming implementations.

The same lesson, a third time, one field over (2026-08-20). The tap recorded an exchange’s headers but not its method, so TRAN-025 and TRAN-039 — the Accept obligations of a POST and of a GET — were enforced as one rule applied to every client request whatever its form. That was wrong twice over, and both halves were found by pointing this workspace’s own reference host at its own reference server over HTTP and validating the tap: the conforming session-teardown DELETE, which owes no Accept header at all, drew two MUST-level failures, while a POST omitting application/json — the verbatim violation TRAN-025 names — was reported pass, because the one shared rule could only demand what both clauses had in common. EventBody::Http now carries the method, each clause is judged against the requests it binds, and an exchange whose method the capture did not record is judged by neither. These two captures were re-recorded so the method is present; that is why their Recorded by row reads 2026-08-20.

Why the runner’s 23/23 and our 60-vs-61 were both right — and how the runner caught up. The suite’s 2026-07-28 scenarios exercise features — list a thing, call a thing, read a thing — and a 2025-11-25 server answers all of them, because rmcp serves a per-request-versioned POST whichever revision the handler advertises. At 0.2.0-alpha.9 that was the whole of the runner’s reading, so it scored both servers 23/23. The registry here judges the specification’s prose instead, so it saw the one place the two servers actually differ: CACH-001, no ttlMs on cacheable results.

0.2.0-alpha.11 found the same thing, six weeks later. Its new wire-schema-valid check validates every message against the negotiated revision’s JSON schema, and it fails the 2025-11-25 leg’s four resource scenarios for must have required property 'cacheScope' / 'ttlMs' — that clause, independently. Re-measured, the legacy leg scores 37 passing / 4 failing and the stateless leg 41 / 0, so the runner now separates the two servers it could not tell apart before. Neither instrument was wrong; they were measuring different things, and one of them saw this first. The pair is still the evidence for taking both readings — now with a worked example of the prose-level reading arriving earlier than the schema-level one.

The 64 not-observed rows are the honest denominator: of the 124 clauses this revision’s registry can judge, these sessions carried subject matter for 60. They open no subscription, present no cursor, draw no error, and send no malformed _meta, so those clauses are neither passed nor failed here — they are untested, and the report says which.

Regenerate both with cargo xtask draft-readiness and copy target/draft-readiness/<revision>/tap/001-stateless.jsonl; the ephemeral port in the host header changes per run, so re-copying is a deliberate act with a golden diff, not something to do casually.

The stdio capture

It is the only capture that exercises subscriptions/listen. The official suite drives no subscription, so the four judged SUBS clauses — and BASE-039, which binds a subscription stream’s notifications — are backed by this file alone: an acknowledgment, a filter the server narrowed, four tagged notifications and an empty graceful-closure result. The same holds for the MRTR clauses. In the two HTTP captures every one of those rows now reads not observed; here they read pass.

Two defects were found by enriching it, and neither could have been found any other way. Until the session read a resource the catalog does not have, nothing in this workspace had ever exercised the server’s not-found path at this revision — and it answered -32002, which 2026-07-28 withdraws outright (basic/index#error-codes lists it under “Implementations of this protocol version MUST NOT emit these codes” and names -32602 as the replacement). The second was ours: the host’s recorder assigned seq when the send future completed rather than when the message was handed to the transport, so a reply received while that future was still pending landed in the trace ahead of its own request. That reads as the server answering an id nobody asked, and it failed BASE-046 on roughly one capture in three. Both are fixed, both have regression tests, and both are the argument for driving more traffic rather than more fixtures.

That asymmetry is the reason to keep all three, and it used to be invisible. Until the vacuous-pass fix, those rows read identically across the three files — pass everywhere, whether or not the session had opened a stream — and this paragraph recorded that as a known reporting limitation. It is a limitation no longer: every check counts the subjects it examined, so a clause with nothing to look at reports not-observed. Read in the other direction, the captures are complementary rather than redundant: the HTTP pair evidences the transport header and status clauses, which no stdio recording can carry, and this one evidences the subscription, MRTR, prompts, resources, logging, completion and error-code clauses, which theirs never reach.

What the 43 not-observed rows still are, and why. Twenty-five are Streamable HTTP clauses in the TRAN-056…TRAN-102 band that a stdio recording structurally cannot carry — the band holds 27 judged clauses, and the two a stdio session does reach (TRAN-060, TRAN-066) are judged here. Eight are server rejection rules — BASE-031, BASE-032, BASE-035, BASE-036, VERS-001, VERS-002, VERS-008, LOG-010 — reachable only by a client that deliberately sends something malformed, which this session does not: it is the conforming capture, and a probe session is a separate recording with a separate expected report. Five need surface this server does not have (CACH-015/CACH-016 need a catalog large enough to paginate; TOOL-033/TOOL-034 need an x-mcp-header designation; PROM-017 needs a prompt carrying audio). TOOL-022 is the interesting one: rmcp’s client caches tools/list under the server’s own ttlMs, so a second listing never reaches the wire — a conforming client cannot exercise the deterministic-order clause within the TTL, which is a property of the caching feature working rather than a gap to close (the official suite’s runner does not cache, and judges it on both of its captures). The remaining four are reachable and not yet driven: TRAN-128 and DISC-002’s dual-era probe, MRTR-024, and VERS-004.

The HTTP capture of the same session

Recorded by the server, not the host, and that is the whole point. The host’s recorder sits at rmcp’s Transport seam, which carries protocol messages and nothing else — redaction by construction, and no HTTP framing to record even if it wanted to. So a host-side HTTP recording would report the Streamable HTTP clauses as not observed exactly like the stdio one. The server’s tap sits above the transport and sees the status line and the conformance-relevant headers, which is why this leg is the only recording in the corpus that can bear on them at all. Same session, both ends, one file each: the difference between the two reports is attributable to the transport and to nothing else.

At 92 of the 124 judgeable clauses it is the best-covered capture here. Its 32 not-observed rows include the server-rejection rules a conforming client never triggers, the pagination and x-mcp-header clauses this server’s surface does not reach, and TOOL-022 (rmcp’s client caches tools/list under the server’s own ttlMs, so a second listing never reaches the wire).

The probe session

A conforming client cannot exercise a rejection rule. The clauses of this revision that say what a server owes a request it must not serve all reported not observed on every recording here, because nothing had ever sent such a request. This file is that request, nine times over.

The probes are built outside rmcp, and that is the point rather than a shortcut: rmcp’s client is what makes the other captures trustworthy — it builds the _meta envelope, mirrors the SEP-2243 headers, and will not emit an ill-formed request — so it is structurally incapable of being the probe. The bytes here are the fixture.

Its verdict is a ledger, not a pass. The probe breaks client-side clauses by construction: BASE-030 because its first request omits a required _meta field, TRAN-071/TRAN-072 because two probes are about exactly those headers. Demanding a clean report would mean demanding a probe that probes nothing. So every finding is listed in conformance/probe-baseline.json with a reason, and the gate holds the set in both directions: a finding not in the ledger is a new defect or a regression, and a listed finding that stopped occurring is either a fix that should retire its entry or a check that quietly stopped firing.

It has already worked in both directions. The first probe run drew two server-side findings — the server served a request naming log level "chatty", and honoured a cursor it never issued — which went into the ledger as open defects. When they were fixed, the gate refused the change until their entries were retired, which is the half of a ratchet that is easy to leave out. Every rejection clause the probe reaches now passes; thirteen clauses pass on this capture and on no other (BASE-031, BASE-032, BASE-035, BASE-036, LOG-010, TRAN-073, TRAN-074, TRAN-075, TRAN-098, TRAN-102, VERS-001, VERS-002, VERS-008). The fabricated cursor is still sent, and judged by nothing: on 2026-10-04 PAGE-010 and PAGE-011 were excluded at this revision, because its caching rules make a cursor issued before the recording began a valid one (the registry entries record the reasoning). The server still refuses it, and a_fabricated_cursor_is_refused_on_the_stateless_revision in mcp-everything-server’s tests holds it to that.

Its provenance is weaker than the pair above, and deliberately labelled so. Both ends of this session are ours: the official suite drives servers over --url only, so there is no third-party client for stdio to be recorded against. What it still supplies that an authored fixture cannot is that neither end was written to satisfy these checks, and that the machinery producing every byte — the stateless lifecycle, the _meta envelope, the MRTR retry loop — is rmcp’s, not this repository’s.

It is also the only capture that judges a verdict rather than pinning a report. cargo xtask draft-capture fails on any finding, because a finding in a session where both ends are ours is a defect here rather than news about somebody else’s code. Regenerate with BLESS=1 cargo xtask draft-capture (which rewrites this file) followed by cargo xtask bless.

What it caught. The first recording of this session reported six failures the HTTP captures could not: TRAN-060/066/119/120 and MRTR-001, because the interactive tools still sent elicitation/create and sampling/createMessage as independent server-to-client requests — the mechanism SEP-2322 replaced — and LOG-008, because the logging tool emitted notifications/message for a request that had not asked for them. The official suite’s 2026-07-28 scenarios exercise no interactive tool and no logging tool, so an HTTP-only corpus would have shipped both defects. That is the argument for capturing both transports rather than treating one as representative.

What the captures evidence, together

The table below is generated from the committed golden reports by cargo xtask draft-coverage and verified by cargo xtask ci — per ADR-0001 these numbers are a projection of the data, not prose anyone maintains. The same gate parses every count stated elsewhere in the living Markdown — the “N of the M judgeable clauses” claims and the “N pass, M fail” verdicts alike — and fails when one disagrees, which is how the counts in this section stopped being wrong. A number quoted rather than asserted goes in backticks, and is read as a specimen instead.

Across all 5 captures, 112 of the 124 judgeable clauses are evidenced by at least one recording. Each capture’s own judged count is what that recording carried subject matter for; everything else it reports not observed rather than counting it as a pass.

The 12 clauses no capture reaches: CACH-015, CACH-016, MRTR-024, PROM-017, TOOL-033, TOOL-034, TRAN-061, TRAN-070, TRAN-079, TRAN-080, TRAN-096, VERS-004.

The probe session closed the largest group — the rejection rules — and the cancellation round closed the next. What is left divides in three, and only the first is ordinary backlog.

Eight need server surface this reference does not have: pagination for CACH-015/CACH-016, an x-mcp-header designation for TOOL-033/TOOL-034 and TRAN-079/TRAN-080/TRAN-096, and a prompt carrying audio for PROM-017.

Two cannot be reached by a conforming host at all, and saying so is more useful than closing them. VERS-004 judges the shape of an extension identifier, so a capture would have to declare an extension — and this host implements none; declaring one to light up a check is precisely the vacuous evidence the corpus exists to avoid. MRTR-024 is a server re-asking after a client’s shortfall, and a retry that omits requested input is the client violation MRTR-015 names: driving it would make the reference host non-conforming, which is the one thing this capture may not be. Both are evidenced by authored violation traces instead, where the fault belongs.

One is a recording gap rather than a session gap: TRAN-070 needs a response stream to close mid-session, and the server’s tap records no lifecycle events at all, so no HTTP capture can carry one however the session is driven.

BASE-047 used to be on this list. It left without a trace being written, because it was never a corpus gap: the check behind it counted only malformed responses as subjects, so every capture full of well-formed results reported the clause not observed — the tool saying it had seen nothing to judge about a session that had carried dozens of them.

2026-07-28 authored (corpus/draft/)

Fixtures for the in-progress revision, judged by the 2026-07-28 registry behind the draft-2026-07-28 feature. Hand-authored rather than captured: no SDK yet serves the stateless surface end to end, so these are minimal traces written to exercise one clause each, which is also what keeps a violation attributable to the check it kills.

Where a trace falsifies more than one requirement, the row says so and why. There are only two reasons, and neither is sloppy authorship: several clauses state one rule across several sections and share a check by design, or one clause’s antecedent is itself a violation by the other party — a server can only fail to reject a bad header if the client sent one. Every other row falsifies exactly the requirement it is named for.