Introduction

This is the reader’s guide to mcp-conformance, an independent, trace-based conformance toolkit for the Model Context Protocol (MCP). It judges whether an MCP implementation conforms to a specific spec revision — the one the recorded session declares, 2026-07-28 or 2025-11-25, or both at once in a single pass — and, just as importantly, it is built so that an ecosystem with its own authoritative test suite has reason to trust the verdict.

Status: pre-release. The crates are published on crates.io at 0.x; the public API and the verdicts it produces may still change between minor releases, and the changelog says so explicitly when they do. This book documents the release the version selector names; the unreleased main branch has its own book at /dev/.

The one idea

The validator’s input is a trace: an ordered, serialized record of a protocol interaction. Its engine is a pure function — &[TraceEvent] -> Report — with no network, no clock, and no I/O of its own. Capturing the trace is mcp-trace-capture’s job — a stdio wrapper and HTTP proxy that forward bytes unchanged — and judging it is the validator’s only job. That split is what makes a verdict deterministic, replayable, language- and SDK-agnostic, and — because it is reproducible from a committed file — auditable rather than a “trust us.”

The credibility mechanism is the agreement check: on every CI run the reference server is driven by the official conformance suite (pinned, at 2025-11-25; the 2026-07-28 surface is measured weekly against the suite’s pre-release), the same sessions are captured as traces, and this toolkit’s verdicts are diffed against the official runner’s. Agreement is the default; an unexplained divergence fails the build. A validator whose verdicts are checked against the recognized authority on every commit is calibrated, not merely asserted to be correct.

The crates

How this book is organized

Each chapter is a curated entry point. The planning documents — charter, ecosystem register, architecture, conformance strategy, engineering standards, security model, roadmap, and decision records — remain the single source of truth, and every chapter links to the authoritative document behind it.

• Architecture — why judge traces instead of live sessions, and the design trade-offs that follow.
• The trace format — the JSON Lines schema, with a worked example embedded from the README so it cannot drift.
• Two revisions at once — how one trace is judged against 2025-11-25 and 2026-07-28 together, and what a migration looks like when it is measured instead of estimated.
• The trace corpus — the golden good/violation fixtures and their provenance, embedded from corpus/README.md.
• Conformance results — the published tier-gap measurements of real SDKs against the official suite.

Getting started

Three steps: install the two tools, record a session of your server or client, and judge the recording. Nothing here depends on the language or SDK your implementation uses — the tools read and write bytes on the wire.

Install

cargo install mcp-trace-capture mcp-trace-validator

cargo binstall mcp-trace-capture mcp-trace-validator fetches prebuilt binaries instead of compiling (Linux x86_64 and aarch64, macOS, Windows x86_64). To build the unreleased source, which may be ahead of the release: cargo install --locked --git https://github.com/tomtom215/mcp-conformance mcp-trace-capture mcp-trace-validator.

Record a session

mcp-trace-capture sits between a client and a server, forwards every byte unchanged, and writes what crossed the wire to a trace file.

A stdio server. Have your client launch the wrapper, with the server’s own command after --:

mcp-trace-capture -o session.jsonl stdio -- python my_server.py

When a client launches it from its configuration file, give -o an absolute path with {session} in the file name — -o /tmp/traces/{session}.jsonl — so each launch writes the next numbered trace (001.jsonl, 002.jsonl, …). Clients relaunch servers, and the wrapper never overwrites a trace, so without {session} the second launch would refuse to start unless --force let it overwrite the first. The capture tool’s README shows the configuration for desktop clients and for the Python and TypeScript SDKs’ stdio clients.

A Streamable HTTP server. Run the proxy in front of it and point the client at the proxy instead:

mcp-trace-capture -o session.jsonl http --upstream http://localhost:3000
# the client connects to http://127.0.0.1:8080/mcp

Stop the proxy with Ctrl-C when the session is over.

Record one session per trace. A trace is judged as one session, so two clients recorded together look like one client breaking the rules (the validator says so when it sees more than one handshake). For a proxy that serves several clients, put {session} in -o and it writes each client session to its own file. 2026-07-28 sessions carry no session id for it to tell them apart by, so run one proxy per client there.

Judge it

mcp-trace-validator validate session.jsonl

For a session that reuses a request id, the report reads:

MCP trace validation — revision 2025-11-25 (declared by the trace)
  FAIL  BASE-003 (MUST NOT)
        seq 5: request "ping" reuses id 2, already used by the same party at seq 3
        spec: "The request ID MUST NOT have been previously used by the requestor within the same session."
        see:  https://modelcontextprotocol.io/specification/2025-11-25/basic#requests
totals: 16 pass, 1 fail, 0 warn, 86 excluded, 0 unsupported, 14 not applicable, 25 not observed
verdict: fail

Each finding names the clause (BASE-003) and its strength (MUST NOT), the event in the trace (seq 5 — every event carries a seq, its position in the recording), what was wrong, the sentence of the specification it breaks, and a link to it. The exit status is the verdict: 0 pass, 1 a clause failed, 2 the run could not judge (a bad invocation, or a trace with nothing in it), 3 the trace is malformed.

Reading a report

The report lists what needs attention; --all lists every clause. The totals always count every clause the revision has, in seven outcomes:

So a pass verdict says nothing this session did broke a clause. How much it covers is in the totals: a short session observes few clauses, and a richer session (more features, errors, pagination, both transports) puts more of the specification to the test.

Which revision. The validator judges the revision the session declares — in initialize, in each request’s _meta, or in the MCP-Protocol-Version header — and says how it chose: (declared by the trace). --revision 2025-11-25 overrides it; naming two (--revision 2025-11-25 --revision 2026-07-28) judges the session under both and shows where they differ. This build knows 2025-11-25 and 2026-07-28. A session of an older revision, such as 2025-06-18, is refused with an explanation rather than judged against rules it was not playing by.

Several traces. mcp-trace-validator validate traces/*.jsonl judges each, prints a section per trace and a closing tally, and exits with the worst trace’s status.

Other formats. --format json is the whole report as data, described by a JSON Schema; --format junit is a test report for CI; --format sarif is a SARIF 2.1.0 log for code scanning. Using it in CI covers all three.

Using it in CI

The shape is the same everywhere: the tests you already run drive your server or client through mcp-trace-capture, and a later step judges the traces they leave. The validator’s exit status is the verdict, so a failing clause fails the build.

Recording from a test suite

Wherever your tests start the server, put the capture in front of it. With the official SDKs’ stdio clients, the server’s command moves behind --:

StdioServerParameters(
    command="mcp-trace-capture",
    args=["-o", "traces/{session}.jsonl", "stdio", "--", "python", "my_server.py"],
)

For an HTTP server, start the proxy before the tests and stop it after:

mcp-trace-capture -o 'traces/http-{session}.jsonl' http --upstream http://127.0.0.1:3000 &
proxy=$!
npm test          # with the client pointed at http://127.0.0.1:8080/mcp
kill -INT $proxy
wait $proxy

{session} gives each test session its own numbered trace: each launch of the stdio wrapper, and each client session through the proxy (the traces/ directory must exist). The repository’s interop/ directory does exactly this with the official TypeScript and Python SDKs, and its run.sh is a working example of both transports.

GitHub Actions

The repository is itself an action. After the step that records the traces:

- uses: tomtom215/mcp-conformance@<commit or tag>
  with:
    traces: traces/*.jsonl
- uses: github/codeql-action/upload-sarif@<sha>
  if: always()
  with:
    sarif_file: mcp-conformance.sarif
    category: mcp-conformance

The action builds mcp-trace-validator from the ref you name (or, with version: 0.6.0, downloads that release’s prebuilt binary and checks its checksum), prints the findings to the log and the job summary, writes mcp-conformance.sarif and mcp-conformance-junit.xml, and fails the step when a clause fails. Uploading the SARIF log needs permissions: security-events: write on the job; findings then appear in the repository’s code scanning view, each on the trace line that holds the event. The inputs — revision, strict, fail-on-findings, the output paths — are listed in action.yml.

Any other CI

mcp-trace-validator validate traces/*.jsonl                       # findings in the log
mcp-trace-validator validate --format junit traces/*.jsonl > mcp-conformance.xml
mcp-trace-validator validate --format sarif traces/*.jsonl > mcp-conformance.sarif

Each invocation exits with the same status, so run the human report as the gate and the other two for the reports your CI ingests (GitLab, Jenkins and most others read JUnit). Several traces give one document: a JUnit suite per trace and revision, one SARIF run with each result in its own trace file.

Choosing what fails the build

A trace that cannot be judged — empty, malformed, or of a revision this build does not know — exits 2 or 3, never 0: a broken recording step fails the build rather than passing it.

Troubleshooting

Each entry starts from what you see.

The validator

error: … judged no requirement at all — an empty or contentless trace is a capture that failed, not a session that conformed (exit 2). The trace has no messages: the client never connected, connected to the server directly instead of through the capture, or the capture was stopped before the session. Check the capture’s exit summary, which counts what it recorded.

error: … records 1 client message(s) and none from the server (exit 2). The client spoke and the server never answered: the capture recorded that it could not reach the server (transport-abort), or the client’s initialize went unanswered. The client’s side alone would pass, so the run is refused rather than reported green. Check that the server is running and reachable — for the HTTP proxy, that --upstream names it — and record again.

error: the trace declares protocol revision(s) 2025-06-18, which no available registry describes (exit 2). The session ran an older revision than this build has requirements for (2025-11-25 and 2026-07-28). Judging it against either would report that revision’s rules as violations, so it is refused. --revision 2025-11-25 judges it anyway, if the differences do not matter to you.

note: this trace records 2 sessions and is judged as one. Two clients, or one client twice, were recorded into one file — a proxy left running across test runs, or a client that reconnected. Findings that span them (a request id the second session reuses, a session id that changed) come from recording them together. Put {session} in the capture’s -o (-o traces/{session}.jsonl) to get a trace per session.

note: the trace declares no protocol revision; judging it against 2026-07-28. The recording holds no initialize, no _meta protocol version and no MCP-Protocol-Version header — often a trace that starts mid-session. Pass --revision to say which rules apply.

error: malformed trace: … (exit 3). The file is not a trace as written by a recorder:

The capture tool

… exists; appending would break the trace's sequence numbers (exit 2). The output file is there from an earlier run. Put {session} in the file name for a new numbered trace each run — the usual choice in a client’s configuration, which relaunches servers — or use --force to overwrite.

The server works directly but not through the HTTP proxy, with 403. The server checks the Host header, which the proxy rewrites to name the upstream. Add --preserve-host.

TRAN-004 (or TRAN-117) fails on a line like "Server starting...". The server writes something other than protocol messages to stdout — log lines, a banner. The capture records each such line as it was written, and the validator judges it: it is a stdio transport violation, and the same line fails BASE-008 (every message follows JSON-RPC). Send logs to stderr.

A client configuration cannot find mcp-trace-capture. Desktop applications often start without your shell’s PATH. Use the absolute path to the binary as the command, for example the output of which mcp-trace-capture.

Still stuck

Open an issue with the command, the full output, and — if you can share it — the trace. Traces record message content in full; read one before attaching it.

Architecture

This chapter orients you in the design. The authoritative treatment — ten decisions, each with its cost — is the standalone design note, written for an upstream audience:

📄 Design note: trace-validation architecture and trade-offs

The internal workspace layout and dependency rules live in 02-architecture.md.

The foundational split: capture, then judge

The validator judges a recorded trace, not a live session. Its engine is a pure function with no network, clock, or I/O. Whoever owns the socket — the server’s session tap, the host’s capture wrapper, or mcp-trace-capture for any other implementation — produces the trace, assigning the total-order seq at capture. The judge only judges.

This buys determinism, replayability, SDK-agnostic input, and auditable verdicts. It costs one hard thing: capture fidelity becomes a first-class engineering problem — interleaving, partial writes, and SSE framing all have to be recorded faithfully by the component that holds the bytes.

The decisions that follow, in brief

• The judge never links the SDK it judges. mcp-trace-validator does not depend on rmcp. A validator built on the SDK under test would inherit that SDK’s interpretation of the spec as ground truth — exactly the thing being examined. The cost is re-expressing the protocol as our own data.
• The spec as data. Every normative clause becomes a registry record with its RFC 2119 level, actor, a verbatim source quote, an optional capability gate, and either one or more mechanical checks or a written reason it cannot be judged from a trace. There is no “not yet looked at” state. A scheduled job re-verifies every quote against the live spec text so the registry cannot silently drift.
• Determinism by canonicalization. All JSON is canonicalized (RFC 8785) before any comparison, so a report is byte-identical across platforms and runs. Correctness here is a fixpoint property, tested as one — a real bug hid until a fuzzer first generated -0.0.
• A pass has to be earned. A requirement gated on a capability that was never negotiated is reported as not-applicable, and one whose subject matter the session never carried is reported not-observed — never as a vacuous pass. Every check counts the subjects it considered, so the report can tell “complied with” from “never came up”. Inflating a pass rate with checks that had nothing to look at is how a conformance tool loses credibility.
• Calibration against the authority. The agreement check (see the Introduction) diffs our verdicts against the official runner’s on every CI run; an unexplained divergence fails the build, and a stale recorded divergence fails it too.
• Falsifiability. Every check is killed by at least one committed injected-violation trace; a check with no killer trace is unfalsifiable. The corpus is both the regression suite and the evidence each check does what it claims.
• Verdicts are a contract. A change that makes a previously-passing trace fail is a breaking change even when no function signature moved; it is named in the changelog and the version is bumped accordingly. API-signature compatibility is additionally checked mechanically against the published crates.io baseline.

The trace format

A trace is JSON Lines: one event per line. Each event carries a capture-assigned seq (the only ordering authority — never inferred later), a direction (client-to-server / server-to-client), a transport, and a kind:

• message events hold the JSON-RPC payload (as parsed JSON: every value, not its whitespace, member order, or the spelling of its numbers);
• http events record the conformance-relevant headers, a response’s status, and a client request’s method — Streamable HTTP binds different obligations to POST, GET, and DELETE, so a clause addressed to one of them is judged only where the recording says which it was; and
• lifecycle events mark transport open/close.

The format is published as a JSON Schema (draft 2020-12), trace-event.schema.json, for recorders written in any language. It states every rule the validator’s reader applies to one record — a test runs both over every corpus record and one violation of each rule — and says where JSON Schema cannot follow: the document-level rules (one record per line, seq strictly increasing), and integers written as 1.0, which the reader refuses. Unknown members are ignored, so producers may add their own.

The redaction rules that keep credential-bearing headers out of a trace by construction are in 05-security-model.md.

One worked example

The example below is embedded verbatim from the README, where a test (readme_examples.rs) pins it to the validator’s real output — so what you read here cannot drift from what the tool actually produces. It is a session that reuses a request ID, and the verdict that catches it:

{"seq":0,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"my-host","version":"1.0.0"}}}}
{"seq":1,"direction":"server-to-client","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","capabilities":{"tools":{}},"serverInfo":{"name":"my-server","version":"1.0.0"}}}}
{"seq":2,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","method":"notifications/initialized"}}
{"seq":3,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"tools/list"}}

and the validator answers with the violated clause, addressed to the offending event:

  FAIL  BASE-003 (MUST NOT)
        seq 3: request "tools/list" reuses id 1, already used by the same party at seq 0
        spec: "The request ID MUST NOT have been previously used by the requestor within the same session."
        see:  https://modelcontextprotocol.io/specification/2025-11-25/basic#requests
totals: 17 pass, 1 fail, 0 warn, 86 excluded, 0 unsupported, 6 not applicable, 32 not observed
verdict: fail

The totals line distinguishes the verdict’s components, and two of them are not passes. The not-applicable rows are capability-gated requirements this session never negotiated (the resources and prompts clauses); the not-observed rows are clauses whose subject matter never appeared at all — nothing was paginated, no binary content was sent, no error was returned. A four-line session earns 17 passes, not a hundred. See Architecture for why that distinction is load-bearing.

Two revisions at once

MCP is a dated specification. 2026-07-28 is the current revision, and 2025-11-25 stays in wide use while implementations migrate. 2026-07-28 removes the initialize handshake outright (SEP-2575), makes every request carry its own _meta envelope, and adds caching hints, multi-round-trip requests, subscriptions, and a discovery probe. That is not a version bump anyone should take on faith.

So this toolkit judges both, and can judge one trace against both in a single pass. A migration stops being an inventory of changes somebody read in a changelog and becomes a measurement.

What “both” means here

The two registries are extracted per revision rather than sharing entries: each clause is quoted from its own revision’s published text, and a clause restated with different words gets its own ID. 2025-11-25’s BASE-003 forbids reusing a request ID within a session; 2026-07-28’s BASE-045 forbids reusing one that is still in flight, so reuse after a response is now legal. Those are two clauses, not one clause with a footnote — and pointing the new one at the old one’s check would have reported conforming traces as violations.

The cost of that method is that no clause is in force at both revisions, so a side-by-side report never shows one clause passing at one revision and failing at the other. What it shows instead is presence, which is what a migration actually consists of.

A trace judged against both

This is a conforming 2025-11-25 handshake — the client offers a version, the server answers, the client confirms:

{"seq":0,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"corpus-client","version":"0.1.0"}}}}
{"seq":1,"direction":"server-to-client","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","capabilities":{},"serverInfo":{"name":"corpus-server","version":"0.1.0"}}}}
{"seq":2,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","method":"notifications/initialized"}}

Judged against both revisions at once:

$ mcp-trace-validator validate --revision 2025-11-25 --revision 2026-07-28 handshake.jsonl

Four rows out of the report say the whole thing:

  LIFE-001   (MUST)  2025-11-25=pass  2026-07-28=absent  *differs
  DISC-001   (MUST)  2025-11-25=absent  2026-07-28=not-observed  *differs
  BASE-030   (MUST)  2025-11-25=absent  2026-07-28=fail  *differs
  BASE-045   (MUST NOT)  2025-11-25=absent  2026-07-28=pass  *differs

• LIFE-001 — “the client MUST initiate the lifecycle” — is a clause the newer revision removes. It reads pass then absent.
• DISC-001 — the server/discover probe that replaces the handshake — is a clause the newer revision adds, and this session carried nothing it binds to, so it reads not-observed rather than counting as a pass.
• BASE-030 — every request must carry an io.modelcontextprotocol/clientCapabilities _meta member — is a clause the newer revision adds that this session breaks. It reads fail, and it is the migration work this trace has actually discovered.
• BASE-045 — the narrowed request-ID rule — reads pass, because reusing an ID after its response is exactly what the new text permits.

And the verdict splits:

per revision:
  2025-11-25: 17 pass, 0 fail, 0 warn, 86 excluded, 0 unsupported, 14 not applicable, 25 not observed — verdict pass
  2026-07-28: 14 pass, 8 fail, 0 warn, 149 excluded, 0 unsupported, 0 not applicable, 102 not observed — verdict fail
overall verdict: fail

A conforming session today, and eight concrete failures tomorrow. Both summary lines account for every clause in their revision — the counts add up to 142 and 273 — because a line that quietly omits an outcome is a line that overstates what was measured.

The example above is executed by a test (book_examples.rs) against the real validator on every cargo test, so this page cannot drift from what the tool actually prints.

Which revision a trace is judged against

A trace is not silent about which revision it belongs to — the handshake states it, a stateless session states it in every request’s _meta, and the HTTP transport states it in a header. So by default the validator judges a trace against the revision it declares, and says so in the report’s first line: revision 2026-07-28 (declared by the trace). A trace declaring nothing is judged against the newest supported revision, with a note on stderr; a trace declaring only revisions this build cannot judge is refused (exit 2) rather than judged against the wrong rules.

Judging against the wrong rules is easy to do on purpose, with --revision. Force 2025-11-25 onto a 2026-07-28 recording and every clause the two revisions disagree about becomes a finding: the session opens with tools/list rather than initialize, so LIFE-001 fails; it reuses request ids after their responses, so BASE-003 fails. Both are correct answers to the question that was asked, and neither is a defect in the implementation. This session says 2026-07-28 twice over:

{"seq":0,"direction":"client-to-server","transport":"streamable-http","kind":"http","method":"POST","headers":{"accept":"application/json, text/event-stream","mcp-protocol-version":"2026-07-28"}}
{"seq":1,"direction":"client-to-server","transport":"streamable-http","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"tools/list","params":{"_meta":{"io.modelcontextprotocol/protocolVersion":"2026-07-28","io.modelcontextprotocol/clientCapabilities":{}}}}}
{"seq":2,"direction":"server-to-client","transport":"streamable-http","kind":"message","payload":{"jsonrpc":"2.0","id":1,"result":{"tools":[],"resultType":"complete"}}}

so validating it with --revision 2025-11-25 says so, above the rows and again under the verdict:

  NOTE  this session declares protocol revision 2026-07-28, not 2025-11-25.
        Every outcome here judges it against rules it was not playing by;
        re-run with `--revision 2026-07-28` to judge it against its own.

It is deliberately quiet. A session that proposed one revision and negotiated another has touched both, so judging it against either draws no note; a version a server refused states nothing, because the session never ran under it; and a recording that declares no revision at all gets no note, since there is nothing to disagree with.

The four ways a clause is not a pass

A side-by-side report is where these stop being pedantry, because all four appear at once and they mean different things:

None of them is a pass, and the report never quietly promotes one. That rule is the whole reason the numbers here are smaller than a conformance tool’s numbers usually are.

What is measured at 2026-07-28 today

• The reference server serves the stateless surface with --protocol-version 2026-07-28, and the official suite’s 2026-07-28 scenarios score 42 passing / 0 failing against it.
• Five committed captures — a conforming session over each transport, a probe session of deliberately malformed requests, and the official runner’s two — evidence 112 of the 124 judgeable clauses between them. What no capture reaches is named, one clause at a time, in the corpus chapter.
• cargo xtask draft-readiness re-runs that measurement and ratchets it against a committed baseline: any change in either direction fails the build, so progress toward the next revision is recorded in the commit that earned it rather than estimated.

That ratchet has already paid for itself in the direction nobody plans for. Through suite 0.2.0-alpha.9 the runner scored the 2025-11-25 server and the 2026-07-28 server identically — its scenarios exercise features, not revisions, so it could not tell them apart. This project’s registry could, and flagged the 2025-11-25 server for returning cacheable results without the ttlMs hint the new revision requires (CACH-001). Six weeks later alpha.11 added a schema check and found the same clause, independently, from the other side. The registry’s finding is recorded as superseded rather than deleted, because the supersession is itself the result: a negative finding about an instrument expires when the instrument improves.

Full detail is in the ecosystem register (row 1.5i) and the conformance strategy.

Trace corpus

Fixtures for the golden-corpus tests (crates/mcp-trace-validator/tests/golden.rs):

• good/ — sessions that must validate with verdict pass.
• violations/ — single-issue sessions, each falsifying at least one check; named after the requirement whose check they exist to kill.
• draft/{good,violations}/ — the same two kinds for revision 2026-07-28, judged against that revision’s registry behind the draft-2026-07-28 feature.
• draft/captured/ — sessions recorded off the wire from implementations this repository did not write. No verdict is asserted for them; their whole report is byte-pinned instead. See Captured versus authored below.
• golden/ — the byte-pinned expected report for every trace, with golden/draft/ holding the 2026-07-28 corpus’s. Regenerate only via cargo xtask bless and review the diff like code.
• golden/exclusions/ — one ledger per revision, holding the clauses that revision’s registry declines to judge from a trace, and why.

The revisions keep separate golden directories because both name traces after requirement IDs drawn from their own registry: base-045-… is one clause at 2025-11-25 and a different one at 2026-07-28, so a shared directory would let one revision’s golden answer for the other’s trace.

What a golden pins

A report states two kinds of fact, and they are pinned separately (ADR-0013).

golden/<stem>.json holds what the trace decided: the revision, the whole totals, and every pass / fail / warn / not-observed / not-applicable / unsupported row, in registry order.

golden/exclusions/<revision>.json holds what the registry decided: the excluded rows, whose outcome and prose come from the registry alone and are identical for every trace judged against it — engine::build_row never consults the session for them. One distinct set per revision, and before the split every golden restated its revision’s whole set: at the time that was 59% of the corpus’s lines.

Every cell is verified against the corpus by cargo xtask ci (xtask::coverage::corpus). It is checked rather than trusted because it had already drifted twice: this paragraph is prose, and prose does not recount itself.

Nothing stopped being pinned. Splicing a golden back together with its ledger reproduces the whole report, and assert_reconstructs_the_full_report proves it does on every trace, on every run — so a judged row that went missing, a ledger that drifted from its registry, or rows that interleave in any order but the registry’s all fail the suite. totals.excluded stays in every golden as the per-trace tie back to the ledger: a clause entering or leaving the excluded set moves every golden for that revision, one line each.

Only excluded collapses. not-observed and not-applicable are per-trace evidence, and the table’s third row is the measure of it: near one distinct set per golden, in both revisions. Collapsing those would delete exactly what makes the captures differentiate each other.

Violation naming contract

A violation trace is named area-nnn-<slug>.jsonl and must produce a Fail/Warn row with findings for exactly requirement AREA-NNN — the golden harness enforces the attribution by name (violation_traces_fail_and_match_goldens, and draft::draft_violation_traces_fail_and_match_goldens for draft/), so a defect re-routed to a different requirement cannot re-bless silently. A stem that does not begin with a requirement ID fails the suite loudly.

Captured versus authored

Everything in good/, violations/ and draft/{good,violations}/ is authored: hand-written to exercise one clause. That is what makes them precise, and it is also their limit — an authored fixture can only ever confirm its author’s reading of the specification. A check that is wrong in the same way the author is wrong passes its unit tests and its corpus alike, and neither notices.

draft/captured/ exists to close that gap. Its traces are real recordings of traffic between implementations neither of which was written to satisfy these checks, so they can disagree with us — and when they do, the disagreement is information rather than a test failure. Their goldens are pinned without any pass/fail expectation, because a real implementation is whatever it actually is; what the golden protects is which requirements fire, on which events, with what detail. A check that starts misfiring on real traffic moves it — which is precisely the half of the report that stays in the per-trace file.

Provenance ledger

Every trace’s origin, in one reviewable place that survives history rewrites (the invariant test in golden.rs fails if a trace is added without a ledger row). All current traces share one provenance: hand-authored for this repository as synthetic sessions (no third-party traffic, no recorded production data), written against the 2025-11-25 spec text fetched live from modelcontextprotocol.io on 2026-06-09 (re-verified clause-by-clause against the live text on 2026-06-11) and validated against the embedded registry at the commit that introduced them.

Traces under draft/captured/ are the exception and record their own origin below: the implementations on both ends, the tool that recorded them, and when.

good/

violations/

Each file injects exactly the violation its name states; golden/ shows every row the trace decided, including any intrinsic secondary findings the injected defect causes (a malformed notification also fails lifecycle accounting, for example).

2026-07-28 captured (corpus/draft/captured/)

Two recordings of the same client driving the same scenarios, differing in one variable: which revision the server was serving. That is what makes the pair worth committing — a single capture shows what one implementation does, while a matched pair shows what the revision costs, and every difference between the two reports is attributable to that one change.

Both carry server/discover, tools/list, tools/call, completion/complete, resources/{list,read}, prompts/{list,get} and progress notifications; every request carries the revision’s per-request _meta envelope and its SEP-2243 metadata headers, and neither carries an initialize anywhere. Their tools/list results differ by one entry, which is the point of recording both: test_url_elicitation is listed by the legacy server and not by the stateless one, because URL-mode elicitation is a feature 2026-07-28 removed.

The one finding. CACH-001 — the legacy server’s complete results carry no ttlMs, on resources/list and three resources/reads. That is our 2025-11-25 server being held to a revision it does not implement, which is the correct answer, and the stateless capture is the control that proves the check reports the server rather than the recording.

Two findings that used to be here were ours, not the implementations’. An earlier version of this pair reported TRAN-058 (client POSTs missing Mcp-Method/Mcp-Name) and TRAN-068 (SSE responses missing X-Accel-Buffering: no). Both were artifacts of the tap’s recording allowlist, which predated SEP-2243 and did not record those headers — the client sent them (rmcp’s transport rejects a 2026-07-28 request without Mcp-Method before dispatch, and the suite scored 23/23) and the server sent them (rmcp sets X-Accel-Buffering on every SSE response it builds). The allowlist was extended; both clauses now pass on both captures. The lesson is recorded rather than quietly fixed: a check can only be as honest as the recording it reads, and a capture path that silently drops evidence manufactures findings against conforming implementations.

The same lesson, a third time, one field over (2026-08-20). The tap recorded an exchange’s headers but not its method, so TRAN-025 and TRAN-039 — the Accept obligations of a POST and of a GET — were enforced as one rule applied to every client request whatever its form. That was wrong twice over, and both halves were found by pointing this workspace’s own reference host at its own reference server over HTTP and validating the tap: the conforming session-teardown DELETE, which owes no Accept header at all, drew two MUST-level failures, while a POST omitting application/json — the verbatim violation TRAN-025 names — was reported pass, because the one shared rule could only demand what both clauses had in common. EventBody::Http now carries the method, each clause is judged against the requests it binds, and an exchange whose method the capture did not record is judged by neither. These two captures were re-recorded so the method is present; that is why their Recorded by row reads 2026-08-20.

Why the runner’s 23/23 and our 60-vs-61 were both right — and how the runner caught up. The suite’s 2026-07-28 scenarios exercise features — list a thing, call a thing, read a thing — and a 2025-11-25 server answers all of them, because rmcp serves a per-request-versioned POST whichever revision the handler advertises. At 0.2.0-alpha.9 that was the whole of the runner’s reading, so it scored both servers 23/23. The registry here judges the specification’s prose instead, so it saw the one place the two servers actually differ: CACH-001, no ttlMs on cacheable results.

0.2.0-alpha.11 found the same thing, six weeks later. Its new wire-schema-valid check validates every message against the negotiated revision’s JSON schema, and it fails the 2025-11-25 leg’s four resource scenarios for must have required property 'cacheScope' / 'ttlMs' — that clause, independently. Re-measured, the legacy leg scores 37 passing / 4 failing and the stateless leg 41 / 0, so the runner now separates the two servers it could not tell apart before. Neither instrument was wrong; they were measuring different things, and one of them saw this first. The pair is still the evidence for taking both readings — now with a worked example of the prose-level reading arriving earlier than the schema-level one.

The 64 not-observed rows are the honest denominator: of the 124 clauses this revision’s registry can judge, these sessions carried subject matter for 60. They open no subscription, present no cursor, draw no error, and send no malformed _meta, so those clauses are neither passed nor failed here — they are untested, and the report says which.

Regenerate both with cargo xtask draft-readiness and copy target/draft-readiness/<revision>/tap/001-stateless.jsonl; the ephemeral port in the host header changes per run, so re-copying is a deliberate act with a golden diff, not something to do casually.

The stdio capture

It is the only capture that exercises subscriptions/listen. The official suite drives no subscription, so the four judged SUBS clauses — and BASE-039, which binds a subscription stream’s notifications — are backed by this file alone: an acknowledgment, a filter the server narrowed, four tagged notifications and an empty graceful-closure result. The same holds for the MRTR clauses. In the two HTTP captures every one of those rows now reads not observed; here they read pass.

Two defects were found by enriching it, and neither could have been found any other way. Until the session read a resource the catalog does not have, nothing in this workspace had ever exercised the server’s not-found path at this revision — and it answered -32002, which 2026-07-28 withdraws outright (basic/index#error-codes lists it under “Implementations of this protocol version MUST NOT emit these codes” and names -32602 as the replacement). The second was ours: the host’s recorder assigned seq when the send future completed rather than when the message was handed to the transport, so a reply received while that future was still pending landed in the trace ahead of its own request. That reads as the server answering an id nobody asked, and it failed BASE-046 on roughly one capture in three. Both are fixed, both have regression tests, and both are the argument for driving more traffic rather than more fixtures.

That asymmetry is the reason to keep all three, and it used to be invisible. Until the vacuous-pass fix, those rows read identically across the three files — pass everywhere, whether or not the session had opened a stream — and this paragraph recorded that as a known reporting limitation. It is a limitation no longer: every check counts the subjects it examined, so a clause with nothing to look at reports not-observed. Read in the other direction, the captures are complementary rather than redundant: the HTTP pair evidences the transport header and status clauses, which no stdio recording can carry, and this one evidences the subscription, MRTR, prompts, resources, logging, completion and error-code clauses, which theirs never reach.

What the 43 not-observed rows still are, and why. Twenty-five are Streamable HTTP clauses in the TRAN-056…TRAN-102 band that a stdio recording structurally cannot carry — the band holds 27 judged clauses, and the two a stdio session does reach (TRAN-060, TRAN-066) are judged here. Eight are server rejection rules — BASE-031, BASE-032, BASE-035, BASE-036, VERS-001, VERS-002, VERS-008, LOG-010 — reachable only by a client that deliberately sends something malformed, which this session does not: it is the conforming capture, and a probe session is a separate recording with a separate expected report. Five need surface this server does not have (CACH-015/CACH-016 need a catalog large enough to paginate; TOOL-033/TOOL-034 need an x-mcp-header designation; PROM-017 needs a prompt carrying audio). TOOL-022 is the interesting one: rmcp’s client caches tools/list under the server’s own ttlMs, so a second listing never reaches the wire — a conforming client cannot exercise the deterministic-order clause within the TTL, which is a property of the caching feature working rather than a gap to close (the official suite’s runner does not cache, and judges it on both of its captures). The remaining four are reachable and not yet driven: TRAN-128 and DISC-002’s dual-era probe, MRTR-024, and VERS-004.

The HTTP capture of the same session

Recorded by the server, not the host, and that is the whole point. The host’s recorder sits at rmcp’s Transport seam, which carries protocol messages and nothing else — redaction by construction, and no HTTP framing to record even if it wanted to. So a host-side HTTP recording would report the Streamable HTTP clauses as not observed exactly like the stdio one. The server’s tap sits above the transport and sees the status line and the conformance-relevant headers, which is why this leg is the only recording in the corpus that can bear on them at all. Same session, both ends, one file each: the difference between the two reports is attributable to the transport and to nothing else.

At 92 of the 124 judgeable clauses it is the best-covered capture here. Its 32 not-observed rows include the server-rejection rules a conforming client never triggers, the pagination and x-mcp-header clauses this server’s surface does not reach, and TOOL-022 (rmcp’s client caches tools/list under the server’s own ttlMs, so a second listing never reaches the wire).

The probe session

A conforming client cannot exercise a rejection rule. The clauses of this revision that say what a server owes a request it must not serve all reported not observed on every recording here, because nothing had ever sent such a request. This file is that request, nine times over.

The probes are built outside rmcp, and that is the point rather than a shortcut: rmcp’s client is what makes the other captures trustworthy — it builds the _meta envelope, mirrors the SEP-2243 headers, and will not emit an ill-formed request — so it is structurally incapable of being the probe. The bytes here are the fixture.

Its verdict is a ledger, not a pass. The probe breaks client-side clauses by construction: BASE-030 because its first request omits a required _meta field, TRAN-071/TRAN-072 because two probes are about exactly those headers. Demanding a clean report would mean demanding a probe that probes nothing. So every finding is listed in conformance/probe-baseline.json with a reason, and the gate holds the set in both directions: a finding not in the ledger is a new defect or a regression, and a listed finding that stopped occurring is either a fix that should retire its entry or a check that quietly stopped firing.

It has already worked in both directions. The first probe run drew two server-side findings — the server served a request naming log level "chatty", and honoured a cursor it never issued — which went into the ledger as open defects. When they were fixed, the gate refused the change until their entries were retired, which is the half of a ratchet that is easy to leave out. Every rejection clause the probe reaches now passes; thirteen clauses pass on this capture and on no other (BASE-031, BASE-032, BASE-035, BASE-036, LOG-010, TRAN-073, TRAN-074, TRAN-075, TRAN-098, TRAN-102, VERS-001, VERS-002, VERS-008). The fabricated cursor is still sent, and judged by nothing: on 2026-10-04 PAGE-010 and PAGE-011 were excluded at this revision, because its caching rules make a cursor issued before the recording began a valid one (the registry entries record the reasoning). The server still refuses it, and a_fabricated_cursor_is_refused_on_the_stateless_revision in mcp-everything-server’s tests holds it to that.

Its provenance is weaker than the pair above, and deliberately labelled so. Both ends of this session are ours: the official suite drives servers over --url only, so there is no third-party client for stdio to be recorded against. What it still supplies that an authored fixture cannot is that neither end was written to satisfy these checks, and that the machinery producing every byte — the stateless lifecycle, the _meta envelope, the MRTR retry loop — is rmcp’s, not this repository’s.

It is also the only capture that judges a verdict rather than pinning a report. cargo xtask draft-capture fails on any finding, because a finding in a session where both ends are ours is a defect here rather than news about somebody else’s code. Regenerate with BLESS=1 cargo xtask draft-capture (which rewrites this file) followed by cargo xtask bless.

What it caught. The first recording of this session reported six failures the HTTP captures could not: TRAN-060/066/119/120 and MRTR-001, because the interactive tools still sent elicitation/create and sampling/createMessage as independent server-to-client requests — the mechanism SEP-2322 replaced — and LOG-008, because the logging tool emitted notifications/message for a request that had not asked for them. The official suite’s 2026-07-28 scenarios exercise no interactive tool and no logging tool, so an HTTP-only corpus would have shipped both defects. That is the argument for capturing both transports rather than treating one as representative.

What the captures evidence, together

The table below is generated from the committed golden reports by cargo xtask draft-coverage and verified by cargo xtask ci — per ADR-0001 these numbers are a projection of the data, not prose anyone maintains. The same gate parses every count stated elsewhere in the living Markdown — the “N of the M judgeable clauses” claims and the “N pass, M fail” verdicts alike — and fails when one disagrees, which is how the counts in this section stopped being wrong. A number quoted rather than asserted goes in backticks, and is read as a specimen instead.

Across all 5 captures, 112 of the 124 judgeable clauses are evidenced by at least one recording. Each capture’s own judged count is what that recording carried subject matter for; everything else it reports not observed rather than counting it as a pass.

The 12 clauses no capture reaches: CACH-015, CACH-016, MRTR-024, PROM-017, TOOL-033, TOOL-034, TRAN-061, TRAN-070, TRAN-079, TRAN-080, TRAN-096, VERS-004.

The probe session closed the largest group — the rejection rules — and the cancellation round closed the next. What is left divides in three, and only the first is ordinary backlog.

Eight need server surface this reference does not have: pagination for CACH-015/CACH-016, an x-mcp-header designation for TOOL-033/TOOL-034 and TRAN-079/TRAN-080/TRAN-096, and a prompt carrying audio for PROM-017.

Two cannot be reached by a conforming host at all, and saying so is more useful than closing them. VERS-004 judges the shape of an extension identifier, so a capture would have to declare an extension — and this host implements none; declaring one to light up a check is precisely the vacuous evidence the corpus exists to avoid. MRTR-024 is a server re-asking after a client’s shortfall, and a retry that omits requested input is the client violation MRTR-015 names: driving it would make the reference host non-conforming, which is the one thing this capture may not be. Both are evidenced by authored violation traces instead, where the fault belongs.

One is a recording gap rather than a session gap: TRAN-070 needs a response stream to close mid-session, and the server’s tap records no lifecycle events at all, so no HTTP capture can carry one however the session is driven.

BASE-047 used to be on this list. It left without a trace being written, because it was never a corpus gap: the check behind it counted only malformed responses as subjects, so every capture full of well-formed results reported the clause not observed — the tool saying it had seen nothing to judge about a session that had carried dozens of them.

2026-07-28 authored (corpus/draft/)

Fixtures for the in-progress revision, judged by the 2026-07-28 registry behind the draft-2026-07-28 feature. Hand-authored rather than captured: no SDK yet serves the stateless surface end to end, so these are minimal traces written to exercise one clause each, which is also what keeps a violation attributable to the check it kills.

Where a trace falsifies more than one requirement, the row says so and why. There are only two reasons, and neither is sloppy authorship: several clauses state one rule across several sections and share a check by design, or one clause’s antecedent is itself a violation by the other party — a server can only fail to reject a bad header if the client sent one. Every other row falsifies exactly the requirement it is named for.

Conformance results

A conformance toolkit should be willing to measure real implementations — its own and others’ — against the official suite, and publish what it finds. These measurements are stewardship artifacts: every number is the official suite’s number, captured by running it, with this project’s registry used only to read each failure back to a specific spec clause. Every claim has a command that reproduces it.

This project’s reference implementations

The reference server and host are a hard CI gate, not a one-off measurement: on every run, the conformance job drives the pinned official suite against mcp-everything-server (server scenarios) and mcp-reference-host (client scenarios), and replays the captured sessions through the validator for the agreement check. A regression fails the build. The live status is whatever the latest main run reports.

rmcp tier-gap report

The published measurement of the official Rust SDK (rmcp) against the suite is the worked example of the method:

📄 rmcp tier-gap report — 2025-11-25

It records the server-scenario pass count, the requirement-level reading of each failure, a concrete close-the-gap checklist, and the exact commands to reproduce it — including a note on why the suite’s tier-check aggregate is not the figure to trust (it is GitHub-token-gated and its conformance counter carries a known upstream counting bug). The headline finding at the report’s measurement date: two failing server scenarios, neither of which is a 2025-11-25 normative-clause violation — one sits just below the spec’s RFC 2119 floor (template-argument substitution is schema prose), the other is a SEP-1330 serialization bug already filed upstream. That is the kind of distinction a requirement-level tool exists to draw; see the report for the current count and the close-the-gap steps.