Introduction

This is the reader’s guide to mcp-conformance, an independent, trace-based conformance toolkit for the Model Context Protocol (MCP). It judges whether an MCP implementation conforms to a specific spec revision (2025-11-25 today) — and, just as importantly, it is built so that an ecosystem with its own authoritative test suite has reason to trust the verdict.

Status: pre-release. The crates are published on crates.io at 0.x; the public API and the verdicts it produces may still change between minor releases, and the changelog says so explicitly when they do. This book tracks main.

The one idea

The validator’s input is a trace: an ordered, serialized record of a protocol interaction. Its engine is a pure function — &[TraceEvent] -> Report — with no network, no clock, and no I/O of its own. Capturing the trace is somebody else’s job; judging it is the validator’s only job. That split is what makes a verdict deterministic, replayable, language- and SDK-agnostic, and — because it is reproducible from a committed file — auditable rather than a “trust us.”

The credibility mechanism is the agreement check: on every CI run the reference server is driven by the official conformance suite, the same sessions are captured as traces, and this toolkit’s verdicts are diffed against the official runner’s. Agreement is the default; an unexplained divergence fails the build. A validator whose verdicts are checked against the recognized authority on every commit is calibrated, not merely asserted to be correct.

The crates

How this book is organized

Each chapter is a curated entry point. The planning documents — charter, ecosystem register, architecture, conformance strategy, engineering standards, security model, roadmap, and decision records — remain the single source of truth, and every chapter links to the authoritative document behind it.

• Architecture — why judge traces instead of live sessions, and the design trade-offs that follow.
• The trace format — the JSON Lines schema, with a worked example embedded from the README so it cannot drift.
• The trace corpus — the golden good/violation fixtures and their provenance, embedded from corpus/README.md.
• Conformance results — the published tier-gap measurements of real SDKs against the official suite.

Architecture

This chapter orients you in the design. The authoritative treatment — ten decisions, each with its cost — is the standalone design note, written for an upstream audience:

📄 Design note: trace-validation architecture and trade-offs

The internal workspace layout and dependency rules live in 02-architecture.md.

The foundational split: capture, then judge

The validator judges a recorded trace, not a live session. Its engine is a pure function with no network, clock, or I/O. Whoever owns the socket — the server’s session tap, the host’s capture wrapper, or any external proxy — produces the trace, assigning the total-order seq at capture. The judge only judges.

This buys determinism, replayability, SDK-agnostic input, and auditable verdicts. It costs one hard thing: capture fidelity becomes a first-class engineering problem — interleaving, partial writes, and SSE framing all have to be recorded faithfully by the component that holds the bytes.

The decisions that follow, in brief

• The judge never links the SDK it judges. mcp-trace-validator does not depend on rmcp. A validator built on the SDK under test would inherit that SDK’s interpretation of the spec as ground truth — exactly the thing being examined. The cost is re-expressing the protocol as our own data.
• The spec as data. Every normative clause becomes a registry record with its RFC 2119 level, actor, a verbatim source quote, an optional capability gate, and either one or more mechanical checks or a written reason it cannot be judged from a trace. There is no “not yet looked at” state. A scheduled job re-verifies every quote against the live spec text so the registry cannot silently drift.
• Determinism by canonicalization. All JSON is canonicalized (RFC 8785) before any comparison, so a report is byte-identical across platforms and runs. Correctness here is a fixpoint property, tested as one — a real bug hid until a fuzzer first generated -0.0.
• Not-applicable is not pass. A requirement gated on a capability that was never negotiated is reported as not-applicable, never as a vacuous pass. Inflating a pass rate with checks for features the server never advertised is how a conformance tool loses credibility.
• Calibration against the authority. The agreement check (see the Introduction) diffs our verdicts against the official runner’s on every CI run; an unexplained divergence fails the build, and a stale recorded divergence fails it too.
• Falsifiability. Every check is killed by at least one committed injected-violation trace; a check with no killer trace is unfalsifiable. The corpus is both the regression suite and the evidence each check does what it claims.
• Verdicts are a contract. A change that makes a previously-passing trace fail is a breaking change even when no function signature moved; it is named in the changelog and the version is bumped accordingly. API-signature compatibility is additionally checked mechanically against the published crates.io baseline.

The trace format

A trace is JSON Lines: one event per line. Each event carries a capture-assigned seq (the only ordering authority — never inferred later), a direction (client-to-server / server-to-client), a transport, and a kind:

• message events hold the JSON-RPC payload verbatim;
• http events record the status and the conformance-relevant headers; and
• lifecycle events mark transport open/close.

The full schema, including the redaction rules that keep credential-bearing headers out of a trace by construction, is in 02-architecture.md and 05-security-model.md.

One worked example

The example below is embedded verbatim from the README, where a test (readme_examples.rs) pins it to the validator’s real output — so what you read here cannot drift from what the tool actually produces. It is a session that reuses a request ID, and the verdict that catches it:

{"seq":0,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-11-25","capabilities":{},"clientInfo":{"name":"my-host","version":"1.0.0"}}}}
{"seq":1,"direction":"server-to-client","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","capabilities":{"tools":{}},"serverInfo":{"name":"my-server","version":"1.0.0"}}}}
{"seq":2,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","method":"notifications/initialized"}}
{"seq":3,"direction":"client-to-server","transport":"stdio","kind":"message","payload":{"jsonrpc":"2.0","id":1,"method":"tools/list"}}

and the validator answers with the violated clause, verbatim from the spec via the registry, addressed to the offending event:

  FAIL  BASE-003 (MUST NOT)
        seq 3: request "tools/list" reuses id 1, already used by the same party at seq 0
totals: 45 pass, 1 fail, 0 warn, 88 excluded, 0 unsupported, 6 not applicable
verdict: fail

The totals line distinguishes the verdict’s components: the five not-applicable rows are capability-gated requirements this session never negotiated (the resources and prompts clauses) — reported as such, never as passes. See Architecture for why that distinction is load-bearing.

Trace corpus

Fixtures for the golden-corpus tests (crates/mcp-trace-validator/tests/golden.rs):

• good/ — sessions that must validate with verdict pass.
• violations/ — single-issue sessions, each falsifying at least one check; named after the requirement whose check they exist to kill.
• golden/ — the byte-pinned expected report for every trace. Regenerate only via cargo xtask bless and review the diff like code.

Violation naming contract

A violation trace is named area-nnn-<slug>.jsonl and must produce a Fail/Warn row with findings for exactly requirement AREA-NNN — the golden harness enforces the attribution by name (violation_traces_fail_and_match_goldens), so a defect re-routed to a different requirement cannot re-bless silently. A stem that does not begin with a requirement ID fails the suite loudly.

Provenance ledger

Every trace’s origin, in one reviewable place that survives history rewrites (the invariant test in golden.rs fails if a trace is added without a ledger row). All current traces share one provenance: hand-authored for this repository as synthetic sessions (no third-party traffic, no recorded production data), written against the 2025-11-25 spec text fetched live from modelcontextprotocol.io on 2026-06-09 (re-verified clause-by-clause against the live text on 2026-06-11) and validated against the embedded registry at the commit that introduced them. Traces produced by capture tooling (roadmap M3) will record the capturing implementation and revision here.

good/

violations/

Each file injects exactly the violation its name states; golden/ shows the full expected report, including any intrinsic secondary findings the injected defect causes (a malformed notification also fails lifecycle accounting, for example).

Conformance results

A conformance toolkit should be willing to measure real implementations — its own and others’ — against the official suite, and publish what it finds. These measurements are stewardship artifacts: every number is the official suite’s number, captured by running it, with this project’s registry used only to read each failure back to a specific spec clause. Every claim has a command that reproduces it.

This project’s reference implementations

The reference server and host are a hard CI gate, not a one-off measurement: on every run, the conformance job drives the pinned official suite against mcp-everything-server (server scenarios) and mcp-reference-host (client scenarios), and replays the captured sessions through the validator for the agreement check. A regression fails the build. The live status is whatever the latest main run reports.

rmcp tier-gap report

The published measurement of the official Rust SDK (rmcp) against the suite is the worked example of the method:

📄 rmcp tier-gap report — 2025-11-25

It records the server-scenario pass count, the requirement-level reading of each failure, a concrete close-the-gap checklist, and the exact commands to reproduce it — including a note on why the suite’s tier-check aggregate is not the figure to trust (it is GitHub-token-gated and its conformance counter carries a known upstream counting bug). The headline finding at the report’s measurement date: two failing server scenarios, neither of which is a 2025-11-25 normative-clause violation — one sits just below the spec’s RFC 2119 floor (template-argument substitution is schema prose), the other is a SEP-1330 serialization bug already filed upstream. That is the kind of distinction a requirement-level tool exists to draw; see the report for the current count and the close-the-gap steps.