You have built an MCP server. It handles initialize, lists tools, returns results. It works in Claude Desktop. Ship it?

Maybe. But how do you know it is actually spec-compliant? Manually testing every edge case - malformed JSON-RPC, unknown methods, missing params - is tedious. CI pipelines catch application bugs but not protocol-level issues. And the MCP spec has enough surface area that “works in one client” does not mean “works everywhere.”

We built mcp-compliance to fix this. One command, a full compliance suite, a letter grade.

One command

npx -y @yawlabs/mcp-compliance test https://your-server.com/mcp

That is it. No config files, no setup. It connects to your server, runs the full compliance suite, and prints a report.

What it tests

88 tests across 8 categories - the same suite behind our findings from testing MCP servers against the spec:

  1. Transport (16) - HTTP POST acceptance, Content-Type validation (JSON/SSE), GET/DELETE handling, session ID enforcement, streaming
  2. Lifecycle (21) - Initialize handshake, protocol version, server info, capabilities, JSON-RPC 2.0 format, ping, cancellation, progress
  3. Tools (4) - List shape, call format, unknown tool handling, input schemas
  4. Resources (5) - List, read, templates, URI validation, subscribe (when declared)
  5. Prompts (3) - List, get, pagination (when declared)
  6. Errors (10) - Unknown method, error codes, malformed JSON-RPC, invalid JSON, missing params
  7. Schema (6) - Tool names, inputSchema shape, resource URIs, prompt argument names
  8. Security (23) - Auth & transport, input validation, tool-description injection, information disclosure, SSRF

Capability-gated tests (tools, resources, prompts, subscribe, logging, completions) only run if the server declares the capability - no false failures for features the server never claimed. Required tests are worth 70% of the score, optional tests 30%. You get a letter grade: A (90+), B (75+), C (60+), D (40+), F (<40). The full rubric - stable rule IDs, per-test weights, spec references - is published under CC BY 4.0; see the open methodology behind the 88-test grading suite. This post is the how-to: running the suite from your terminal and wiring it into CI.

Example output

$ npx -y @yawlabs/mcp-compliance test https://my-server.example.com/mcp

Testing https://my-server.example.com/mcp...

MCP Compliance Report
Spec: 2025-11-25

  Transport      16/16  PASS
  Lifecycle      19/21  2 optional failures
  Tools           4/4   PASS
  Resources       5/5   PASS
  Prompts         3/3   PASS
  Errors          9/10  1 required failure
  Schema          6/6   PASS
  Security       22/23  1 optional failure

  Required failures:
  FAIL  errors-04  JSON-RPC error code for unknown method must be -32601
                   Received: -32600 (invalid request)

  Optional failures:
  WARN  lifecycle-15  Server did not echo client's protocolVersion
  WARN  lifecycle-17  initialized notification not acknowledged
  WARN  security-22   Tool description contains suspicious instruction phrasing

  Grade: B  Score: 86%  Overall: PASS
  Tests: 84 passed / 4 failed / 88 total

A grade B is typical. Most servers nail the basics but have gaps in error handling and the security long tail - exactly what we found when testing servers across the ecosystem.

CI integration

Add --strict and the CLI exits with code 1 if any required test fails. Drop it into GitHub Actions:

# .github/workflows/mcp-compliance.yml
name: MCP Compliance
on: [push]
jobs:
  compliance:
    runs-on: ubuntu-latest
    steps:
      - name: MCP Compliance Check
        run: npx -y @yawlabs/mcp-compliance test ${{ env.MCP_SERVER_URL }} --strict

Want JSON output for further processing? Use --format json:

npx -y @yawlabs/mcp-compliance test https://your-server.com/mcp --format json > compliance.json

MCP server mode

mcp-compliance also runs as an MCP server itself. Add it to Claude Code and your agent can test servers from within conversations:

{
  "mcpServers": {
    "mcp-compliance": {
      "command": "npx",
      "args": ["-y", "@yawlabs/mcp-compliance", "mcp"]
    }
  }
}

This exposes three tools: mcp_compliance_test (run the full suite), mcp_compliance_badge (get badge markdown), and mcp_compliance_explain (explain what a specific test checks). Useful when you are debugging a server and want your agent to run compliance checks as part of the conversation.

Compliance badge

Get an embeddable badge for your README:

npx -y @yawlabs/mcp-compliance badge https://your-server.com/mcp

This outputs markdown you can paste into your README. Report your current grade alongside a link back to @yawlabs/mcp-compliance so readers can re-run the suite themselves.

Open source

mcp-compliance is MIT-licensed and open source.

mcp-compliance is part of a broader set of tools we are building at Yaw Labs for AI infrastructure reliability:

Jeff Yaw, Yaw Labs. Follow along at tokenlimit.news for weekly notes on AI infrastructure.