You have built an MCP server. It handles initialize, lists tools, returns results. It works in Claude Desktop. Ship it?

Maybe. But how do you know it is actually spec-compliant? Manually testing every edge case - malformed JSON-RPC, unknown methods, missing params - is tedious. CI pipelines catch application bugs but not protocol-level issues. And the MCP spec has enough surface area that “works in one client” does not mean “works everywhere.”

We built mcp-compliance to fix this. One command, a full compliance suite, a letter grade.

One command

npx -y @yawlabs/mcp-compliance test https://your-server.com/mcp

That is it. No config files, no setup. It connects to your server, runs the full compliance suite, and prints a report.

What it tests

Tests across multiple categories, covering the same checks we described in our findings from testing MCP servers against the spec:

  1. Transport - HTTP POST acceptance, Content-Type validation, GET/DELETE handling, batch rejection, session enforcement
  2. Lifecycle - Initialize handshake, protocol version, server info, capabilities, JSON-RPC 2.0 format, ping, instructions, logging, completions
  3. Tools - List shape, call format, unknown tool handling, annotations, output schemas
  4. Resources - List, read, templates, URI/name validation (when declared)
  5. Prompts - List, schema validation, get (when declared)
  6. Error handling - Unknown method, error codes, malformed JSON-RPC, invalid JSON, invalid requests
  7. Schema validation - Tool schemas, prompt schemas, annotations, output schemas

Required tests are worth 70% of the score, optional tests 30%. You get a letter grade: A (90+), B (75+), C (60+), D (40+), F (<40).

Example output

$ npx -y @yawlabs/mcp-compliance test https://my-server.example.com/mcp

Testing https://my-server.example.com/mcp...

MCP Compliance Report
Spec: 2025-11-25  |  Tool: v0.3.0

  Transport (4/4)
  PASS  HTTP POST accepted (required)
  PASS  Responds with JSON or SSE (required)
  PASS  GET returns SSE stream or 405
  PASS  DELETE accepted or returns 405

  Lifecycle (7/8)
  PASS  Initialize handshake (required)
  PASS  Returns valid protocol version (required)
  PASS  Includes serverInfo
  PASS  Returns capabilities object (required)
  PASS  Response is valid JSON-RPC 2.0 (required)
  PASS  Responds to ping (required)
  PASS  Instructions field is valid
  FAIL  Response ID matches request ID (required)

  Tools (4/4)
  PASS  tools/list returns valid response
  PASS  All tools have name and inputSchema
  PASS  Tool annotations are valid
  PASS  Tools with outputSchema are valid

  Error Handling (4/5)
  PASS  Returns JSON-RPC error for unknown method (required)
  FAIL  Uses correct JSON-RPC error code for unknown method
  PASS  Handles malformed JSON-RPC (required)
  PASS  Handles invalid JSON body
  PASS  Returns -32600 for invalid request

  Grade: B  Score: 79%  Overall: PASS
  Tests: 19 passed / 2 failed / 21 total

A grade B is typical. Most servers nail the basics but have gaps in error handling - exactly what we found when testing servers across the ecosystem.

CI integration

Add --strict and the CLI exits with code 1 if any required test fails. Drop it into GitHub Actions:

# .github/workflows/mcp-compliance.yml
name: MCP Compliance
on: [push]
jobs:
  compliance:
    runs-on: ubuntu-latest
    steps:
      - name: MCP Compliance Check
        run: npx -y @yawlabs/mcp-compliance test ${{ env.MCP_SERVER_URL }} --strict

Want JSON output for further processing? Use --format json:

npx -y @yawlabs/mcp-compliance test https://your-server.com/mcp --format json > compliance.json

MCP server mode

mcp-compliance also runs as an MCP server itself. Add it to Claude Code and your agent can test servers from within conversations:

{
  "mcpServers": {
    "mcp-compliance": {
      "command": "npx",
      "args": ["-y", "@yawlabs/mcp-compliance", "mcp"]
    }
  }
}

This exposes three tools: mcp_compliance_test (run the full suite), mcp_compliance_badge (get badge markdown), and mcp_compliance_explain (explain what a specific test checks). Useful when you are debugging a server and want your agent to run compliance checks as part of the conversation.

Compliance badge

Get an embeddable badge for your README:

npx -y @yawlabs/mcp-compliance badge https://your-server.com/mcp

This outputs markdown you can paste into your README. Report your current grade alongside a link back to @yawlabs/mcp-compliance so readers can re-run the suite themselves.

Open source

mcp-compliance is MIT-licensed and open source.

mcp-compliance is part of a broader set of tools we are building at Yaw Labs for AI infrastructure reliability:

Jeff Yaw, Yaw Labs. Follow along at tokenlimit.news for weekly notes on AI infrastructure.