What is Mcp Validator

mcp-validator is a testing suite designed to validate MCP server implementations against the open MCP protocol specification. It assists developers in ensuring protocol compliance and interoperability.

Use cases

Use cases for mcp-validator include testing new MCP server implementations, ensuring compliance with the latest MCP protocol specifications, validating interoperability between different MCP servers, and providing a reliable environment for debugging and development.

How to use

To use mcp-validator, developers can run the provided reference implementations of the Minimal MCP Server or Minimal HTTP MCP Server. They can execute tests using the MCP Testing Framework to verify their server’s compliance with the MCP specification.

Key features

Key features of mcp-validator include reference implementations for both STDIO and HTTP transports, a comprehensive testing framework, modular architecture for various server configurations, detailed compliance reporting, and support for both synchronous and asynchronous operations.

Where to use

mcp-validator is primarily used in the development and validation of MCP servers, making it suitable for software developers and engineers working on applications that implement the Model Context Protocol.

MCP Validator

A testing suite and reference implementation for the Model Context Protocol (MCP).

Summary

The MCP Validator provides a comprehensive environment for testing and validating MCP server implementations. It includes reference implementations and a testing framework to ensure compliance with the MCP specification.

Latest Protocol Support: The validator now supports MCP protocol version 2025-06-18, which includes major enhancements like structured tool output, OAuth 2.1 authentication, elicitation support, removal of JSON-RPC batching, and enhanced security features.

🔐 OAuth 2.1 Authentication Support

The validator includes comprehensive OAuth 2.1 authentication support for the 2025-06-18 protocol version, providing a complete framework for secure MCP implementations.

Authentication Features

• OAuth 2.1 Compliance: Full RFC 6749, RFC 6750, and OAuth 2.1 draft compliance
• Bearer Token Support: Proper Bearer token extraction and validation
• WWW-Authenticate Headers: Standards-compliant 401 responses with authentication challenges
• Resource Server Capabilities: Complete OAuth 2.1 resource server implementation
• MCP-Protocol-Version Headers: Protocol version negotiation via HTTP headers
• Security Headers: CORS, Origin validation, and DNS rebinding attack prevention

OAuth 2.1 Configuration

The reference HTTP server supports OAuth 2.1 authentication through environment variables:

# Enable OAuth 2.1 authentication
export MCP_OAUTH_ENABLED=true
export MCP_OAUTH_INTROSPECTION_URL=https://auth.example.com/oauth/introspect
export MCP_OAUTH_REQUIRED_SCOPES=mcp:read,mcp:write
export MCP_OAUTH_REALM=mcp-server

# Optional: Configure resource indicators (RFC 8707)
export MCP_OAUTH_RESOURCE_INDICATORS=https://api.example.com/mcp

Authentication Testing

Test OAuth 2.1 compliance with the built-in authentication test suite:

# Test OAuth 2.1 features (framework validation)
python mcp_testing/scripts/http_compliance_test.py --debug

# Test with authentication enabled
MCP_OAUTH_ENABLED=true python ref_http_server/reference_mcp_server.py --port 8088 &
python mcp_testing/scripts/http_compliance_test.py --server-url http://localhost:8088

# Test Bearer token handling
curl -H "Authorization: Bearer your-token-here" \
     -H "MCP-Protocol-Version: 2025-06-18" \
     -H "Content-Type: application/json" \
     -d '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"client_info":{"name":"Test"},"client_capabilities":{"protocol_versions":["2025-06-18"]}}}' \
     http://localhost:8088/messages

🚀 2025-06-18 Protocol Features

Core Enhancements

• Structured Tool Output: Tools return structured responses with content, isError, and structuredContent fields
• Enhanced Tool Schema: Tools use inputSchema (renamed from parameters), title, and outputSchema fields
• JSON-RPC Batching Removal: Batch requests are properly rejected for 2025-06-18 protocol
• Elicitation Support: Framework for user interaction and data collection
• Protocol Version Headers: HTTP requests include MCP-Protocol-Version header

Security Improvements

• OAuth 2.1 Authentication: Complete authentication framework
• Enhanced Error Handling: Improved error responses with security considerations
• Origin Validation: DNS rebinding attack prevention
• Session Security: Secure session management with per-session protocol versions

STDIO Compliance Testing

The validator includes a comprehensive testing suite for STDIO-based MCP servers.

Running STDIO Tests

# Run compliance tests for the STDIO server (latest protocol)
python -m mcp_testing.scripts.compliance_report --server-command "python ref_stdio_server/stdio_server_2025_03_26.py" --protocol-version 2025-06-18

# Run with previous protocol versions
python -m mcp_testing.scripts.compliance_report --server-command "python ref_stdio_server/stdio_server_2025_03_26.py" --protocol-version 2025-03-26

STDIO Test Coverage

The STDIO compliance tests verify:

1. Protocol Initialization
2. Tools Functionality
   • Basic tools (echo, add)
   • Async tools (sleep) for 2025-03-26+ versions
   • Structured tool output (2025-06-18)
3. Error Handling
4. Protocol Version Negotiation
5. Advanced Features (2025-06-18)
   • Elicitation support
   • Enhanced validation
   • JSON-RPC batching restrictions

Testing Different STDIO Server Types

The validator supports testing any STDIO-based MCP server, whether it’s run directly from a command or installed via pip. Here’s how to test different types of servers:

Direct Command Testing

For servers that run directly from a Python file or command:

# Test a local Python file (latest protocol)
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/your/server.py" --protocol-version 2025-06-18

# Test with specific timeouts
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/server.py" --protocol-version 2025-06-18 --test-timeout 30 --tools-timeout 15

# Focus on tools testing with dynamic discovery
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/server.py" --protocol-version 2025-06-18 --test-mode tools --dynamic-only

# Test with previous protocol versions for compatibility
python -m mcp_testing.scripts.compliance_report --server-command "python path/to/server.py" --protocol-version 2025-03-26

Testing Pip-Installed Servers

For servers installed via pip (like mcp-server-fetch):

# Ensure you're in the correct virtual environment
source .venv/bin/activate

# Install the server and dependencies
pip install your-mcp-server  # Replace with actual package name

# Run compliance tests
python -m mcp_testing.scripts.compliance_report --server-command "python -m your_server_module" --protocol-version 2024-11-05

# Run tools-only tests
python -m mcp_testing.scripts.compliance_report --server-command "python -m your_server_module" --protocol-version 2024-11-05 --test-mode tools

# example brave search server
BRAVE_API_KEY=api-key python -m mcp_testing.scripts.compliance_report --server-command "npx -y @modelcontextprotocol/server-brave-search" --protocol-version 2024-11-05 

Test Configuration Options

Common options for both types:

• --test-mode tools: Focus on testing tool functionality
• --dynamic-only: Automatically discover and test available tools
• --test-timeout 30: Set timeout for regular tests (seconds)
• --tools-timeout 15: Set timeout for tool-specific tests (seconds)
• --required-tools tool1,tool2: Specify required tools to test
• --skip-tests test1,test2: Skip specific tests
• --skip-async: Skip async tool testing

Note: Tool-related tests that timeout are treated as non-critical, allowing testing to continue.

Test Reports

Each test run generates a detailed report containing:

• Server information (command, protocol version)
• Test execution timestamp
• Test duration
• Success rate
• Detailed results for each test case
• Server capabilities
• Session information

Running Tests

You can run different types of tests using module-style commands:

# Basic interaction test (latest protocol)
python -m mcp_testing.scripts.basic_interaction --server-command "python -m mcp_server_fetch" --protocol-version 2025-06-18

# Compliance tests with tools-only mode
python -m mcp_testing.scripts.compliance_report --server-command "python -m mcp_server_fetch" --protocol-version 2025-06-18 --test-mode tools

# Set custom timeouts for tools tests vs. other tests
python -m mcp_testing.scripts.compliance_report --server-command "python -m mcp_server_fetch" --protocol-version 2025-06-18 --test-timeout 30 --tools-timeout 15

# Test only specific functionality
python -m mcp_testing.scripts.compliance_report --server-command "/path/to/server" --protocol-version 2025-06-18 --test-mode tools

# Skip async tests if experiencing hanging issues
python -m mcp_testing.scripts.compliance_report --server-command "/path/to/server" --protocol-version 2025-06-18 --skip-async

# HTTP debug output
python -m mcp_testing.scripts.http_test --server-url http://localhost:8000/mcp --debug

# Test backward compatibility with older protocols
python -m mcp_testing.scripts.compliance_report --server-command "python -m mcp_server_fetch" --protocol-version 2024-11-05

Note: Tool-related tests that timeout are treated as non-critical, allowing testing to continue.

HTTP Compliance Testing

The validator includes a comprehensive compliance testing suite for HTTP-based MCP servers with full 2025-06-18 protocol support.

Running HTTP Tests

# Start the reference HTTP server (runs on port 8088)
python ref_http_server/reference_mcp_server.py

# Run compliance tests and generate a detailed report
python -m mcp_testing.scripts.http_compliance_test --output-dir reports

# Test with OAuth 2.1 authentication enabled
MCP_OAUTH_ENABLED=true python ref_http_server/reference_mcp_server.py --port 8088 &
python -m mcp_testing.scripts.http_compliance_test --server-url http://localhost:8088 --debug

HTTP Test Coverage

The HTTP compliance test suite verifies:

1. Protocol Initialization

   • 2025-06-18 protocol negotiation
   • Session management with protocol-specific features
   • MCP-Protocol-Version header handling

2. Tools Functionality

   • Structured tool output (2025-06-18)
   • Legacy tool responses (older protocols)
   • Enhanced tool schema validation

3. Error Handling

   • Enhanced error responses (2025-06-18)
   • OAuth 2.1 authentication errors
   • Proper HTTP status codes

4. Batch Request Processing

   • Batch request restrictions (2025-06-18)
   • Legacy batch support (older protocols)

5. Session Management

   • Per-session protocol versions
   • Session security and validation

6. Protocol Negotiation

   • Multi-version support (2024-11-05, 2025-03-26, 2025-06-18)
   • Intelligent version selection

7. Authentication & Security

   • OAuth 2.1 Bearer token validation
   • WWW-Authenticate header compliance
   • CORS and origin validation

8. Ping Utility

   • Connection testing and validation

Production Deployment

For production deployments with 2025-06-18 protocol:

# Enable HTTPS (required for OAuth 2.1 in production)
export MCP_TLS_CERT_FILE=/path/to/cert.pem
export MCP_TLS_KEY_FILE=/path/to/key.pem

# Configure OAuth 2.1 authentication
export MCP_OAUTH_ENABLED=true
export MCP_OAUTH_INTROSPECTION_URL=https://auth.example.com/oauth/introspect
export MCP_OAUTH_REQUIRED_SCOPES=mcp:read,mcp:write

# Configure CORS for web clients
export MCP_CORS_ORIGINS=https://myapp.example.com,https://anotherapp.example.com

# Enable rate limiting and security features
export MCP_RATE_LIMIT_ENABLED=true
export MCP_MAX_REQUESTS_PER_MINUTE=100

# Start the server
python ref_http_server/reference_mcp_server.py --port 443 --production

Testing Scripts Overview

The following scripts are available in mcp_testing/scripts/:

Active and Maintained

• http_compliance_test.py: Primary script for HTTP server testing with 2025-06-18 support (7/7 tests passing)
• compliance_report.py: Primary script for STDIO server testing with 2025-06-18 support (enhanced test coverage)

Supporting Scripts mixed working/in progress

• basic_interaction.py: Simple tool for testing basic server functionality
• http_test.py: Lower-level HTTP testing utilities with OAuth 2.1 support
• http_compliance.py: Core HTTP compliance testing logic
• http_compliance_report.py: Report generation for HTTP tests
• run_stdio_tests.py: Lower-level STDIO testing utilities
• session_test.py: Session management testing utilities

📋 Protocol Version Support

🧪 Test Results Summary

Current test coverage for 2025-06-18 protocol:

✅ HTTP Compliance Test: 7/7 tests passed
✅ OAuth 2.1 Framework: 6/6 tests passed  
✅ Protocol Features: 7/7 tests passed
✅ Multi-Protocol Support: 3/3 versions supported
✅ Backward Compatibility: 100% maintained
✅ Security Features: Authentication, CORS, Origin validation

📚 Additional Resources

• JSON Schema: Complete schemas available in schema/ directory
• Specification Documents: Full specification files in specification/ directory
• Reference Implementations: HTTP and STDIO servers in ref_http_server/ and ref_stdio_server/
• Test Reports: Generated reports available in reports/ directory

License

SPDX-License-Identifier: AGPL-3.0-or-later
Copyright © 2025 Scott Wilcox