What is Rquest

mcp-rquest is a Model Context Protocol (MCP) server designed to enhance HTTP request functionalities for language models, particularly Claude and others. It provides capabilities such as realistic browser emulation, allowing models to interact with websites more dynamically and effectively bypassing anti-bot measures. Additionally, it includes features for converting PDF and HTML documents into Markdown format, optimizing the content for processing by language models.

Use cases

mcp-rquest is ideal for applications needing to scrape or interact with web content while avoiding detection as a bot. It’s particularly useful in scenarios involving data extraction from websites, automated testing of web applications, or obtaining information in a structured format suitable for further processing by language models, like transforming documents into Markdown for easier handling.

How to use

Users can install mcp-rquest using pip or by running it directly with uv. After installation, it can be launched as a script, and configuration details can be added to enable it for use with Claude or other applicable systems. The server can be customized with authentication methods and request parameters, enhancing its flexibility in handling various tasks.

Key features

Key features of mcp-rquest include support for all standard HTTP methods, advanced browser fingerprinting capabilities, automatic handling of large responses, conversion of HTML/PDF to Markdown, diverse authentication support, and SSL security using BoringSSL. The system also includes tools for specific HTTP requests and response handling, accommodating a wide range of user needs.

Where to use

mcp-rquest can be utilized in web scraping applications, automated web interactions, or contexts where web content needs to be accessed programmatically without triggering bot detection. Its capabilities make it suitable for research, data extraction, and scenarios where structured data from web pages or documents is required for further analysis or processing.

mcp-rquest

A Model Context Protocol (MCP) server that provides advanced HTTP request capabilities for Claude and other LLMs. Built on rquest, this server enables realistic browser emulation with accurate TLS/JA3/JA4 fingerprints, allowing models to interact with websites more naturally and bypass common anti-bot measures. It also supports converting PDF and HTML documents to Markdown for easier processing by LLMs.

Features

• Complete HTTP Methods: Support for GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS, and TRACE
• Browser Fingerprinting: Accurate TLS, JA3/JA4, and HTTP/2 browser fingerprints
• Content Handling:
  • Automatic handling of large responses with token counting
  • HTML to Markdown conversion for better LLM processing
  • PDF to Markdown conversion using the Marker library
  • Secure storage of responses in system temporary directories
• Authentication Support: Basic, Bearer, and custom authentication methods
• Request Customization:
  • Headers, cookies, redirects
  • Form data, JSON payloads, multipart/form-data
  • Query parameters
• SSL Security: Uses BoringSSL for secure connections with realistic browser fingerprints

Available Tools

• HTTP Request Tools:

  • http_get - Perform GET requests with optional parameters
  • http_post - Submit data via POST requests
  • http_put - Update resources with PUT requests
  • http_delete - Remove resources with DELETE requests
  • http_patch - Partially update resources
  • http_head - Retrieve only headers from a resource
  • http_options - Retrieve options for a resource
  • http_trace - Diagnostic request tracing

• Response Handling Tools:

  • get_stored_response - Retrieve stored large responses, optionally by line range
  • get_stored_response_with_markdown - Convert HTML or PDF responses to Markdown format for better LLM processing
  • get_model_state - Get the current state of the PDF models loading process
  • restart_model_loading - Restart the PDF models loading process if it failed or got stuck

PDF Support

mcp-rquest now supports PDF to Markdown conversion, allowing you to download PDF files and convert them to Markdown format that’s easy for LLMs to process:

1. Automatic PDF Detection: PDF files are automatically detected based on content type
2. Seamless Conversion: The same get_stored_response_with_markdown tool works for both HTML and PDF files
3. High-Quality Conversion: Uses the Marker library for accurate PDF to Markdown transformation
4. Optimized Performance: Models are pre-downloaded during package installation to avoid delays during request processing

Installation

Using uv (recommended)

When using uv no specific installation is needed. We will
use uvx to directly run mcp-rquest.

Using pip

Alternatively you can install mcp-rquest via pip:

pip install mcp-rquest

After installation, you can run it as a script using:

python -m mcp_rquest

Configuration

Configure for Claude.app

Add to your Claude settings:

Using uvx:

{
  "mcpServers": {
    "http-rquest": {
      "command": "uvx",
      "args": [
        "mcp-rquest"
      ]
    }
  }
}

Using pip:

{
  "mcpServers": {
    "http-rquest": {
      "command": "python",
      "args": [
        "-m",
        "mcp_rquest"
      ]
    }
  }
}

Using pipx:

{
  "mcpServers": {
    "http-rquest": {
      "command": "pipx",
      "args": [
        "run",
        "mcp-rquest"
      ]
    }
  }
}

Browser Emulation

mcp-rquest leverages rquest’s powerful browser emulation capabilities to provide realistic browser fingerprints, which helps bypass bot detection and access content normally available only to standard browsers. Supported browser fingerprints include:

• Chrome (multiple versions)
• Firefox
• Safari (including iOS and iPad versions)
• Edge
• OkHttp

This ensures that requests sent through mcp-rquest appear as legitimate browser traffic rather than bot requests.

Development

Setting up a Development Environment

1. Clone the repository
2. Create a virtual environment using uv:

   uv venv
   

3. Activate the virtual environment:

   # Unix/macOS
   source .venv/bin/activate
   # Windows
   .venv\Scripts\activate
   

4. Install development dependencies:

   uv pip install -e ".[dev]"
   

Acknowledgements

• This project is built on top of rquest, which provides the advanced HTTP client with browser fingerprinting capabilities.
• rquest is based on a fork of reqwest.