What is Mcp Server Fetch

mcp-server-fetch is a Model Context Protocol server designed to fetch web content based on a provided URL. It utilizes browser automation, OCR, and various extraction methods to retrieve and process content from web pages, including those requiring JavaScript rendering.

Use cases

Use cases include fetching articles from news websites, extracting product information from e-commerce sites, gathering data for academic research, and automating the retrieval of web content for machine learning models.

How to use

To use mcp-server-fetch, you can install it via Docker. Build the Docker image with ‘docker build -t mcp-server-fetch .’ and run it using ‘docker run --rm -i mcp-server-fetch’. You can then use the ‘fetch’ tool to extract content from a URL by providing the URL as an argument.

Key features

Key features include browser automation with undetected-chromedriver, OCR capabilities using pytesseract, HTML extraction with requests/BeautifulSoup, and a sophisticated scoring system that evaluates content quality based on length, structure, and readability.

Where to use

mcp-server-fetch can be used in various fields such as web scraping, data extraction for research, content aggregation, and any application that requires reliable retrieval of web content, especially from dynamic pages.

Fetch MCP Server

A Model Context Protocol server that provides web content fetching capabilities using browser automation, OCR, and multiple extraction methods. This server enables LLMs to retrieve and process content from web pages, even those that require JavaScript rendering or use techniques that prevent simple scraping.

Available Tools

• fetch - Fetches a URL from the internet using browser automation and multi-method extraction (including OCR).
  • url (string, required): URL to fetch
  • raw (boolean, optional): Get the actual HTML content if the requested page, without simplification (default: false)

The server uses multiple methods to extract content:

1. Browser automation with undetected-chromedriver
2. OCR using pytesseract with layout detection
3. HTML extraction using requests/BeautifulSoup
4. Document parsing (PDF, DOCX, PPTX)
5. Original markdown conversion method

The server uses a sophisticated scoring system to select the best result, considering:

1. Base content score (up to 50 points)

   • Points awarded based on content length (1 point per 100 characters, max 50)
   • Penalizes extremely short content (<100 characters)

2. Structure bonus (up to 20 points)

   • Awards points for well-structured content with paragraphs
   • More paragraphs indicate better content organization

3. Quality penalties

   • Detects and penalizes error messages
   • Reduces score for content containing error indicators
   • Validates content structure and readability

The scoring system ensures the most reliable and high-quality content is selected, regardless of the extraction method used. Debug logging is available to track scoring decisions.

Prompts

• fetch
  • Fetch a URL and extract its contents as markdown using browser automation
  • Arguments:
    • url (string, required): URL to fetch

Installation

Using Docker

To install and run mcp-server-fetch using Docker, follow these steps:

1. Build the Docker image:

   docker build -t mcp-server-fetch .
   

2. Run the Docker container:

   docker run --rm -i mcp-server-fetch
   

Configuration

Configure Roo Code or Claude App

Add to your Claude settings:

{
  "mcpServers": {
    "fetch": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "mcp-server-fetch"
      ],
      "disabled": false,
      "alwaysAllow": []
    }
  }
}

Customization - User-agent

By default, depending on if the request came from the model (via a tool), or was user initiated (via a prompt), the
server will use either the user-agent

ModelContextProtocol/1.0 (Autonomous; +https://github.com/modelcontextprotocol/servers)

or

ModelContextProtocol/1.0 (User-Specified; +https://github.com/modelcontextprotocol/servers)

This can be customized by adding the argument --user-agent=YourUserAgent to the args list in the configuration.

Browser Automation and OCR

The server now includes advanced content extraction capabilities:

• Automated handling of cookie consent banners
• Full-page screenshot capture
• OCR with layout detection using pytesseract
• Multiple extraction methods with automatic selection of best results

Contributing

We encourage contributions to help expand and improve mcp-server-fetch. Whether you want to add new tools, enhance existing functionality, or improve documentation, your input is valuable.

For examples of other MCP servers and implementation patterns, see:
https://github.com/modelcontextprotocol/servers

Pull requests are welcome! Feel free to contribute new ideas, bug fixes, or enhancements to make mcp-server-fetch even more powerful and useful.

License

mcp-server-fetch is licensed under the MIT License. This means you are free to use, modify, and distribute the software, subject to the terms and conditions of the MIT License. For more details, please see the LICENSE file in the project repository.