What is Seo Mcp Tools

seo-mcp-tools is a Model Context Protocol (MCP) server that provides specialized SEO tools for analyzing robots.txt files and extracting sitemap information from them.

Use cases

Use cases include extracting sitemap URLs for website audits, improving SEO strategies by analyzing robots.txt files, and integrating with other tools for automated SEO analysis.

How to use

To use seo-mcp-tools, clone the repository, install the required dependencies, and run the MCP server using the command ‘python mcp_server.py’. You can then call the tools to extract sitemap URLs from robots.txt files.

Key features

Key features include ‘Get First Sitemap URL’ which extracts the first sitemap URL from a robots.txt file, and ‘Get All Sitemap URLs’ which extracts all sitemap URLs from a robots.txt file.

Where to use

seo-mcp-tools can be used in web development, SEO analysis, and digital marketing to optimize website visibility and indexing by analyzing sitemap information.

SEO MCP Tools

A Model Context Protocol (MCP) server that provides SEO tools for analyzing robots.txt files. This server implements a set of specialized tools that help extract and analyze sitemap information from robots.txt files across the web.

Features

The SEO MCP Tools server provides two primary tools:

1. Get First Sitemap URL: Extract the first sitemap URL from a robots.txt file.
2. Get All Sitemap URLs: Extract all sitemap URLs from a robots.txt file.

Requirements

• Python 3.8+
• FastMCP
• advertools
• requests
• pandas

Installation

1. Clone this repository:

   git clone https://github.com/yourusername/seo-mcp-tools.git
   cd seo-mcp-tools
   

2. Install the required dependencies:

   pip install fastmcp advertools requests pandas
   

Usage

Running the Server

Start the MCP server with:

python mcp_server.py

Alternatively, use the FastMCP development tools for an interactive testing environment:

fastmcp dev mcp_server.py

MCP Tool Calls

Get First Sitemap URL

from mcp_client import MCP_Client  # Assuming you're using an MCP client

client = MCP_Client("http://localhost:8000")

# Extract the first sitemap URL from a robots.txt file
first_sitemap = client.call("get_sitemap_url_from_robots_txt", {
    "robots_txt_url": "https://example.com/robots.txt"
})
print(f"First sitemap URL: {first_sitemap}")

Get All Sitemap URLs

# Extract all sitemap URLs from a robots.txt file
all_sitemaps = client.call("get_all_sitemap_urls_from_robots_txt", {
    "robots_txt_url": "https://example.com/robots.txt"
})
print(f"All sitemap URLs: {all_sitemaps}")

Claude Desktop Integration

1. Create a minimal MCP configuration file:

   {
     "version": "v1",
     "endpoints": [
       {
         "name": "SEO Tools",
         "transport": {
           "type": "stdio",
           "command": {
             "path": "/path/to/python",
             "args": [
               "/path/to/mcp_server.py"
             ]
           }
         },
         "input_format": "json",
         "output_format": "json"
       }
     ]
   }
   

2. Install in Claude Desktop:

   fastmcp install mcp_server.py
   

For development/testing mode, use: fastmcp dev mcp_server.py

Testing

Run the tests to verify server functionality:

pytests

The test suite includes validation against real websites’ robots.txt files to ensure the tools work correctly with various robots.txt structures.

How It Works

The server uses direct HTTP requests to fetch robots.txt files from URLs, then applies regular expression patterns to extract relevant directives. This approach ensures reliable parsing of robots.txt files from various websites, regardless of formatting differences.

License

MIT