What is Mcp Llms Txt Explorer

mcp-llms-txt-explorer is a Model Context Protocol server designed to explore websites that implement the llms.txt standard. It facilitates the discovery and analysis of these websites.

Use cases

Use cases include validating llms.txt files for compliance, discovering websites that implement the llms.txt standard, and analyzing structured data from these websites.

How to use

To use mcp-llms-txt-explorer, you can install it via Smithery or manually by cloning the repository, installing dependencies, and building the server. Configuration is required for integration with Claude Desktop.

Key features

Key features include checking for llms.txt files on websites, parsing and validating their contents, and accessing structured data about compliant websites. Tools like ‘check_website’ and ‘list_websites’ enhance functionality.

Where to use

mcp-llms-txt-explorer can be used in web development, data analysis, and compliance checking for websites that utilize the llms.txt standard.

MCP LLMS.txt Explorer

A Model Context Protocol server for exploring websites with llms.txt files. This server helps you discover and analyze websites that implement the llms.txt standard.

Features

Resources

• Check websites for llms.txt and llms-full.txt files
• Parse and validate llms.txt file contents
• Access structured data about compliant websites

Tools

• check_website - Check if a website has llms.txt files
  • Takes domain URL as input
  • Returns file locations and validation status
• list_websites - List known websites with llms.txt files
  • Returns structured data about compliant websites
  • Supports filtering by file type (llms.txt/llms-full.txt)

Development

Install dependencies:

pnpm install

Build the server:

pnpm run build

For development with auto-rebuild:

pnpm run watch

Installation

Installing via Smithery

To install mcp-llms-txt-explorer for Claude Desktop automatically via Smithery:

npx -y @smithery/cli install @thedaviddias/mcp-llms-txt-explorer --client claude

Installing Manually

To use this server:

# Clone the repository
git clone https://github.com/thedaviddias/mcp-llms-txt-explorer.git
cd mcp-llms-txt-explorer

# Install dependencies
pnpm install

# Build the server
pnpm run build

Configuration with Claude Desktop

To use with Claude Desktop, add the server config:

On MacOS: ~/Library/Application Support/Claude/claude_desktop_config.json
On Windows: %APPDATA%/Claude/claude_desktop_config.json

For npx usage, you can use:

{
  "mcpServers": {
    "llms-txt-explorer": {
      "command": "npx",
      "args": [
        "-y",
        "@thedaviddias/mcp-llms-txt-explorer"
      ]
    }
  }
}

Debugging

Since MCP servers communicate over stdio, debugging can be challenging. We recommend using the MCP Inspector, which is available as a package script:

pnpm run inspector

The Inspector will provide a URL to access debugging tools in your browser.

License

This project is licensed under the MIT License—see the LICENSE file for details.