What is Rust Local Rag

rust-local-rag is a high-performance local Retrieval-Augmented Generation (RAG) server built in Rust. It integrates with Claude Desktop via the Model Context Protocol (MCP) to enable private searching and analysis of PDF documents without external API calls.

Use cases

Use cases include searching for specific information within academic papers, analyzing legal contracts, and extracting data from business reports without the risk of data exposure to external services.

How to use

To use rust-local-rag, install the server and configure it to work with Claude Desktop. Users can then upload PDF documents for local processing and perform semantic searches directly within their conversations in Claude.

Key features

Key features include local PDF processing using poppler for text extraction, generation of embeddings with local Ollama models, semantic search capabilities, seamless integration with Claude Desktop via MCP, and maintaining user privacy by keeping all data processing local.

Where to use

rust-local-rag can be used in various fields such as academic research, legal document analysis, and any scenario where secure and private document processing is essential.

Rust Local RAG

A high-performance, local RAG (Retrieval-Augmented Generation) system built in Rust that integrates with Claude Desktop via the Model Context Protocol (MCP). Search and analyze your PDF documents directly within Claude conversations without sending data to external services.

🎯 Purpose

This project demonstrates how to build a production-ready MCP server using Rust that:

• Processes PDF documents locally using poppler for text extraction
• Generates embeddings using local Ollama models (no external API calls)
• Provides semantic search through document collections
• Integrates seamlessly with Claude Desktop via MCP protocol
• Maintains privacy by keeping all data processing local

🏗️ What is MCP?

The Model Context Protocol (MCP) is a standard that allows AI assistants like Claude to interact with external tools and data sources. Instead of Claude being limited to its training data, MCP enables it to:

• Call external tools and functions
• Access real-time data sources
• Integrate with local applications
• Maintain context across interactions

🦀 How This Project Uses Rust MCP SDK

This implementation leverages the rmcp crate - the official Rust SDK for MCP - to create a server that exposes RAG capabilities to Claude Desktop.

MCP Architecture in This Project

┌─────────────────┐    MCP Protocol     ┌──────────────────┐
│                 │    (stdin/stdout)   │                  │
│  Claude Desktop │ ◄─────────────────► │   Rust RAG       │
│                 │                     │   MCP Server     │
└─────────────────┘                     └──────────────────┘
                                                │
                                                ▼
                                        ┌──────────────────┐
                                        │  Local RAG Stack │
                                        │                  │
                                        │  • PDF Parser    │
                                        │  • Ollama        │
                                        │  • Vector Store  │
                                        │  • Search Engine │
                                        └──────────────────┘

Key MCP Components Used

1. Server Handler Implementation

#[tool(tool_box)]
impl ServerHandler for RagMcpServer {
    fn get_info(&self) -> ServerInfo {
        // Provides server metadata to Claude
    }
}

2. Tool Definitions

Uses rmcp macros to expose RAG functionality as MCP tools:

#[tool(description = "Search through uploaded documents using semantic similarity")]
async fn search_documents(&self, query: String, top_k: Option<usize>) -> Result<CallToolResult, McpError>

#[tool(description = "List all uploaded documents")]  
async fn list_documents(&self) -> Result<CallToolResult, McpError>

#[tool(description = "Get RAG system statistics")]
async fn get_stats(&self) -> Result<CallToolResult, McpError>

3. Transport Layer

// Uses stdin/stdout transport for Claude Desktop integration
let service = server.serve(stdio()).await?;

✨ Features

🔍 Semantic Document Search

• Vector-based similarity search using Ollama embeddings
• Configurable result count (top-k)
• Relevance scoring for search results

📁 Document Management

• Automatic PDF text extraction via poppler
• Document chunking for optimal embedding generation
• Real-time document list and statistics

🔒 Privacy-First Design

• All processing happens locally
• No external API calls for document content
• Embeddings stored locally for fast retrieval

⚡ High Performance

• Rust’s memory safety and performance
• Async/await for non-blocking operations
• Efficient vector storage and retrieval

🚀 Quick Start

1. Prerequisites

# Install Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

# Install Ollama
brew install ollama

# Install Poppler (for PDF parsing)
brew install poppler

# Start Ollama and install embedding model
make setup-ollama

2. Build and Install

git clone <this-repository>
cd rust-local-rag
make install

3. Configure Claude Desktop

Add to ~/Library/Application Support/Claude/claude_desktop_config.json:

{
  "mcpServers": {
    "rust-local-rag": {
      "command": "/Users/yourusername/.cargo/bin/rust-local-rag",
      "env": {
        "DATA_DIR": "/Users/yourusername/Documents/data",
        "DOCUMENTS_DIR": "/Users/yourusername/Documents/rag",
        "LOG_DIR": "/tmp/rust-local-rag",
        "LOG_LEVEL": "info",
        "LOG_MAX_MB": "10"
      }
    }
  }
}

4. Add Documents and Use

# Add PDFs to documents directory
cp your-files.pdf ~/Documents/rag/

# Restart Claude Desktop
# Now ask Claude: "Search my documents for information about X"

🏛️ Architecture

Technology Stack

• 🦀 Rust: Core application language for performance and safety
• 📡 rmcp: Official Rust MCP SDK for Claude integration
• 🤖 Ollama: Local embedding generation (nomic-embed-text)
• 📄 Poppler: PDF text extraction
• 🗃️ Custom Vector Store: In-memory vector database for fast search

Data Flow

1. Document Ingestion: PDFs → Text extraction → Chunking
2. Embedding Generation: Text chunks → Ollama → Vector embeddings
3. Indexing: Embeddings → Local vector store
4. Search: Query → Embedding → Similarity search → Results
5. MCP Integration: Results → Claude Desktop via MCP protocol

🛠️ MCP Integration Details

Why MCP Over HTTP API?

MCP Tools Exposed

1. search_documents

   • Purpose: Semantic search across document collection
   • Input: Query string, optional result count
   • Output: Ranked search results with similarity scores

2. list_documents

   • Purpose: Document inventory management
   • Input: None
   • Output: List of all indexed documents

3. get_stats

   • Purpose: System monitoring and debugging
   • Input: None
   • Output: Embedding counts, memory usage, performance metrics

📚 Documentation

• Setup Guide: Complete installation and configuration
• Usage Guide: Claude Desktop integration and usage examples

🤝 Contributing

Contributions are welcome! This project demonstrates practical MCP server implementation patterns that can be adapted for other use cases.

Development

# Run in development mode
make run

# Check formatting
cargo fmt --check

# Run linter
cargo clippy

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• Model Context Protocol for the specification
• rmcp for the excellent Rust MCP SDK
• Ollama for local embedding generation
• Claude Desktop for MCP integration support

Built with ❤️ in Rust | Powered by MCP | Privacy-focused RAG