Mcp Webscrapper
What is Mcp Webscrapper
mcp_webscrapper is a Model Context Protocol (MCP) server designed to scrape websites using the Decodo scraping API. It allows users to extract text content from specific HTML elements on web pages.
Use cases
Use cases include extracting product information from e-commerce sites, gathering news articles from online publications, and collecting data for academic research from various web sources.
How to use
To use mcp_webscrapper, install it by following the setup instructions in the README. After installation, connect it to Claude Desktop and use commands to scrape websites by specifying the desired HTML elements.
Key features
Key features include website scraping of any publicly accessible site, targeted content extraction from specific HTML div elements by ID, and seamless integration with AI assistants through the MCP framework.
Where to use
mcp_webscrapper can be used in various fields such as data analysis, content aggregation, research, and any area where web data extraction is needed.
Clients Supporting MCP
The following are the main client software that supports the Model Context Protocol. Click the link to visit the official website for more information.
Overview
What is Mcp Webscrapper
mcp_webscrapper is a Model Context Protocol (MCP) server designed to scrape websites using the Decodo scraping API. It allows users to extract text content from specific HTML elements on web pages.
Use cases
Use cases include extracting product information from e-commerce sites, gathering news articles from online publications, and collecting data for academic research from various web sources.
How to use
To use mcp_webscrapper, install it by following the setup instructions in the README. After installation, connect it to Claude Desktop and use commands to scrape websites by specifying the desired HTML elements.
Key features
Key features include website scraping of any publicly accessible site, targeted content extraction from specific HTML div elements by ID, and seamless integration with AI assistants through the MCP framework.
Where to use
mcp_webscrapper can be used in various fields such as data analysis, content aggregation, research, and any area where web data extraction is needed.
Clients Supporting MCP
The following are the main client software that supports the Model Context Protocol. Click the link to visit the official website for more information.
Content
🕷️ Decodo Website Scraper
A Model Context Protocol (MCP) server that provides website scraping capabilities using the Decodo scraping API. This tool allows you to extract text content from specific HTML elements on web pages.
✨ Features
- 🌐 Website Scraping: Scrapes any publicly accessible website using the Decodo API
- 🎯 Targeted Content Extraction: Extracts text from specific HTML div elements by ID
- 🔌 MCP Integration: Built as an MCP server for seamless integration with AI assistants
📦 Installation
- Download Claude Desktop from this website and log in.
https://claude.ai/download
-
Create a Decodo account and get your API key:
Claim your free trial on Decodo: https://visit.decodo.com/aOL4yR -
Clone this repository:
git clone https://github.com/abdullahtarek/mcp_webscrapper
cd decodo-webscrapper
-
Replace the API key at the top main.py with your Decodo API key.
-
Install uv:
pip install uv
- Run MCP server and connect it to Claude Desktop:
uv run mcp install -e. main.py
🚀 Usage
Once installed and connected to Claude Desktop, you can use the scraper by asking Claude to scrape websites. The tool will extract content from specified HTML elements using the Decodo API.
⚙️ Requirements
- Python 3.8+
- uv package manager
- Decodo API key
- Claude Desktop
🤝 Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
📄 License
This project is open source and available under the MIT License.
Dev Tools Supporting MCP
The following are the main code editors that support the Model Context Protocol. Click the link to visit the official website for more information.
