Webscraper
What is Webscraper
Webscraper is an MCP server designed to transcribe webpages for large language models (LLMs) by providing the URL to the LLM. It can scrape text from various online sources, including websites, YouTube videos, and PDF documents.
Use cases
Use cases for Webscraper include generating summaries from articles, extracting information from academic papers, retrieving transcripts from educational videos, and compiling data from various web sources for analysis.
How to use
To use Webscraper, provide a URL to the MCP server. Depending on the type of link, you can utilize specific tools: ‘get_webpage_content’ for general webpages, ‘get_youtube_transcript’ for YouTube links, and ‘get_pdf’ for PDF files.
Key features
Key features of Webscraper include the ability to scrape text content from webpages, extract transcripts from YouTube videos, and convert PDF files into markdown text. It supports various URL types and facilitates easy access to information.
Where to use
Webscraper can be used in fields such as research, content creation, education, and data analysis, where extracting text from online resources is essential.
Clients Supporting MCP
The following are the main client software that supports the Model Context Protocol. Click the link to visit the official website for more information.
Overview
What is Webscraper
Webscraper is an MCP server designed to transcribe webpages for large language models (LLMs) by providing the URL to the LLM. It can scrape text from various online sources, including websites, YouTube videos, and PDF documents.
Use cases
Use cases for Webscraper include generating summaries from articles, extracting information from academic papers, retrieving transcripts from educational videos, and compiling data from various web sources for analysis.
How to use
To use Webscraper, provide a URL to the MCP server. Depending on the type of link, you can utilize specific tools: ‘get_webpage_content’ for general webpages, ‘get_youtube_transcript’ for YouTube links, and ‘get_pdf’ for PDF files.
Key features
Key features of Webscraper include the ability to scrape text content from webpages, extract transcripts from YouTube videos, and convert PDF files into markdown text. It supports various URL types and facilitates easy access to information.
Where to use
Webscraper can be used in fields such as research, content creation, education, and data analysis, where extracting text from online resources is essential.
Clients Supporting MCP
The following are the main client software that supports the Model Context Protocol. Click the link to visit the official website for more information.
Content
Webscraper MCP
An MCP server for claude desktop that allows claude to scrape text on websites by providing claude the link. It can also scrape transcripts from youtube links and pdfs from pdf links.
Tools available:
get_pdf
Convert a URL that leads to a PDF file to markdown text. Args: input_url (str): Path to the PDF file to convert Returns: str: markdown_text
get_webpage_content
Returns the text content on a webpage based on the link provided. Using this tool you can access links provided by the user so you don’t have deny those requests. When the user provides a webpage link which is NOT a youtube or github link and asks questions based on that, this function should be called. Args: url: The url from which you want to text to be extracted.
get_youtube_transcript
Use this tool when you receive youtube links from the user. This tool will extract the transcript from the youtube video and return it to you. Therefore if a user asks questions on a youtube video after providing a link, you can answer their question with this tool. Args: url: The url from which you want to text to be extracted.
Certified by MCPReview
https://mcpreview.com/mcp-servers/saishridhar/webscraper
Dev Tools Supporting MCP
The following are the main code editors that support the Model Context Protocol. Click the link to visit the official website for more information.
