What is Puppeteer Mcp Service

The Puppeteer MCP Service is a browser automation server that allows Large Language Models (LLMs) and programmatic clients to interact with web pages using a real browser environment. It is built with Python and utilizes the Playwright library for various automation tasks.

Use cases

Use cases include automated testing of web applications, scraping data from websites, generating screenshots for visual testing, and executing JavaScript for dynamic content manipulation.

How to use

To use the Puppeteer MCP Service, clients communicate with the server via the JSON-RPC 2.0 protocol over HTTP. Users can send requests to navigate, interact with elements, extract content, take screenshots, and execute JavaScript code.

Key features

Key features include comprehensive browser automation with Playwright, a standardized JSON-RPC 2.0 interface, page navigation and interaction capabilities, rich content extraction, screenshot capabilities, JavaScript execution, browser context management, and configurable behavior.

Where to use

The Puppeteer MCP Service can be used in various fields such as web scraping, automated testing, data collection, and any application requiring interaction with dynamic web content.

Puppeteer MCP (Model Context Protocol) Service

1. Introduction

The Puppeteer MCP Service is a powerful browser automation server designed to enable Large Language Models (LLMs) and other programmatic clients to interact with web pages in a real browser environment. Built with Python using the Playwright library, it provides a robust set of tools for navigation, content extraction, element interaction, screenshotting, and JavaScript execution. Communication with the service is handled via the JSON-RPC 2.0 protocol over HTTP, making it easy to integrate with various client applications.

This service aims to bridge the gap between language models and the dynamic web, allowing for sophisticated web scraping, automated testing, data collection, and other tasks requiring browser interaction. It is designed to be configurable and extensible, providing a solid foundation for complex browser automation workflows.

2. Key Features

• Comprehensive Browser Automation: Leverages Playwright to control modern web browsers (Chromium by default, configurable for Firefox and WebKit).
• JSON-RPC 2.0 Interface: Standardized and straightforward API interaction using JSON-RPC over HTTP.
• Page Navigation and Interaction: Navigate to URLs, click elements, fill forms, and retrieve page content.
• Rich Content Extraction: Get page titles, current URLs, full HTML content, and text from specific elements.
• Screenshot Capabilities: Capture full-page or element-specific screenshots in various formats (PNG, JPEG) with quality and path customization.
• JavaScript Execution: Execute arbitrary JavaScript code within the context of the current page and retrieve results.
• Browser Context Management: Create, switch between, and close isolated browser contexts for managing sessions, cookies, and storage independently. This is a key feature for handling multiple independent tasks or user sessions.
• Configurable Behavior: Customize browser launch options, default timeouts, user agents, and viewport sizes through a config.json file.
• Error Handling: Clear and structured JSON-RPC error responses for easier debugging.
• Health Check Endpoint: A simple /health endpoint to monitor service and browser status.
• Placeholder for Console Log Monitoring: While the API endpoint puppeteer.get_console_logs exists, it currently serves as a placeholder. Full console log retrieval would typically involve event listeners within the browser management layer to collect messages as they occur.

3. Getting Started

3.1. Prerequisites

• Python 3.8+ (Python 3.11 recommended, as used in the development environment).
• pip for installing Python packages.
• Playwright browser drivers. The service will attempt to install the Chromium driver on first run if not found, but manual installation might be required in some environments:

  playwright install chromium
  

3.2. Installation

1. Ensure all project files are in a directory (e.g., /home/ubuntu/puppeteer_mcp_service).
2. Install the required Python dependencies:

   pip3 install -r /home/ubuntu/puppeteer_mcp_service/requirements.txt
   

   The requirements.txt file should contain:

   fastapi
   uvicorn[standard]
   playwright
   # Add other dependencies if any were introduced
   

3.3. Configuration

The service is configured using a config.json file located in the project root directory (/home/ubuntu/puppeteer_mcp_service/config.json).

Default config.json:

{
  "browser_type": "chromium",
  "headless": true,
  "slow_mo": 0,
  "default_timeout": 30000,
  "user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/99.0.4844.51 Safari/537.36",
  "viewport": {
    "width": 1280,
    "height": 720
  },
  "service_host": "0.0.0.0",
  "service_port": 8080,
  "log_level": "INFO",
  "playwright_browser_args": []
}

Configuration Options:

• browser_type (string): The browser to use. Supported values: “chromium”, “firefox”, “webkit”. Default: “chromium”.
• headless (boolean): Whether to run the browser in headless mode. Default: true.
• slow_mo (integer): Slows down Playwright operations by the specified amount in milliseconds. Useful for debugging. Default: 0.
• default_timeout (integer): Default timeout in milliseconds for Playwright operations (e.g., navigation, waiting for selectors). Default: 30000 (30 seconds).
• user_agent (string): Default user agent string for new browser contexts. Default: A common Chrome user agent.
• viewport (object): Default viewport size for new pages.
  • width (integer): Viewport width in pixels. Default: 1280.
  • height (integer): Viewport height in pixels. Default: 720.
• service_host (string): Host address for the MCP service to listen on. Default: “0.0.0.0”.
• service_port (integer): Port for the MCP service to listen on. Default: 8080.
• log_level (string): Logging level for the service. Supported values: “DEBUG”, “INFO”, “WARNING”, “ERROR”, “CRITICAL”. Default: “INFO”.
• playwright_browser_args (array of strings): Additional arguments to pass to the browser instance on launch (e.g., ["--disable-gpu"]). Default: [].

3.4. Running the Service

Navigate to the project root directory (/home/ubuntu/puppeteer_mcp_service) and run the service using Uvicorn:

export PYTHONPATH=/home/ubuntu/puppeteer_mcp_service:$PYTHONPATH
/home/ubuntu/.local/bin/uvicorn main:app --host 0.0.0.0 --port 8080

Or, if Uvicorn is in your system path:

export PYTHONPATH=/home/ubuntu/puppeteer_mcp_service:$PYTHONPATH
uvicorn main:app --host 0.0.0.0 --port 8080

The service will be available at http://0.0.0.0:8080. The JSON-RPC endpoint is http://0.0.0.0:8080/jsonrpc.

4. API Endpoints (JSON-RPC Methods)

All API methods are invoked by sending a POST request to the /jsonrpc endpoint (e.g. http://localhost:8080/jsonrpc). The request body must be a JSON-RPC 2.0 object.

Common Request Structure:

Common Success Response Structure:

Common Error Response Structure:

4.1. puppeteer.navigate

Navigates the active page to the specified URL.

• Parameters:
  • url (string, required): The URL to navigate to.
  • timeout (integer, optional): Navigation timeout in milliseconds. Defaults to default_timeout from config.json.
  • wait_until (string, optional): When to consider navigation successful. Options: “load”, “domcontentloaded”, “networkidle”, “commit”. Default: “load”.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.navigate", "params": {"url": "http://example.com"}, "id": "nav1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "url": "http://example.com/",
      "http_status": 200
    },
    "id": "nav1"
  }
  

4.2. puppeteer.get_current_url

Retrieves the URL of the active page.

• Parameters: None (empty object {} or omit params).
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.get_current_url", "params": {}, "id": "geturl1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (after navigating to example.com):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "url": "http://example.com/"
    },
    "id": "geturl1"
  }
  

4.3. puppeteer.get_page_title

Retrieves the title of the active page.

• Parameters: None.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.get_page_title", "params": {}, "id": "gettitle1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (after navigating to example.com):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "title": "Example Domain"
    },
    "id": "gettitle1"
  }
  

4.4. puppeteer.take_page_screenshot

Captures a screenshot of the active page.

• Parameters:
  • path (string, optional): Absolute path to save the screenshot. If not provided, the image is returned as a base64 encoded string.
  • full_page (boolean, optional): Whether to capture the full scrollable page. Default: true.
  • type (string, optional): Screenshot type. “png” or “jpeg”. Default: “png”.
  • quality (integer, optional): Quality for JPEG screenshots (0-100). Default: Not set.
  • omit_background (boolean, optional): Hides default white background for PNGs, allowing capture of transparent websites. Default: false.
• Example Request (Save to File):

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.take_page_screenshot", "params": {"path": "/home/ubuntu/puppeteer_mcp_service/example.png"}, "id": "screenfile1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (Save to File):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "file_path": "/home/ubuntu/puppeteer_mcp_service/example.png"
    },
    "id": "screenfile1"
  }
  

• Example Request (Base64):

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.take_page_screenshot", "params": {"type": "jpeg", "quality": 80}, "id": "screenbase64"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (Base64):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "image_base64": "/9j/4AAQSkZJRgABAQAAAQABAAD/... (long base64 string) ..."
    },
    "id": "screenbase64"
  }
  

4.5. puppeteer.get_page_content

Retrieves the full HTML content of the active page.

• Parameters: None.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.get_page_content", "params": {}, "id": "getcontent1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "content": "<!doctype html><html><head>...</body></html>"
    },
    "id": "getcontent1"
  }
  

4.6. puppeteer.click_element

Clicks an element specified by a CSS selector.

• Parameters:
  • selector (string, required): CSS selector for the target element.
  • timeout (integer, optional): Timeout in milliseconds to wait for the element. Defaults to default_timeout.
  • button (string, optional): Mouse button to use (“left”, “middle”, “right”). Default: “left”.
  • click_count (integer, optional): Number of times to click. Default: 1.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.click_element", "params": {"selector": "a[href=\"https://www.iana.org/domains/example\"]"}, "id": "click1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success"
    },
    "id": "click1"
  }
  

4.7. puppeteer.fill_form_field

Fills a form field (input, textarea) with the specified value.

• Parameters:
  • selector (string, required): CSS selector for the target form field.
  • value (string, required): The value to fill into the field.
  • timeout (integer, optional): Timeout in milliseconds. Defaults to default_timeout.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.fill_form_field", "params": {"selector": "#searchInput", "value": "Playwright"}, "id": "fill1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success"
    },
    "id": "fill1"
  }
  

4.8. puppeteer.get_element_text

Retrieves the text content of an element specified by a CSS selector.

• Parameters:
  • selector (string, required): CSS selector for the target element.
  • timeout (integer, optional): Timeout in milliseconds. Defaults to default_timeout.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.get_element_text", "params": {"selector": "h1"}, "id": "gettext1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (on example.com):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "text": "Example Domain"
    },
    "id": "gettext1"
  }
  

4.9. puppeteer.get_element_attribute

Retrieves the value of a specified attribute from an element.

• Parameters:
  • selector (string, required): CSS selector for the target element.
  • attribute_name (string, required): The name of the attribute to retrieve (e.g., “href”, “src”, “value”).
  • timeout (integer, optional): Timeout in milliseconds. Defaults to default_timeout.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.get_element_attribute", "params": {"selector": "a", "attribute_name": "href"}, "id": "getattr1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (on example.com, for the first link):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "value": "https://www.iana.org/domains/example"
    },
    "id": "getattr1"
  }
  

4.10. puppeteer.execute_javascript

Executes a JavaScript function or expression in the context of the active page.

• Parameters:
  • script (string, required): The JavaScript code to execute. This should be a string representing a function body or an expression. Example: "() => document.title" or "document.querySelector('h1').innerText".
  • args (any, optional): An argument to pass to the JavaScript function. If the script is a function, args will be passed as its argument. Playwright handles serialization.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.execute_javascript", "params": {"script": "() => ({ title: document.title, url: window.location.href })"}, "id": "execjs1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "result": {
        "title": "Example Domain",
        "url": "http://example.com/"
      }
    },
    "id": "execjs1"
  }
  

4.11. puppeteer.create_context

Creates a new isolated browser context.

• Parameters:
  • context_options (object, optional): Playwright browser context options (e.g., user_agent, viewport, storage_state). See Playwright documentation for available options.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.create_context", "params": {"context_options": {"user_agent": "MyCustomAgent/1.0"}}, "id": "createctx1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "context_id": "context_abc123xyz"
    },
    "id": "createctx1"
  }
  

4.12. puppeteer.switch_context

Switches the active browser context.

• Parameters:
  • context_id (string, required): The ID of the context to switch to (obtained from puppeteer.create_context or the initial default context ID).
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.switch_context", "params": {"context_id": "context_abc123xyz"}, "id": "switchctx1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

4.13. puppeteer.close_context

Closes a browser context and all its pages.

• Parameters:
  • context_id (string, required): The ID of the context to close.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.close_context", "params": {"context_id": "context_abc123xyz"}, "id": "closectx1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response:

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "closed_context_id": "context_abc123xyz"
    },
    "id": "closectx1"
  }
  

4.14. puppeteer.get_console_logs

Note: This method is currently a placeholder.
Retrieves console log messages from the browser. A full implementation would require collecting these messages via event listeners in the BrowserManager.

• Parameters: None.
• Example Request:

  curl -X POST -H "Content-Type: application/json" \
  -d '{"jsonrpc": "2.0", "method": "puppeteer.get_console_logs", "params": {}, "id": "getlogs1"}' \
  http://localhost:8080/jsonrpc
  

• Example Success Response (Current Placeholder):

  {
    "jsonrpc": "2.0",
    "result": {
      "status": "success",
      "logs": [],
      "message": "Log retrieval is not fully implemented yet. Console messages should be monitored via Playwright's event system."
    },
    "id": "getlogs1"
  }
  

5. Error Handling

The service uses standard JSON-RPC 2.0 error objects. Common error codes include:

• -32700: Parse error. Invalid JSON was received by the server.
• -32600: Invalid Request. The JSON sent is not a valid Request object.
• -32601: Method not found. The method does not exist / is not available.
• -32602: Invalid params. Invalid method parameter(s).
• -32603: Internal error. Internal JSON-RPC error.
• -32000: Browser Operation Error. Generic error during a browser operation (e.g., navigation failure, screenshot failure).
• -32001: Page Not Available Error. No active page is available, or creation of a new page failed.
• -32002: Element Not Found Error. A specified element could not be found on the page within the timeout.
• -32003: Configuration Error. Problem with service or browser configuration.

Example Error Response:

{
  "jsonrpc": "2.0",
  "error": {
    "code": -32602,
    "message": "Invalid params: 'url' parameter is required and must be a string."
  },
  "id": "nav_error_1"
}

6. Health Check

A simple HTTP GET endpoint is available at /health for monitoring the service status.

• Endpoint: GET /health (e.g., http://localhost:8080/health)
• Success Response (Browser Connected):

  {
    "status": "ok",
    "browser_status": "connected"
  }
  

• Success Response (Browser Disconnected/Degraded):

  {
    "status": "degraded",
    "browser_status": "disconnected"
  }
  

• Error Response (Browser Unavailable):

  {
    "status": "error",
    "browser_status": "unavailable_or_failed_to_start"
  }
  

7. Deployment Notes

• The service is currently designed to be run directly using Uvicorn.
• For production deployments, consider running Uvicorn behind a reverse proxy like Nginx or using a process manager like Supervisor or systemd to ensure reliability and manage logging.
• Ensure the environment has sufficient resources (CPU, memory) for running browser instances, especially if handling concurrent requests or multiple contexts.
• The playwright install command might need to be run in the deployment environment to ensure browser binaries are available.

This concludes the initial documentation for the Puppeteer MCP Service.