What is Mcp Pyautogui Server

mcp-pyautogui-server is a Model Context Protocol (MCP) server that enables automated GUI testing and control using PyAutoGUI, allowing users to interact with graphical user interfaces programmatically.

Use cases

Use cases include automated testing of applications, creating bots for repetitive tasks, taking screenshots for documentation, and performing image recognition tasks on the screen.

How to use

To use mcp-pyautogui-server, install it via pip with the command ‘pip install mcp-pyautogui-server’. Configure the server settings in the appropriate configuration file based on your operating system, and then run the server to start automating GUI tasks.

Key features

Key features include mouse control (movement, clicks, drag-and-drop), keyboard control (typing, key presses, hotkey combinations), screen operations (taking screenshots, finding images, getting pixel colors), and cross-platform support for Windows, macOS, and Linux.

Where to use

mcp-pyautogui-server can be used in software testing, automated user interface interactions, and any application requiring GUI automation across various platforms.

mcp-pyautogui-server

A MCP (Model Context Protocol) server that provides automated GUI testing and control capabilities through PyAutoGUI.

Features

• Control mouse movements and clicks
• Simulate keyboard input
• Take screenshots
• Find images on screen
• Get screen information
• Cross-platform support (Windows, macOS, Linux)

Tools

The server implements the following tools:

Mouse Control

• Move mouse to specific coordinates
• Click at current or specified position
• Drag and drop operations
• Get current mouse position

Keyboard Control

• Type text
• Press individual keys
• Hotkey combinations

Screen Operations

• Take screenshots
• Get screen size
• Find image locations on screen
• Get pixel colors

Installation

Prerequisites

• Python 3.12+
• PyAutoGUI
• Other dependencies will be installed automatically

Install Steps

Install the package:

pip install mcp-pyautogui-server

Claude Desktop Configuration

On MacOS:

~/Library/Application\ Support/Claude/claude_desktop_config.json

On Windows:

%APPDATA%/Claude/claude_desktop_config.json

Development/Unpublished Servers Configuration:

{
  "mcpServers": {
    "mcp-pyautogui-server": {
      "command": "uv",
      "args": [
        "--directory",
        "/path/to/mcp-pyautogui-server",
        "run",
        "mcp-pyautogui-server"
      ]
    }
  }
}

Published Servers Configuration:

{
  "mcpServers": {
    "mcp-pyautogui-server": {
      "command": "uvx",
      "args": [
        "mcp-pyautogui-server"
      ]
    }
  }
}

Development

Building and Publishing

1. Sync dependencies and update lockfile:

uv sync

2. Build package distributions:

uv build

3. Publish to PyPI:

uv publish

Note: Set PyPI credentials via environment variables or command flags:

• Token: --token or UV_PUBLISH_TOKEN
• Username/password: --username/UV_PUBLISH_USERNAME and --password/UV_PUBLISH_PASSWORD

Debugging

For the best debugging experience, use the MCP Inspector.

Launch the MCP Inspector via npm:

npx @modelcontextprotocol/inspector uv --directory /path/to/mcp-pyautogui-server run mcp-pyautogui-server

The Inspector will display a URL that you can access in your browser to begin debugging.

License

This project is licensed under the MIT License - see the LICENSE file for details.