What is Foundrybrowserautomation

FoundryBrowserAutomation is a modern Electron desktop application that integrates Microsoft Foundry Local SDK for AI model inference with Playwright MCP for intelligent browser automation.

Use cases

Use cases include automating repetitive web tasks, testing web applications, and providing a user-friendly interface for non-technical users to interact with web browsers using AI.

How to use

To use FoundryBrowserAutomation, install Node.js v16+, run the application, and utilize the AI Model Tab to configure AI models, the Browser Automation Tab to execute commands in natural language, and the Logs Tab to monitor real-time activity.

Key features

Key features include local AI integration, intelligent browser automation using natural language commands, automatic screenshot capture, a modern UI with dark/light mode support, real-time monitoring, and built-in diagnostics.

Where to use

FoundryBrowserAutomation can be used in various fields such as software development, web testing, and automation tasks for both technical and non-technical users.

🚀 Foundry Local & Browser Automation Demo

A modern Electron desktop application that demonstrates the integration of Microsoft Foundry Local SDK for AI model inference with Playwright MCP (Model Context Protocol) for intelligent browser automation.

✨ What This App Does

This application showcases the powerful combination of local AI models and browser automation in a sleek, modern interface. It demonstrates how AI can intelligently control web browsers using natural language commands, making web automation accessible to both developers and non-technical users.

🎯 Key Capabilities

• 🤖 Local AI Integration: Execute AI models locally using Microsoft Foundry Local SDK
• 🌐 Intelligent Browser Automation: Control browsers with natural language using Playwright MCP
• 💬 Natural Language Commands: Tell the browser what to do in plain English
• 📸 Visual Feedback: Automatic screenshot capture and display
• 🎨 Modern UI: Beautiful tabbed interface with dark/light mode support
• 📊 Real-time Monitoring: Live logs and status indicators
• 🔧 Built-in Diagnostics: Comprehensive testing and troubleshooting tools

🖼️ Application Interface

The application features a modern, tabbed interface with three main sections:

1. AI Model Tab: Configure and test AI model connections
2. Browser Automation Tab: Execute natural language browser commands
3. Logs Tab: Monitor real-time application activity

Note: To see the application interface, run the app using the instructions below.

🚀 Quick Start

Prerequisites

• Node.js v16+ and npm
• Microsoft Foundry Local SDK (optional - app includes fallback functionality)
• Windows/macOS/Linux (cross-platform support)

Installation & Setup

1. Clone the repository

   git clone <repository-url>
   cd electron-app
   

2. Install dependencies (automatically installs Playwright browsers)

   npm install
   

3. Start the application

   npm start
   

That’s it! The app will automatically set up everything needed and launch with a beautiful splash screen.

📋 How to Use

🤖 AI Model Testing

1. Navigate to the AI Model tab
2. Select a model from the dropdown (if Foundry Local is available)
3. Enter a prompt and click Submit
4. View the AI response in the interface

🌐 Browser Automation

1. Go to the Browser Automation tab

2. Try these example commands in the MCP prompt field:

   Basic Navigation:

   Go to microsoft.com and take a screenshot
   

   Complex Interactions:

   Navigate to github.com, search for "playwright", and screenshot the results
   

   Content Extraction:

   Visit wikipedia.org, search for "artificial intelligence", and take a screenshot of the article
   

3. Click Run MCP Prompt to execute

4. View screenshots and results in the interface

🔧 Testing & Diagnostics

• Run MCP Test: Click to run comprehensive browser automation tests
• Diagnose MCP: Check MCP installation and configuration
• View Logs: Monitor all application activity in real-time

🛠️ Available Commands

🏗️ Architecture

Core Components

├── main.js              # Electron main process & IPC handling
├── renderer.js          # UI logic & event handling
├── preload.js          # Secure IPC bridge
├── browser-automation.js # Playwright MCP integration
├── fallback-mcp.js     # Fallback automation (when MCP unavailable)
├── theme.js            # Dark/light mode & UI animations
└── tests/
    ├── mcp-test.js     # Comprehensive MCP testing
    └── find-mcp.js     # MCP diagnostics

UI Structure

├── index.html          # Main application layout
├── styles.css          # Modern styling & animations
├── animation.css       # UI transition effects
└── splash.html         # Startup splash screen

🎨 Features Showcase

🌓 Theming

• Automatic dark/light mode based on system preferences
• Manual theme toggle with smooth transitions
• Consistent color scheme throughout the interface

📱 Responsive Design

• Tabbed interface for organized content
• Card-based layout with subtle shadows and rounded corners
• Responsive controls that adapt to window size

⚡ Performance

• Lazy loading of heavy components
• Efficient IPC communication between processes
• Background processing for non-blocking operations

🔒 Security

• Context isolation enabled in Electron
• Secure IPC communication via preload scripts
• Sandboxed renderer process

🧪 Testing

Automated Tests

Run all tests:

npm test

Test specific functionality:

npm run test:mcp      # Test browser automation
npm run test:browser  # Test basic browser functionality

Manual Testing Examples

Try these commands to test the application:

1. Simple Navigation Test

   Go to example.com and take a screenshot
   

2. Search Test

   Navigate to duckduckgo.com, search for "electron apps", take a screenshot
   

3. Multi-step Test

   Go to github.com, click on "Explore", then take a screenshot of the page
   

🚨 Troubleshooting

Common Issues

MCP Not Working?

npm run diagnose:mcp  # Check MCP installation
npm run fix:mcp       # Reinstall MCP components

Browser Won’t Launch?

npx playwright install  # Install browser binaries

Connection Issues?

• Ensure Foundry Local is running (if using AI features)
• Check firewall settings
• Verify network connectivity

Getting Help

1. Check the Logs tab in the application for detailed error messages
2. Run diagnostics using npm run diagnose:mcp
3. Check the console output when running from terminal
4. Review error messages in the application interface

🔧 Advanced Configuration

Custom Model Configuration

Edit the model dropdown options in renderer.js to add your preferred models.

Browser Automation Settings

Modify browser launch options in browser-automation.js for specific requirements.

UI Customization

Update styles.css and animation.css to customize the application appearance.

📦 Building & Distribution

Create distributable packages:

npm run package

This creates platform-specific packages in the dist/ directory:

• Windows: .exe installer
• macOS: .dmg disk image
• Linux: .AppImage portable app

🛡️ Security & Privacy

• Local Processing: All AI inference happens locally (no cloud dependency)
• Secure Automation: Browser automation runs in isolated contexts
• No Data Collection: Application doesn’t collect or transmit user data
• Open Source: Full source code available for review

📚 Technologies Used

🎯 Use Cases

This application demonstrates several practical use cases:

• 🧪 AI Model Testing: Quick testing of local AI models
• 🕷️ Web Scraping: Automated data extraction from websites
• 🧑‍💻 Browser Testing: Automated UI testing with natural language
• 📊 Monitoring: Automated website monitoring and screenshot capture
• 🎓 Education: Learning browser automation and AI integration
• 🚀 Prototyping: Rapid prototyping of automation workflows

🔮 Future Enhancements

• 🤖 Advanced AI Integration: Support for more AI providers
• 📝 Script Recording: Record browser actions as reusable scripts
• 🔄 Workflow Builder: Visual workflow creation interface
• ☁️ Cloud Integration: Optional cloud AI model support
• 📱 Mobile Support: Electron-based mobile app version

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🤝 Contributing

Contributions are welcome! Please feel free to submit a Pull Request. For major changes, please open an issue first to discuss what you would like to change.

🆘 Support

If you encounter any issues or have questions:

1. Check the troubleshooting section above
2. Run the built-in diagnostics (npm run diagnose:mcp)
3. Review the application logs in the Logs tab
4. Open an issue on GitHub with detailed information

Built with ❤️ using Electron, Playwright, and Microsoft Foundry Local SDK