What is Auto Browse Ts

Auto-Browse is an AI-enabled browser automation tool that allows users to interact with web pages using natural language, facilitating the automation of browser tasks and the writing of Playwright tests.

Use cases

Use cases for Auto-Browse include automating repetitive browser tasks, conducting automated testing for web applications, and creating BDD-style tests for better collaboration between developers and testers.

How to use

To use Auto-Browse, install it via npm with the command npm install @auto-browse/auto-browse. Ensure that you meet the Playwright version requirements and configure the necessary environment variables for the LLM.

Key features

Key features of Auto-Browse include natural language interaction with web pages, automation of browser tasks, and integration with Playwright for testing. It simplifies the process of writing tests using BDD patterns.

Where to use

Auto-Browse can be used in various fields such as software testing, web scraping, and any scenario requiring automated interactions with web applications.

Auto-Browse: AI Enabled Browser Automation

Auto Browse is the easiest way to connect your AI agents with the browser using natural language.

🎥 Watch the launch video

Quick start

An AI-powered browser automation agent for automating browser tasks and Write Playwright tests that enables natural language interactions with web pages.

Examples

Check out our TypeScript BDD Example Repository to see a complete implementation using Auto Browse with BDD testing patterns.

Installation

npm install @auto-browse/auto-browse

⚠️ Important: Playwright Version Requirements

Note: Auto Browse currently requires specific versions of Playwright. This requirement will be relaxed in future versions.

Required Versions

"@playwright/test": "1.52.0-alpha-1743011787000"
"playwright": "1.52.0-alpha-1743011787000"

Version Conflicts

If you’re using Auto Browse alongside an existing Playwright setup, you must upgrade to these specific versions. Here’s how to handle common issues:

1. Installation Conflicts

   npm install --legacy-peer-deps
   

   This flag helps resolve peer dependency conflicts during installation.

2. Multiple Playwright Versions

   • Remove existing Playwright installations
   • Clear npm cache if needed: npm cache clean --force
   • Reinstall with the required versions

3. Project Compatibility

   • Update your project’s Playwright configuration
   • Ensure your existing tests are compatible with the alpha version
   • Consider using a separate test environment if needed

🔄 Future releases will support a wider range of Playwright versions. Subscribe to our GitHub repository for updates.

Configuration

Auto Browse requires environment variables for the LLM (Language Model) configuration. Create a .env file in your project root:

# OpenAI (default)
OPENAI_API_KEY=your_openai_api_key_here
LLM_PROVIDER=openai  # Optional, defaults to openai
AUTOBROWSE_LLM_MODEL=gpt-4o-mini  # Optional, defaults to gpt-4o-mini

# Google AI
GOOGLE_API_KEY=your_google_key_here
LLM_PROVIDER=google
AUTOBROWSE_LLM_MODEL=gemini-2.0-flash-lite

# Azure OpenAI
AZURE_OPENAI_API_KEY=your_azure_key_here
AZURE_OPENAI_ENDPOINT=https://your-endpoint.openai.azure.com/
AZURE_OPENAI_API_VERSION=2024-12-01-preview
AZURE_OPENAI_API_DEPLOYMENT_NAME=your-deployment-name
LLM_PROVIDER=azure

# Anthropic
ANTHROPIC_API_KEY=your_anthropic_key_here
LLM_PROVIDER=anthropic
AUTOBROWSE_LLM_MODEL=claude-3

# Google Vertex AI
GOOGLE_APPLICATION_CREDENTIALS=path/to/credentials.json
LLM_PROVIDER=vertex

# Ollama
BASE_URL=http://localhost:11434  # Optional, defaults to this value
LLM_PROVIDER=ollama
AUTOBROWSE_LLM_MODEL=llama3.1

You can find an example configuration in example.env.

Environment Variables

Variable	Description	Default	Required For
LLM_PROVIDER	LLM provider to use	openai	No
AUTOBROWSE_LLM_MODEL	The LLM model to use	gpt-4o-mini	No
OPENAI_API_KEY	OpenAI API key	-	OpenAI
GOOGLE_API_KEY	Google AI API key	-	Google AI
AZURE_OPENAI_API_KEY	Azure OpenAI API key	-	Azure
AZURE_OPENAI_ENDPOINT	Azure OpenAI endpoint URL	-	Azure
AZURE_OPENAI_API_VERSION	Azure OpenAI API version	2024-12-01-preview	Azure
AZURE_OPENAI_API_DEPLOYMENT_NAME	Azure OpenAI deployment name	-	Azure
ANTHROPIC_API_KEY	Anthropic API key	-	Anthropic
GOOGLE_APPLICATION_CREDENTIALS	Path to Google Vertex credentials file	-	Vertex AI
BASE_URL	Ollama API endpoint	http://localhost:11434	No

Supported LLM Providers

Auto Browse supports multiple LLM providers:

• OpenAI (default) - GPT-4 and compatible models
• Google AI - Gemini models
• Azure OpenAI - GPT models on Azure
• Anthropic - Claude models
• Google Vertex AI - PaLM and Gemini models
• Ollama - Run models locally

Usage

Standalone Mode (Without Playwright Test)

Auto Browse can also be used outside of Playwright test context. Here’s a complete form automation example:

import { auto } from "@auto-browse/auto-browse";

async function main() {
	try {
		// Navigate to the form
		await auto("go to https://httpbin.org/forms/post");

		// Take a snapshot to analyze the page structure
		await auto("take a snapshot");

		// Fill out the form
		await auto('type "John Doe" in the customer name field');
		await auto('select "Large" for size');
		await auto('select "Mushroom" for topping');
		await auto('check "cheese" in extras');

		// Submit the form
		await auto("click the Order button");

		// Take a snapshot of the response page
		await auto("take a snapshot of the response page");
	} catch (error) {
		console.error("Error:", error);
	}
}

// Run the script
main().catch(console.error);

In standalone mode, Auto Browse automatically:

• Manages browser lifecycle
• Creates and configures pages
• Handles cleanup

To run standalone scripts:

npx ts-node your-script.ts

Playwright Test Mode

import { test, expect } from "@playwright/test";
import { auto } from "@auto-browse/auto-browse";

test("example test", async ({ page }) => {
	await page.goto("https://example.com");

	// Get text using natural language
	const headerText = await auto("get the header text", { page });

	// Type in an input using natural language
	await auto('type "Hello World" in the search box', { page });

	// Click elements using natural language
	await auto("click the login button", { page });
});

Auto-Detection Mode

The package automatically detects the current page context, so you can skip passing the page parameter:

import { test, expect } from "@playwright/test";
import { auto } from "@auto-browse/auto-browse";

test("simplified example", async ({ page }) => {
	await page.goto("https://example.com");

	// No need to pass page parameter
	const headerText = await auto("get the header text");
	await auto('type "Hello World" in the search box');
	await auto("click the login button");
});

BDD Mode with Playwright-BDD

Auto Browse seamlessly integrates with playwright-bdd for behavior-driven development. This allows you to write expressive feature files and implement steps using natural language commands.

Example Feature File

# features/homepage.feature
Feature: Playwright Home Page

  Scenario: Check title
    Given navigate to https://playwright.dev
    When click link "Get started"
    Then assert title "Installation"

Step Definitions

import { auto } from "@auto-browse/auto-browse";
import { Given, When as aistep, Then } from "./fixtures";

// Generic step that handles any natural language action
aistep(/^(.*)$/, async ({ page }, action: string) => {
	await auto(action, { page });
});

Setup Requirements

1. Install dependencies:

npm install --save-dev @playwright/test @cucumber/cucumber playwright-bdd

2. Configure playwright.config.ts:

import { PlaywrightTestConfig } from "@playwright/test";

const config: PlaywrightTestConfig = {
	testDir: "./features",
	use: {
		baseURL: "https://playwright.dev"
	}
};

export default config;

This integration enables:

• Natural language test scenarios
• Reusable step definitions
• Cucumber reporter integration
• Built-in Playwright context management

Supported Actions

1. Clicking Elements

   await auto("click the submit button");
   await auto("click the link that says Learn More");
   

2. Typing Text

   await auto('type "username" in the email field');
   await auto('enter "password123" in the password input');
   

Features

Core Features:

• Natural language commands for browser automation
• AI-powered computer and browser agent
• Automate any browser task
• Automatic page/context detection
• TypeScript support
• Playwright test integration
• Zero configuration required

Supported Operations:

• Page Navigation (goto URL, back, forward)
• Element Interactions (click, type, hover, drag-and-drop)
• Form Handling (select options, file uploads, form submission)
• Visual Verification (snapshots, screenshots, PDF export)
• Keyboard Control (key press, text input)
• Wait and Timing Control
• Assertions and Validation

Best Practices

1. Be Descriptive

   // Good
   await auto("click the submit button in the login form");
   
   // Less Clear
   await auto("click submit");
   

2. Use Quotes for Input Values

   // Good
   await auto('type "John Doe" in the name field');
   
   // Not Recommended
   await auto("type John Doe in the name field");
   

3. Leverage Existing Labels

   • Use actual labels and text from your UI in commands
   • Maintain good accessibility practices in your app for better automation

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

Thanks to Playwright Team for creating Playwright MCP and Playwright BDD.

License

MIT