What is Bruniai Action

bruniai-action is a GitHub Action designed for visual regression testing using Playwright MCP and OpenAI. It automates the process of comparing images to identify visual differences, ensuring consistency in web applications and UI-driven projects.

Use cases

Use cases for bruniai-action include testing visual changes in web applications after code updates, ensuring design systems maintain visual integrity, and providing quick feedback on UI changes during development.

How to use

To use bruniai-action, set up a virtual environment, install the required dependencies, run the Playwright MCP server, and execute the provided script with your base and PR URLs. Additionally, configure your OpenAI API key in the .env file.

Key features

Key features include automatic image comparison, highlighting visual differences, easy integration into CI/CD pipelines, configurable thresholds for visual changes, and fast automated feedback for commits and pull requests.

Where to use

bruniai-action is suitable for web development, design systems, and any UI-driven projects where visual consistency is critical. It can be integrated into CI/CD workflows for continuous visual testing.

🎨 Visual Regression Testing with Playwright and OpenAI

This GitHub Action automates visual regression testing by comparing screenshots and highlighting differences between production and pull request versions. It’s perfect for CI/CD pipelines, helping teams catch unintended visual changes in web applications, design systems, and UI-driven projects.

🚀 Quick Start

Add this to your existing workflow file or or create a workflow file like this if you don’t have one (e.g. .github/workflows/visual-regression.yml):

name: Visual Regression Test
on:
  pull_request:
    branches: [main]

jobs:
  visual-regression:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Run Visual Regression Test
        uses: nevinbuilds/[email protected]
        with:
          # Your production URL.
          base-url: "https://www.example.com/"
          # Update to your preview URL format.
          pr-url: "https://example-git-${{ steps.branch-name.outputs.branch_name }}-{{github.actor}}.vercel.app"
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Saving artifacts

If you want to save artifacts in your github workflow add the following to the previous snippet :

- name: Upload images as artifacts
  uses: actions/upload-artifact@v4
  if: always()
  with:
    name: visual-diff-images
    path: images
    retention-days: 1

📋 Requirements

• 📦 GitHub repository

• 🔑 OpenAI API key (set as OPENAI_API_KEY secret in your repository in /settings/secrets/actions)

• 🔐 GitHub token with the following permissions to the repository:

  • Read access to code, deployments, and metadata
  • Read and Write access to commit statuses and pull requests

The Github token is automatically provided by the GitHub Action when installing the bruniai app in your repository.

If you don’t want to install the app you can provide the token as a env variable in your workflow similar to how you pass the openai key :

env:
  OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

This will ensure that the action can create comments in your PR, read the title and description from the PR and other related things that need access to the repository.

🔍 Preview URL format

The preview URL format depends on your deployment platform. Here are the common formats:

🌐 Vercel Preview URLs

Vercel automatically creates preview deployments for pull requests. The URL format is:

https://[project-name]-git-[branch-name]-[username].vercel.app

Example:

pr-url: "https://myapp-git-${{ github.head_ref }}-myusername.vercel.app"

🏗️ Netlify Preview URLs

Netlify creates preview deployments with the following format:

https://deploy-preview-[PR-number]--[site-name].netlify.app

Example:

pr-url: "https://deploy-preview-${{ github.event.pull_request.number }}--myapp.netlify.app"

🔎 Finding Your Preview URL Format

1. For Vercel:

   • Go to your Vercel dashboard
   • Click on your project
   • Look at the preview deployment URLs for recent PRs
   • The format will be consistent across all preview deployments

2. For Netlify:

   • Go to your Netlify dashboard
   • Click on your site
   • Look at the “Deploy previews” section
   • The URL format will follow the pattern shown above

🌍 Custom Domains

If you’re using custom domains, replace the platform-specific domain with your custom domain while keeping the preview-specific parts of the URL.

✨ Features

• 📸 Automated screenshot comparison
• 🤖 AI-powered analysis of visual changes
• 🔄 GitHub PR integration
• 📊 Detailed reporting
• ⚡ Easy CI/CD integration

📚 Documentation

• Getting Started - Local development setup
• Configuration Guide - Configuration options
• Usage Examples - Common use cases
• Troubleshooting - Common issues and solutions
• Contributing - How to contribute
• Code of Conduct - Community guidelines

📄 License

See LICENSE for details.

Testing

See tests/README.md for setup and usage.

To run all tests:

pytest

To check coverage:

pytest --cov=src --cov-report=term-missing

To generate an HTML coverage report:

pytest --cov=src --cov-report=html
open htmlcov/index.html