Automation Testing Using Browser Agents

End-to-End Business Workflows

AI browser agents excel at automating complete user journeys that span multiple pages and interactions. A single prompt can initiate a workflow, navigate through the application, validate results, and report the outcome.

Example: Log in to the HR application, create a new employee, assign the employee to the Sales department, verify the employee appears in the directory, and capture a screenshot of the profile.

Quality Assurance and Testing

AI browser agents can be super helpful for quality assurance teams. These teams can leverage these agents to run functional and non-functional tests across multiple browsers and websites.

AI browser agents can simulate end-to-end user journeys, identify issues such as broken links and non-functional forms, and generate detailed execution reports to help teams validate application functionality and improve the overall user experience.

E-Commerce and Retail

AI browser agents can automate many repetitive tasks across e-commerce platforms. They can navigate product catalogs, validate pricing information, verify product availability, update inventory records, and test end-to-end customer journeys such as product search, cart management, and checkout.

They can also help with web scraping and enable retailers to automate routine browser-based workflows while reducing manual effort.

Sales and CRM Workflows

Sales teams can use AI browser agents to automate browser interactions across CRM systems and business applications. For example, agents can navigate prospect directories, populate lead information into CRM platforms, validate customer records, and automate repetitive administrative tasks.

By handling these routine activities, sales teams can spend more time building customer relationships and less time performing manual data entry.

Financial Services and Accounting

In financial and accounting applications, AI browser agents can automate browser-based workflows such as validating transaction records, reconciling data across portals, downloading reports, and verifying invoice details. AI agents can help in monitoring transactions, managing invoices, and generating financial reports.

They can navigate secure financial portals, retrieve transaction details, validate financial records, and categorize expenses based on predefined business rules.

Content Management and Social Media

AI browser agents can automate a variety of browser-based tasks involved in content management and social media operations. They can schedule and publish content, monitor brand mentions, collect engagement metrics, and assist in managing multiple social media platforms from a unified workflow.

With these repetitive tasks automated, teams can maintain a consistent online presence, improve operational efficiency, and focus more on creating high-quality content and engage with their audience.

Understanding the Browser AI Agent

Before diving into the setup, it's helpful to understand how the Browser AI Agent works.

The Browser AI Agent accepts natural language instructions and determines how to execute them. It uses a local large language model to understand the user's intent, plan the sequence of browser actions, identify the appropriate page elements, and execute those actions in a remote browser hosted on Browser Cloud.

The Browser AI Agent is written in TypeScript. It combines Ollama, the Qwen3:8b large language model, and Browser Cloud. Ollama runs the Qwen3:8b model locally, enabling the agent to interpret user prompts and generate an execution plan. Browser Cloud then provides a scalable browser environment where the planned actions are executed and monitored.

The overall architecture is shown below:

With this Browser AI Agent in place, let's see how to configure it to use Browser Cloud and execute browser automation tests using a simple natural language prompt.

Configure Browser Cloud in AI Agent

The following steps should be followed to configure Browser Cloud in AI Agent:

Step 1: Install the Browser Cloud SDK

Install the Browser Cloud package using npm:

npm install @testmuai/browser-cloud

Step 2: Configure Browser Cloud Credentials

Create a .env file (or update the existing one) with your Browser Cloud credentials:

LT_USERNAME=<your_lambdaTest_username>
LT_ACCESS_KEY=<your_lambdaTest_access_key>

These credentials are used to authenticate with Browser Cloud and create remote browser sessions.

Step 3: Import and Initialize the Browser Client

A browser.ts file has been created to import the Browser client into the AI Agent project.

import "dotenv/config";
import { Browser } from "@testmuai/browser-cloud";

export class BrowserClient {
  private readonly client: Browser;

  constructor() {
    this.client = new Browser();
  }

  get instance(): Browser {
    return this.client;
  }
}

This class creates a single reusable instance of the Browser Cloud client by initializing the Browser from @testmuai/browser-cloud. It provides controlled access to this instance through the instance getter, allowing the rest of the application to interact with Browser Cloud without repeatedly creating new client objects.

Step 4: Create a Browser Session

A new browser session should be created before executing any browser actions.

import { BrowserClient } from "./browser.js";
import { BrowserSession } from "./types.js";
import { config } from "../config/config.js";

export class SessionManager {
  private readonly client = new BrowserClient();

  async create(): Promise<BrowserSession> {
    const sdk = this.client.instance;

    const session = await sdk.sessions.create({
      adapter: "playwright",
      lambdatestOptions: {
        build: "Browser AI Agent",
        name: "AI Session",
        platformName: config.platformName,
        browserName: config.browserName,
        browserVersion: config.browserVersion,
        "LT:Options": {
          username: config.LT_USERNAME,
          accessKey: config.LT_ACCESS_KEY,
          video: true,
          console: true,
        },
      },
    });

    const connection = await sdk.playwright.connect(session);

    return {
      id: session.id,
      browser: connection.browser,
      context: connection.page.context(),
      page: connection.page,
    };
  }

  async release(session: BrowserSession): Promise<void> {
    try {
      await session.browser.close();
    } finally {
      await this.client.instance.sessions.release(session.id);
    }
  }
}

The SessionManager class is responsible for creating and managing browser sessions in Browser Cloud. The values for the platform name, browser name, and browser session are read from the .env file.

It starts a new Playwright-compatible browser session using the configured browser and TestMu AI settings, then returns the browser, context, and page objects needed for automation.

When the session is no longer required, it closes the browser and releases the cloud session to free up resources.

Step 5: Connect the AI Agent to the session

Once the session is created, the session information should be passed to the browser service so AI-generated actions such as navigation, clicking, typing, etc. are executed in the remote browser using Browser Cloud.

import { SessionManager } from "./session.js";
import { BrowserSession } from "./types.js";
import { BrowserResult } from "../models/browser-result.js";
import { LocatorResolver } from "./locator-resolver.js";

export class BrowserService {
  private readonly sessionManager = new SessionManager();
  private session?: BrowserSession;
  private resolver?: LocatorResolver;

  async start(): Promise<void> {
    if (!this.session) {
      this.session = await this.sessionManager.create();
    }
  }

  async close(): Promise<BrowserResult> {
    if (this.session) {
      await this.sessionManager.release(this.session);
      this.session = undefined;
    }

    return {
      success: true,
      message: "Browser session closed.",
    };
  }

//...

The BrowserService class acts as the central interface for managing browser interactions throughout the AI agent. It initializes a browser session on demand using the SessionManager and maintains that session for reuse across multiple actions.

When automation is complete, it cleanly closes the browser and releases the cloud session, ensuring that resources are properly cleaned up.

Provide a Natural Language Prompt

The Browser AI Agent expects a prompt.txt file with the scenarios described in plain English

Navigate to https://ecommerce-playground.lambdatest.io/
Enter "iPhone" into the textbox with placeholder "Search for Products"
Click on the "Search" button, and wait for the results page to load
Verify the page title contains "iPhone"

The Browser AI agent reads the prompt.txt file and parses it into steps for further execution. It analyzes each instruction using Ollama, running the Qwen3:8b model. The model understands the intent of each step, determines the browser actions required, locates the appropriate elements on the page, and generates the sequence of browser commands needed to execute the test.

Execute the Test on Browser Cloud

The Browser AI agent sends the generated automation commands to Browser Cloud. The cloud browser performs each action, including navigating to the application, locating the search field, entering the text, clicking the Search button, waiting for the results page to load, and validating that the page title contains "iPhone".

================================================================================
Browser AI Agent
================================================================================
[2026-08-05T13:31:58.753Z] [INFO] [APP] Application started
[2026-08-05T13:31:58.753Z] [INFO] [APP] Reading prompt from: /Users/faisalkhatri/github/browser-ai-agent/prompt.txt
================================================================================
New Request
================================================================================
[2026-08-05T13:31:58.759Z] [INFO] [USER] Navigate to https://ecommerce-playground.lambdatest.io/
Enter "iPhone" into in the textbox with placeholder "Search for Products"
Click on the "Search" button next to the search field, and Wait for the results page to load
Verify the page title contains "iPhone"
[2026-08-05T13:31:58.759Z] [INFO] [APP] Execution plan contains 4 step(s)
================================================================================
Executing Step 1
================================================================================
[2026-08-05T13:31:58.760Z] [INFO] [STEP] Navigate to https://ecommerce-playground.lambdatest.io/
[2026-08-05T13:31:58.773Z] [INFO] [AGENT] Invoking AI agent
[2026-08-05T13:32:09.999Z] [INFO] [TOOL: ] navigate -> https://ecommerce-playground.lambdatest.io/
[2026-08-05T13:33:26.960Z] [INFO] [TOOL: ] navigate -> https://ecommerce-playground.lambdatest.io/index.php
[2026-08-05T13:34:08.453Z] [INFO] [STEP] #1 navigate (41492 ms)
[2026-08-05T13:34:08.455Z] [INFO] [INPUT] {
  "url": "https://ecommerce-playground.lambdatest.io/index.php"
}
[2026-08-05T13:34:08.455Z] [INFO] [OUTPUT] {
  "success": true,
  "message": "Navigation completed.",
  "data": {
    "url": "https://ecommerce-playground.lambdatest.io/index.php"
  }
}
[2026-08-05T13:34:08.455Z] [SUCCESS] [STEP] navigate completed successfully
[2026-08-05T13:34:08.456Z] [SUCCESS] [TOOL] Navigation completed: https://ecommerce-playground.lambdatest.io/index.php
[2026-08-05T13:34:57.637Z] [SUCCESS] [AGENT] Execution completed in 178864 ms
[2026-08-05T13:34:57.639Z] [INFO] [AGENT] Final Response:
[2026-08-05T13:34:57.639Z] [INFO] [ASSISTANT] The browser has successfully navigated to the URL `https://ecommerce-playground.lambdatest.io/index.php`. The page is now ready for further interactions.

================================================================================
Execution Summary
================================================================================
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Step 1: navigate | PASSED | 20242 ms
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Step 2: fill | PASSED | 245 ms
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Step 3: click | PASSED | 793 ms
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Step 4: get_title | PASSED | 326 ms
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Total Steps : 4
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Passed      : 4
[2026-08-05T13:41:09.918Z] [INFO] [SUMMARY] Failed      : 0
[2026-08-05T13:41:09.919Z] [INFO] [APP] Shutting down Browser AI Agent
[2026-08-05T13:41:09.923Z] [SUCCESS] [APP] Browser session closed
[2026-08-05T13:41:09.923Z] [INFO] [APP] Application stopped

This approach combines the reasoning capabilities of an AI model with the reliability and scalability of cloud-based browser execution.

Review the Execution Results

One of the biggest advantages of running AI-driven browser automation on Browser Cloud is the rich execution data available after every run. We can inspect:

• Step-by-step execution timeline
• Detailed execution logs
• Screenshots captured during the test
• Full video recording of the browser session

These artifacts make it much easier to understand how the AI agent executed each instruction, verify the results, and troubleshoot failures when they occur. Instead of relying solely on console logs, we can gain complete visibility into the browser session, making AI-powered automation more reliable, easier to debug, and more practical for real-world testing workflows.