Scriptless Test Automation: How It Works, Tools, and Limits

The Steps a Tester Writes

The test is a list of steps in a JSON file, steps.json. Each row names an action and, when the action needs them, an element and a value. There is no code in it:

[
  { "action": "open", "value": "https://ecommerce-playground.lambdatest.io/" },
  { "action": "type", "element": "searchBox", "value": "iPhone" },
  { "action": "click", "element": "searchButton" },
  { "action": "verifyText", "element": "resultsTitle", "value": "Search - iPhone" }
]

A second test is a second file of rows. Keeping test data in rows like this is the same idea behind data-driven testing, applied to the steps themselves.

The Object Repository

Element names map to CSS locators in one file, elements.json, so the tests never contain a locator:

{
  "searchBox": "#search input[name='search']",
  "searchButton": "#search button[type='submit']",
  "resultsTitle": "#product-search h1"
}

The Engine That Runs the Steps

The engine is the only code, and it is written once. It is printed here in three parts that join into one file, engine.js. The first part is the action library, which loads both JSON files, looks up element names in the repository, and maps each action to a Selenium WebDriver call:

// engine.js, part 1 of 3: action library
const { Builder, By, until } = require('selenium-webdriver');
const steps = require('./steps.json');
const elements = require('./elements.json');

const find = (driver, name) =>
  driver.wait(until.elementLocated(By.css(elements[name])), 10000);

const actions = {
  open: (driver, step) => driver.get(step.value),
  type: async (driver, step) =>
    (await find(driver, step.element)).sendKeys(step.value),
  click: async (driver, step) => (await find(driver, step.element)).click(),
  verifyText: async (driver, step) => driver.wait(until.elementTextContains(
    await find(driver, step.element), step.value), 10000),
};

The second part runs the steps in order and stops at the first failure. It reports each step by its number, its action, and the element it used, or the URL for the open step:

// engine.js, part 2 of 3: step runner and reporting
async function runSteps(driver) {
  for (const [index, step] of steps.entries()) {
    const label = 'step ' + (index + 1) + ' ' + step.action + ' '
      + (step.element || step.value);
    try {
      await actions[step.action](driver, step);
      console.log('passed  ' + label);
    } catch (err) {
      console.log('FAILED  ' + label + ' (' + err.name + ')');
      throw err;
    }
  }
}

The third part is the execution layer. It opens a Chrome session on the TestMu AI grid, runs the steps, and marks the session passed or failed:

// engine.js, part 3 of 3: execution on the TestMu AI grid
const capabilities = { browserName: 'chrome', browserVersion: 'latest',
  'LT:Options': { username: process.env.LT_USERNAME,
    accessKey: process.env.LT_ACCESS_KEY, platformName: 'Windows 11',
    build: 'Scriptless engine demo', name: 'Ecommerce search' } };

(async () => {
  const driver = await new Builder().withCapabilities(capabilities)
    .usingServer('https://hub.lambdatest.com/wd/hub').build();
  let status = 'failed';
  try {
    await runSteps(driver);
    status = 'passed';
  } finally {
    await driver.executeScript('lambda-status=' + status);
    await driver.quit();
  }
})();

To run it, save the three parts in order as engine.js beside steps.json and elements.json, install selenium-webdriver from npm, set LT_USERNAME and LT_ACCESS_KEY, and run node engine.js. I used selenium-webdriver 4.50.0, which needs Node.js 22 or later, and the run on Chrome 154 and Windows 11 (build 107903667) passed in 9 seconds with this output:

passed  step 1 open https://ecommerce-playground.lambdatest.io/
passed  step 2 type searchBox
passed  step 3 click searchButton
passed  step 4 verifyText resultsTitle

Because the engine is plain Selenium, the same steps run on any of the 3,000+ browser and OS combinations on TestMu AI's Automation Cloud once you change the capabilities, and starting one engine process per combination runs them in parallel. The Selenium With JavaScript guide covers the credentials and capabilities the engine reads.

What Happens When a Locator Changes?

When a locator changes, every scriptless test that uses that element fails at the same step until its entry in the object repository is updated. Self-healing tools re-find the element on their own instead.

To see what a tester sees, I pointed resultsTitle at an h2 that does not exist on the results page and ran the same steps again. The first three steps passed, and the engine named the step and the element that broke:

passed  step 1 open https://ecommerce-playground.lambdatest.io/
passed  step 2 type searchBox
passed  step 3 click searchButton
FAILED  step 4 verifyText resultsTitle (TimeoutError)

Below those lines, Node printed Selenium's error, "Waiting for element to be located By(css selector, #product-search h2)", and exited with code 1, so a CI job running the engine fails too.

The fix is one line in elements.json, and every test that uses resultsTitle picks it up. Commercial platforms re-find a changed element on their own, which saves the edit but can pick the wrong element, as the self-healing test in the no-code test automation guide linked above shows.

Austin Siewert

Co-Founder, Steadfast Systems

Discovered @TestMu AI yesterday. Best browser testing tool I've found for my use case. Great pricing model for the limited testing I do 👏

2M+ Devs and QAs rely on TestMu AI

Deliver immersive digital experiences with Next-Generation Mobile Apps and Cross Browser Testing Cloud

Sign up with googleSign up with Email

Benefits of Scriptless Testing

• Wider authoring - Manual testers, business analysts, and product managers can automate the checks they know best, without learning a framework.
• Change isolation - A changed locator is fixed once in the object repository, or re-found by the tool, instead of in every test that touches the element.
• Readable tests - Steps such as type searchBox iPhone read as a description of the test, so reviewers can check coverage against requirements.

Limits of Scriptless Testing

• Logic ceiling - Conditions, loops, and computed checks need step types the engine supports, or a coded step.
• Format lock-in - Steps stored in a vendor's format move with you only if the tool exports them to a framework you can run elsewhere.
• Debugging distance - When a step fails, you work from the tool's step trace and screenshots instead of a debugger with breakpoints.
• Healing false positives - Self-healing saves edits but can bind to the wrong element and pass a test that should fail, so healed steps need a reviewer.

How to Choose a Scriptless Testing Tool

Match the tool to the people who will write the tests and to where the tests must run, then check it against the limits above:

• Authoring style - Recordings suit testers who work through the UI, keyword rows suit teams with a shared step vocabulary, and plain English suits product owners and analysts.
• Platform coverage - Confirm the tool covers every surface you test, such as web, native mobile, desktop, and API.
• Code export - Prefer tools that export tests to a framework you can run elsewhere, so the suite is not tied to one vendor's format.
• CI runner - Look for a command-line runner or an API that returns standard exit codes, so a failed test fails the pipeline.
• Healing review - Check that every self-healed step is logged and can be accepted or rejected before it changes the test.
• Debug artifacts - Expect step screenshots, logs, and video, since you debug from the tool's trace instead of a debugger.

Low-code tools such as Katalon Studio also let you drop into Groovy or Java for a step the tool cannot express, which raises the logic ceiling but brings back the need for someone who codes.