How To Decide What Should Be Automated?

What Is Its Priority?

Some features of the application or the project have a higher chance of failing as compared to others. Such features must be automated as the chances of bug detection are comparatively less with manual testing. For instance, with manual testing there are chances of the error being missed or judged with less precision. Hence, to avoid any risks, such features must be tested using automation tools. The test case would be run similarly during each cycle, thus giving ideal data for comparison and error detection.

Priority is only half the question, because stability decides what the script costs you later. A screen whose layout and locators change every sprint forces you to repair the test about as often as you run it, so the upkeep stays high while the payback stays flat. Automate the high-risk paths that have settled, and cover the areas still under active redesign manually until the interface stops moving.

Key Takeaway: Automate the high-risk features whose layout and locators have settled, and keep areas still under active redesign manual until the interface stops moving.

How Often Do You Need To Run It?

Automated testing is an ideal choice for the suites where you need to run the test cases many times. Repeating the test cases is not a time-consuming activity when it comes to automated testing. However, if a feature needs to be tested as soon as possible, then it is not preferable to automate it. Tests that require quick results should be performed manually. Regression testing and smoke testing suites are the standing examples on the automate side, because both re-run on every build and both re-check behavior that already worked once.

Key Takeaway: A test suite that has to run many times over is worth automating, while a check whose result is needed immediately is faster to run manually.

How Many Test Cases Have You Got To Deal With?

Another important parameter that decides whether a test suite needs to be automated or not, is the number of test cases under the suite. It would not be practical enough to automate a suite consisting of ten test cases. The frequency of the test-runs does not matter much here. However, if the test suite consists of a considerably higher number of test cases required to be run more than once, then automating it is a good choice as it would be worth the cost of the automation tools.

Pick the layer as well as the count. A check that only confirms a business rule or a response payload runs faster and breaks far less often in unit testing or API testing than in the browser, so pushing it down the stack keeps the browser suite small and the feedback quick. Reserve the browser layer for the journeys a user actually clicks through end to end, and let the layers below carry the rest of the cases.

Key Takeaway: A suite of about ten test cases rarely repays the cost of automation tools, while a large suite that runs more than once does, and checks on business rules or response payloads belong in unit or API tests rather than browser tests.

Are You Going To Execute More Than One Test Simultaneously?

Are you going to run a single test at a time or a given test case simultaneously on multiple machines? This also plays a role in deciding whether the suite should be automated or not. Manually running a test case on a single machine is good but, the simultaneous execution of the same test case on multiple machines may require more members of the team. To save such wastage of human resources, such test suites are often automated. Teams already automating testing in DevOps pipelines schedule the scripts to run the test cases at exactly the same time on multiple machines.

Key Takeaway: Running one test case on several machines at the same time ties up extra team members when done by hand, so a suite that needs simultaneous execution is a strong automation candidate.

Are You Trying To Perform Cross-Browser Testing?

If you have tested your web application in a single browser only, you are definitely missing out on a lot. Cross-browser testing is an important parameter on which your app must perform well. It makes sure that your application performs equally well across various browsers, operating systems and platforms. However, the process can be cumbersome and time-consuming at times. It is recommended that you automate your cross-browser tests unless you want to see your team engaged with the same test for hours or even days!

Key Takeaway: Checking an application across many browsers, operating systems, and platforms takes a team hours or days by hand, so cross-browser tests are worth automating.

What Is The Volume Of The Input Data?

If the volume of the test data is quite large, you must automate it. Even if such a test needs to be run only once, it makes all the sense to automate the test. Not only will it become convenient for you, but also save your time.

Key Takeaway: A test that feeds a large volume of input data is worth automating even when it runs only once, because automation saves the time that entering the data manually would consume.

How Urgently Do You Need The Report?

How urgently or well-documented you want your test report to be also decides whether the testing should be automated or manual. The top automation tools, along with flawless testing, generate automatic, excellent and quick test reports for ready reference once the testing cycle completes. With a ready availability of test log, the crucial "go-live" decision can be taken efficiently and quickly.

Key Takeaway: Automation tools produce a documented test log as soon as the cycle completes, so a team that needs the go-live decision made quickly benefits from automating.

How Expert Is Your Team?

Automated testing requires specific resources to be implemented. The automation tools require a certain level of expertise from your team. A team with a limited technical knowledge can perform manual testing efficiently but cannot assure the same when it comes to working with leading automation tools. For every tool, a panel with an advanced knowledge of the framework is required.

The skill bar has moved since this decision was first framed, and the move is visible in the tooling. Playwright 1.56 shipped three agent definitions that walk a language model through building a test: a planner that explores the application and writes a Markdown test plan, a generator that turns that plan into Playwright Test files, and a healer that runs the suite and repairs the tests that fail. A team installs them with npx playwright init-agents. A tester who could not previously design a locator strategy can now produce a runnable spec.

Treat that output as a draft rather than a verdict. The 2025 Stack Overflow Developer Survey reports that 66% of developers name AI output that is "almost right, but not quite" as their single biggest frustration, and 45.2% report that debugging AI-generated code is more time consuming. A generated test that passes for the wrong reason is worse than no test at all, because it reports green while the behavior underneath is broken. Someone on the team still has to read the assertion and confirm it checks the thing that matters.

So AI shifts two inputs to the automate or do not automate call without replacing the call itself. Authoring cost falls, which pulls borderline suites across the line. Repair cost falls too, because a healing agent absorbs some of the locator churn that made an unstable screen a poor candidate. Review cost does not fall. Keep the rule that a test nobody can review is a test nobody should ship, and keep exploratory passes, usability checks and visual judgment with a person.

Key Takeaway: AI test agents such as the Playwright planner, generator, and healer lower the cost of writing and repairing automated tests, but someone on the team still has to confirm that each generated assertion checks the right behavior.

Does The Test Require User Involvement?

Automated testing has its own advantages and is known for giving accurate results but, in some cases, it cannot be used. For instance, in case any of your features require actual user interaction, it would not make much sense to automate the test.

Key Takeaway: A feature that depends on actual user interaction should be tested manually, because an automated test cannot stand in for a real person using the application.

Should You Let An AI Agent Run The Test Instead Of Scripting It?

No, not for anything you need as a repeatable gate. An agent driving a browser live is useful for exploration, while a scripted test is what you can trust to fail a build. The option exists because of tooling such as the Playwright MCP server, which drives a real browser through the accessibility tree rather than screenshots, so a language model can open the application, work through a flow, and report what it found without anyone writing a spec first.

That makes exploration cheap, and exploration is exactly the column this guide keeps with a person rather than a script. It does not turn the run into a regression check. The same prompt can take a different path on the next run, so a pass tells you the agent got through, not that the build is sound. The Playwright MCP documentation also states plainly that the server is not a security boundary, which rules it out of an unattended pipeline pointed at production data.

The healer agent carries a related caveat. The Playwright test agents documentation describes the healer as replaying the failing steps, inspecting the current interface, and re-running the test until it passes or until guardrails stop the loop. A test that was patched until it went green can be green because the assertion moved, not because the defect is gone. Read the patch the healer proposes before you commit it.

So treat an agent run as a third column beside manual and automated. Use it to widen exploratory coverage and to cut the authoring cost on suites that already pass the tests above. Keep a deterministic scripted suite as the thing that blocks a release.

Key Takeaway: An AI agent driving the browser live is an exploration tool rather than a regression gate, because the run is not reproducible. Keep a deterministic scripted suite as the check that blocks a release.

What Are Some Cases Where You Must Not Automate?

Although, it varies from situation to situation, yet there are some cases where you must not automate as a rule of thumb. Test cases that you need to run only once, until exceptionally complex and involving a large volume of input data, should not be automated. Similarly, the cases requiring exploratory testing, visual confirmation and user response should be tested manually. Besides, partially automating a test suite is not a very good idea. Intermittent tests, the tests which do not produce a predictable and reliable result in order to produce a success or failure condition, are also not good candidates for automated testing.

Thus, this was a quick guide to deciding whether you should automate a test suite or not. Although, the choice is completely at the tester's disposal yet going by the rules of the game can ease your task a bit and give you better returns on investment in terms of time and money.

Wish to add something to the discussion? Comment in the section below!

Key Takeaway: Tests that run only once, exploratory passes, visual confirmation checks, and intermittent tests that produce no predictable pass or fail result should not be automated.