Enabling Agile Software Testing With Automation

Persona 1: How To Get Started If Your Team Has No Automation In Place

Identify the skill set of the team:

QA teams have different make-ups. Some are 100% manual testers. Some are 100% developers, maybe with zero testing experience. Others have a combination of manual and technical testing experience.

When working with an existing team, it’s important to know your team’s background and skill sets so that their strengths & skills can be leveraged in the best possible manner.

Source

If an existing team’s QA members are solely manual testers with no technical background, they’ll need a different introduction to automation than developers or SDETs (Software Development Engineers In Test).

They might be interested in coding and should therefore be given proper programming training before delving into automation. This will ensure their long-term success.

On the other hand, a manual tester may not want to code but can still contribute to automated testing activities. In this case, the team may want to consider an automation implementation using BDD (Behavior Driven Development) frameworks like Cucumber so that non-technical team members can chip in and help write test scenarios.

Run Selenium, Cypress, and Appium tests on TestMu AI to scale with the demand of your website and web apps.

Automation should be used to boost efficiency in manual testing:

Examples include using Postman’s test scripts while manually testing APIs, scalable cross browser testing tools like TestMu AI, creating JavaScript bookmarklets for easier form-filling, or custom bashing scripts to generate test data. Automation engineers and developers can utilize automation in creative ways to speed up manual testing!

For a team with a mix of manual and technical testers, I highly recommend using the philosophy of paired programming and partnering up with people on the team that have complementary skills.

If you have someone with a lot of domain knowledge who doesn’t have a coding background and another person with strong technical skills but less subject matter expertise, these two could be an extremely powerful pair!

In any case, the team must take inventory of their skills when laying the foundation for their automation solution. Once this is established, there are other factors to consider before jumping into automation.

There are too many to list (and might vary from team to team or project to project), but here are a few common ones:

• Tech stack: If the technical team members solely program in JavaScript, you likely wouldn’t choose a Python-specific automation framework.
• Technical skill sets: If non-technical team members want to contribute to automated testing, consider a Behavior-Driven Development (BDD) automation solution.
• Application functionality: Mobile app? Web app? API? There are frameworks for everything!
• Understand what you’re automating: Lay out test scenarios, identify critical functionality of the application, user flows – this could help you choose the best-suited test automation tools. For example, if your app has graphs, charts, complex diagrams, etc., that are critical pieces of functionality, you might consider a solution that supports visual UI testing.

The key here is to dedicate time to planning the solution before jumping in. Involving developers and technical leads is beneficial.

Once the team has laid out the groundwork, you’re ready to start automating. I highly recommend becoming familiar with the Test Automation Pyramid concept before you start. I’ll go over some anti-patterns and things not to do below.

Persona 2: Course-correcting To Get Your Automated Testing On The Right Path

Unfortunately, it’s very common for teams to throw together an automated test solution without taking a planned approach to it from the start. As a result, testers end up with unreliable tests in an unmaintainable suite.

In my own experience, I have joined teams that have two or more years invested into an automation framework, only to discover it is not scalable for their needs and scrap the whole thing for a new one.

Source

While I am a proponent of switching out tool sets for new, shiny ones that enable you to work more efficiently, no one wants to discard years of hard work.

If any of these pain points sound familiar, it might be time to look at how you can improve your automation.

• Tests take a long time to execute (several hours or longer).
• Your regression tests are being run on testers’ local machines.
• Tests fail a lot for unknown reasons.
• There is no easy way to run a subset of the full test suite.
• Testers are spending hours upon hours updating tests that break when there are UI changes in the application.
• Your regression suite has hundreds of UI, end-to-end, and user story-based tests.
• Test runs are unreliable and have false positives/false failures.

I’ve experienced each of these at least once! While all of these can cause a lot of pain for a tester, there are solutions. For Agile teams, everything moves fast. The most common problem that I see teams face, however, is time. How do you balance automation work with all the other day-to-day activities going on?

Prove the potential of automation:

If QA is having a hard time getting Product Owners/BAs/leadership on board allocating time to automate, show them a proof-of-concept or demo of how much time can be saved with automating.

Automated testing creates more time for QAs to test how only humans can provide faster feedback to developers and save time during regression. All of this equates to delivering features faster.

Ensure there is visibility into the automated tests:

The entire development team (QA, Devs, Product Owners) should feel some level of ownership in the quality of the product – including automated testing.

I have found it extremely valuable to get the Product Owner’s inputs on critical user scenarios, which I then turned into automated smoke testing run against every pull request, giving the whole team higher confidence that the most important pieces of functionality were always working.

When the team knows what tests are running, they’ll chime in during new feature development and make sure there’s time to automate new tests! Automation and testing effort should also be included in the ‘definition of done’ for a story. Another great way to keep an eye on the automation is to demo framework enhancements or test runs in Sprint Reviews!

Make it useful:

I have seen teams that dedicate hundreds of hours to a huge automation suite, only to run it once a month. Automation is a time investment, and it only pays off if it’s utilized fruitfully.

Ideally, you have automated tests that run against every pull request – even if that is a small subset of smoke tests. Using CI/CD tools enables you to schedule nightly regression runs, trigger executions based on code merge, and more.

Persona 3: Your Automation Is Awesome – How You Can Make It Even Better

Even the most impressive test suites can always be improved! On teams with a mature automation setup, here are a few things I like to tackle with them:

• Cleaning up spec files and focusing on improving the maintainability of the entire automation framework.
• Identifying any areas for flakiness and making them more reliable – every test suite has some weak spots!
• Looking for opportunities to move tests closer to the application code by utilizing API calls or database interactions to take steps out of brittle UI tests.
• If the QA team comprises dedicated automation engineers and dedicated manual testers, blend these roles so that there are no bottlenecks in the testing process.
• Running tests in parallel (be cautious with shared test environments, test accounts, or test data) using a tool like TestMu AI to speed up regression execution exponentially.

Source

Continuously dedicate time to automation:

Do not let automation become ‘set and forget.’ Automation can quickly become outdated and obsolete if the team is not actively maintaining it. When developing new features, look closely at how to ensure test coverage while keeping the maintainability of the test suite intact.

Shift the testing left:

Run automated tests against every PR, after every merge request, and against every release build. The earlier in the development cycle that an issue is caught, the less costly it is to fix.

Read How “Shift-Left” Testing Can Help Your Product Quality?

Always explore new things:

Have you heard of Contract testing? What about Visual UI testing? Automation tools and frameworks are evolving and bringing us innovative ways to speed up testing as we’ve never seen before. It’s up to us, as engineers, to use them to solve problems for the betterment of our teams.

Wrapping it Up!

On Agile teams, it is crucial to have some form of automated testing in place. Without it, testing inevitably becomes a bottleneck to delivering features faster. Manual testing will always have its place in ensuring that software functions well for humans. No tool or software will ever perfectly replace or replicate the human eye and the critical thinking that is such a valuable component of testers’ mentalities. I challenge you to boundary test the current limits of your testing and take your software’s quality assurance to the next level. Your team will thank you for it!