TestMu AI Helps Emburse Reduce Cost By 50% With 20% Reduction In Test Execution Time

TestMu AI’s Solution: Scalable, Cost-Effective Cloud Testing

Emburse chose TestMu AI’s cloud Selenium Grid for its scalability, cost-efficiency, and ability to streamline the testing process. TestMu AI enabled Emburse to scale test execution rapidly and effectively, eliminating the need to manage infrastructure.

Automation testing became indispensable for shorter release cycles as they scaled

While automation testing is part of their Definition of Done in the agile scrum methodology, they were on the lookout for a better-suited automation testing solution that could speed up test execution time and accelerate TTM.

They preferred to be able to use multiple programming languages and frameworks with the Selenium Grid. Since QA would be running E2E automated tests, having a single cloud that can run unit tests would have been an advantage.

Another reason that compelled them to go for a faster test execution was that their business was affected due to the poorly managed infrastructure and legacy platforms.

By using TestMu AI, they were able to perform UI testing seamlessly in the cloud without having the need to maintain an in-house Selenium Grid.

Now, most of Emburse’s testing happens on LambdaTest (now TestMuAI), including close to 2100 regression tests and 190 sanity tests across different categories. Regression testing is usually run every day in different time zones to ensure that new feature implementation has not resulted in any breakages in existing functionality.

Alan Harwood

Senior Manager, Engineering, Emburse

As a part of the Emburse QA automation practice, the types of testing they do include unit testing, API testing, UI testing, and manual testing. While the frameworks and languages used for UI testing include the following:

• Selenium framework

• Selenium Grid

• Robot framework (Python framework)

• Python language

• Jenkins (CI/CD)

While Selenium testing is their primary method for UI testing, Postman is their primary method for API testing. They supplement this with manual testing when automated tests are not applicable.

Their UI regression lifecycle starts with writing tests in the Robot Framework, then committing to GitHub. Jenkins is where they kick off that test run. Next, they use Python to retrieve tests stored in GitHub and convert the Robot Framework keywords into a test that Selenium can understand and use. And then, Selenium WebDriver executes each test using the TestMu AI Selenium Grid.

According to Alan, migration to TestMu AI took minutes. Then, a new Jenkins job was created with the new endpoints, and it was off to the races. This Jenkins job actually made a copy of the Jenkins job for TestMu AI and pointed to the TestMu AI URL and made it pretty simple for them to get up and running. They didn't have to do too much.

So each sprint included a process from Running full regression suite in staging → Deployment to QA → Running sanity suite in QA → Deployment to production → Running sanity suite in production.

They have created different clusters for different geographies and environments as part of the Sanity suite checklist, which helps them quickly validate the results against each category of tests.

They also leverage TestMu AI to run security scans once the code is executed every night. Since Selenium can also be used to support backend use cases, they used Selenium additionally to run security tests.

We run our full Selenium regression suite every night in our lower environments. We run our sanity test suite whenever we have a major deployment in our upper environments. Our API tests run on demand with each repository change.

Alan Harwood

Senior Manager, Engineering, Emburse

TestMu AI’s powerful yet intuitive web-based dashboard allows Alan and his team to run automated tests confidently and faster irrespective of the location, without managing resources internally or requiring any DevOps support.

Enhanced Automation Capabilities Across Web, Mobile, and Native Applications

By leveraging TestMu AI’s advanced automation capabilities, Emburse was able to run multiple parallel sessions for both web and mobile browser automation, as well as native app testing. This comprehensive approach allowed Emburse to efficiently test their entire suite of applications across various environments and devices. The flexibility of running parallel sessions minimized setup time and accelerated the testing process, ensuring consistency and high quality across all platforms.

Eliminating Test Flakiness and Ensuring Consistency

One of the most significant challenges Emburse faced with their in-house grid was test flakiness. Frequent failures due to network issues, disk errors, and high resource consumption made automated testing unreliable and inconsistent. However, with TestMu AI’s cloud-based infrastructure, these issues were resolved, providing Emburse with the reliability they needed to run automated tests frequently and confidently.

With TestMu AI, Emburse could also ensure that their applications were compatible across multiple browsers and operating systems, an essential factor in their testing strategy. The platform’s ability to support cross-browser testing allowed Emburse to test across different environments without needing to maintain multiple test setups.

Powerful Results: Faster Execution and Major Cost Reductions

20% Faster Testing and 50% Savings

Emburse saw a 20% reduction in test execution time, allowing them to run regression test cycles faster and improve Time to Market (TTM). The reduced test cycle times enabled Emburse to focus more on building high-quality test cases by freeing up internal resources, rather than waiting for lengthy test executions.

In addition to faster tests, TestMu AI’s cloud solution enabled Emburse to cut its infrastructure costs by 50%. With $TestMu AI managing the scaling and maintenance of the testing infrastructure, Emburse was no longer burdened with the costs and complexities associated with managing an in-house grid. This cost savings allowed the team to allocate resources more effectively and focus on critical business functions. Moreover, TestMu AI provided the flexibility to run tests from anywhere, at any time, on any device, further enhancing efficiency and adaptability in their testing processes.

**2021: Prod Acceptance: 43 mins (down from 54 mins in 2020 owing to the test execution benefits offered by TestMu AI cloud Grid)

**2021: QA Acceptance: 56 mins (includes time spent on troubleshooting)

What’s Next: Scaling, Expanding, and Embracing AI with TestMu AI

As Emburse continues to grow and scale, they see TestMu AI not only as a tool but as an integral part of their testing strategy. The company has already begun exploring advanced AI-driven testing solutions as part of its future roadmap. Through a trial with KaneAI, TestMu AI’s intelligent automation tool, Emburse gained insights into the potential of AI to further enhance testing efficiency and accuracy. This trial marks the beginning of their exploration into leveraging AI-Native features to optimize test execution and improve the overall quality of their testing processes.

With the potential to expand parallel sessions, integrate AI-driven automation tools, and explore new testing projects, the partnership between Emburse and TestMu AI is set to evolve further. The journey of continuous improvement and strategic collaboration is just beginning, with exciting opportunities for innovation and growth on the horizon.