Panel Discussion: Removing Interrupts From End-to-End Test Automation [Testμ 2023]

The Evolution of End-to-End Test Automation

Mohit Juneja then gave a brief overview of the event. The much-anticipated panel discussion is centered around the theme of Removing Interrupts from End-to-End Test Automation Workflows. Interruptions in software testing workflows can lead to inefficiencies, and bottlenecks and hinder the agility required for ensuring high-quality release cycles.

Ensuring testers maintain their flow is of paramount importance. Organizations striving for swift, high-quality release cycles are poised to outshine their competitors, making the need for seamless testing workflows even more critical. Our esteemed panelists are here to share their insights, experiences, and strategies for addressing this challenge head-on.

Mohit now asks panelists to share their insights on how end-to-end test automation has evolved over the past decade, highlighting pivotal changes and trends that have shaped the current landscape.

Richard Clarke: The last decade has brought about significant shifts in the testing landscape. The transition from monolithic applications to microservices has been a game-changer. This shift fueled the adoption of continuous delivery practices and the need for testing to integrate seamlessly within the development pipeline.

Testing is no longer a separate step but an integral part of the process. This awareness has led to more mature testing practices among users. While a decade ago, end-to-end testing focused on vertical components like UI, APIs, databases, etc., but recent years have shifted towards horizontal end-to-end testing.

Businesses now seek to automate APIs across different systems, testing complete business processes. Leveraging test orchestration and execution platforms like TestMu AI has enabled us to execute test cases in parallel, fostering Agile testing within CI/CD pipelines.

Tamar Schapira: Building upon Richard’s insights, Tamar emphasizes the significant transformation in digital accessibility. Over the last decade, accessibility testing has become essential to the testing process. Ensuring inclusivity and compliance has become a crucial focus.

Organizations realize that accessibility testing must be integrated seamlessly throughout the development lifecycle, not just as an add-on. This change parallels the shift toward earlier testing stages and iterative feedback loops. The industry is moving towards integrating accessibility as a core testing component, with tools and strategies continually advancing to meet these demands.

Addressing Challenges at Scale: Overcoming Automation Bottlenecks

Mohit asks panelists to share their viewpoints on addressing automation bottlenecks in large-scale complex organizations.

For a quick visual walkthrough of this concept, watch the video below from TestMu AI.

Subscribe to the TestMu AI YouTube channel for more tutorials on testing and automation.

Artem Golubev: Scaling automation is paramount in the modern landscape. While automation has existed for years, only a fraction of test cases are automated. To overcome this, we must shift focus from implementation details to user-centric testing. Emerging technologies like generative AI are revolutionizing automation scalability. With automation’s vital role in security, organizations must strive for effective scalability to stay ahead.

Nishan Joseph: Scalability hinges on three key aspects: technology, process, and people. From a technological standpoint, embracing an integrated approach to testing diverse systems is essential. Process-wise, managing change and dependencies becomes a critical challenge. And finally, people with varying skill sets and roles must collaborate effectively. Balancing these factors enables organizations to scale automation successfully.

Coty Rosenblath: In large organizations, the challenges at scale often revolve around interlocking teams, skill sets, and unexpected corporate initiatives. Collaboration among diverse teams is key. Equipping teams with modern skills, as well as addressing unforeseen demands, requires a holistic approach. Encouraging a culture of continuous learning and adapting to change is vital in navigating these challenges.

Strategic Tool Integration: Navigating the Tooling Landscape

Mohit Juneja then asked panelists about transitioning to the tooling landscape and how their organizations approach strategic tool integration. Also, how do they ensure a broader perspective while allowing developers and testers the flexibility to choose diverse tool sets.

Here are the views on the strategic tool integration presented by the panelists.

Coty: At the low end of the customer spectrum, we have a suite of tools – a test automation development platform, a reporting platform, and an execution platform – designed to work together seamlessly. These tools are built together and are tightly integrated for a smooth user experience.

However, customers might have chosen other tools to perform similar processes even within their user base. This is even more evident in enterprise environments where different tools are often used. To address this, an organization should adhere to formal or community-driven standards. They aim to create an environment where other tools can easily integrate into their system.

Coty also highlights the significance of community involvement and collaboration between organizations and their communities. They emphasize leveraging the broader ecosystem of tools and solutions their community and other organizations are developing. This approach allows them to stay responsive to industry developments and incorporate those advancements into their offerings.

Mav: A significant portion of our discussions revolves around the concept of open versus closed ecosystems. This essentially boils down to a vendor’s decision – they can opt for a closed ecosystem approach, where they aim to take control of the entire technology stack, necessitating consolidation on their offerings. While this is a legitimate strategy, personally, I don’t see it as the most effective one. I believe many of us share this sentiment, especially when dealing with large enterprises. It’s unrealistic to expect widespread consolidation on a single platform at that scale.

As a vendor, my perspective is aligned with the idea of working within existing systems and technologies. Rather than imposing a rigid “my way or the highway” approach, I find it more productive to integrate with what’s already in place. Our success comes from understanding the challenges you face and providing solutions that simplify your experience. Requiring you to uproot established tools and processes across numerous development teams just to adopt our seems counterproductive and difficult to achieve.

Our role as vendors should be to facilitate out-of-the-box integrations. I agree with Cody’s point about the significance of community content. Our focus should be on creating extensions that are easy to implement. This involves having open APIs and well-documented platforms, ensuring that each individual on this call can excel in their role without being constrained by vendor lock-in or forced transformations.

Ultimately, the developers and testers working on the ground want to focus on their tasks and achieve success in their roles. As vendors, our responsibility is to empower them rather than hinder their progress. Requiring them to undergo major overhauls can impede their success, whereas providing adaptable solutions can truly enable them to thrive.

Tamar: We understand the importance of striking a balance between a closed and open ecosystem. We offer integrated platforms that work seamlessly together for streamlined testing. However, we also recognize the diversity of tools organizations already use. Hence, we encourage adherence to formal and community-driven standards, enabling tools to plug into our ecosystem. This approach empowers flexibility without sacrificing a broader strategic view.

Richard Clark: I agree with Tamar. An open ecosystem is paramount. The industry has moved beyond one-size-fits-all solutions. Our tools are built to work together, creating a cohesive experience. But we also embrace existing tools and standards, allowing for integration that respects the diversity of tools in the market. Collaboration within the community and leveraging external tools aligns with our approach, ensuring a comprehensive tooling landscape.

Innovation Through Adversity: Driving Test Automation Evolution

In the dynamic landscape of software development, innovation isn’t born from smooth sailing but from overcoming challenges—these challenges often pave the way for groundbreaking solutions that reshape the industry. Mohit asked panelists to share a time when their organization faced a significant challenge in test automation and how it drove them to innovate or pivot their strategy to create something usable out there in the world.

Richard shared an example involving parallel testing enhancement, focusing on accessibility and the use of Selenium grids. The traditional setup fell short of meeting customer needs, lacking in performance, security, and consistency. This prompted a strategic pivot towards collaboration with test orchestration and execution platforms like LambdaTest. By leveraging LambdaTest’s infrastructure and focusing on their core strengths, Richard’s organization was able to create an enhanced product in a remarkably short time.

Nishan then emphasized the transformative power of LambdaTest in addressing challenges. He highlighted that challenges are omnipresent, but the pivotal factor is how you adapt and evolve. LambdaTest’s superior performance, flexibility, and support made them stand out as an ally. It empowered his team to overcome challenges by providing quick results and efficient solutions.

Tamar, a strong advocate for accessibility testing, emphasized that adversity has led her to envision a future where accessibility testing becomes an integral part of the testing process. She stressed that it’s not an isolated aspect but should be woven into the fabric of testing itself. Tamar believes that the next generation of leaders must recognize that accessibility testing isn’t a mere interruption but an enhancement to user-friendliness and efficiency.

Coty brought the focus back to the essence of continuous improvement and intentional learning. He encouraged the audience to think beyond automation’s initial adoption and consider the evolution of tools. In his view, AI will play a pivotal role, supporting everything from generating unit tests to predicting and responding to system changes. His message was clear: don’t stagnate, be curious, and adapt to the ever-changing landscape.

Artem reinforced the importance of understanding customers beyond their technical needs, recognizing cultural values and mutual respect. By acknowledging the limitations of products and being candid with customers, long-term partnerships can be fostered, enabling organizations to adapt and thrive.

Guiding Principles for Future Leaders in Test Automation

The conversation shifted to a discussion of the future of end-to-end test automation over the next 5-10 years. The panelists envisioned a landscape driven by AI, generative capabilities, and continuous improvement. They anticipated a shift from scripting-centric processes to high-level direction and interaction with the system, enabled by AI. They anticipated significant advancements in accessibility testing and a seamless integration of generative AI to enhance performance and efficiency.

The panelists concluded the discussion by sharing guiding principles for the next generation of leaders in test automation. The message was unified: collaboration, empathy, and continuous learning are key. They urged future leaders to embrace change, focus on delivering value, and cultivate long-term partnerships with customers. The conversation demonstrated that test automation’s evolution isn’t just about technology—it’s about understanding people, values, and the unending pursuit of better solutions.

As the software industry continues to evolve, the stories and insights shared by these industry leaders serve as a beacon for the next generation. Adversity isn’t a roadblock—it’s a catalyst for innovation. As we navigate the ever-changing landscape of test automation, let us remember the importance of collaboration, empathy, and continuous learning. With these principles as our compass, we can build a future that’s not only technologically advanced but also focused on delivering genuine value to users.

Have questions on how to remove interrupts from end-to-end automation? Feel free to drop them at the TestMu AI Community.

Testing web or mobile apps without writing code? KaneAI by TestMu AI turns plain-English instructions into tests.

To run your first test, start free, or

book a demo

for a full walkthrough of the product.