Taking Your Pipeline from Meh to Awesome: Samantha Connelly [Testμ 2022]

What is Patch in Mercurial?

Samantha then talks about how the patch system works in Mercurial. They further explain that the patch is applied in a queue, and instead of working on a branch, you create a patch and make all your changes there.

Everyone else’s patches will work in your local repository if you’re in the queue. This is great for encouraging regular small commits. If you have a piece higher up in the queue, it’s a great way to get incidental testing done. Also, if there’s anyone below you, they can perform random testing with your changes in place. Thus, conflicts are managed a little differently using this patch system.

Although you cannot check out your patch if you’re working on a file that conflicts with a patch further up, you tend to get fast feedback if someone’s working on the same code.

Samantha shares that they enjoy working on a patch system as it encourages small and regular commits rather than having long-lived feature branches, which can also introduce its challenges from a testing point of view. It makes it hard to trace back specific changes to broken tests, and they think that’s also a challenge using Git repositories as they’re unable to trace broken changes further down the pipeline back to a single code change.

They carry on to showcase the scripts that they have created:

• A bash script to automate some of the regular tasks that we tend to do on a day-to-day basis from the command line, and
• A script to go through every single patch in the queue between releases and perform a Unit test against them.

Samantha then gives a tour of the old Jenkins. They further discuss their bug backlog management system and how it is handy for their testers. They also showcase their test environment using their Jenkins instance in AWS.

UI testing issues

Samantha then goes on to discuss some of the UI testing issues. They insist on how seamless it would be to use a tool like TestMu AI, but it cannot be used with the current framework. However, they mentioned that DataDog synthetic UI tests are helpful as they allow you to monitor different test environments.

What is the future of testing?

Samantha concludes by answering the million-dollar question – What is the future of testing? According to Samantha, the future of testing is neither shift left nor shift right. They insist that the future of testing is just getting more involved, picking up more skills, and adding more value; whether it’s on the business side or the engineering side, it would do wonders. Using the tools and the capabilities to help your teams, whether it’s improving your pipelines or just learning the technology your team is using, is what holds the future of testing.

They further answered the questions put forth by the audience, where Somesh Ojha, Director of Sales at TestMu AI, took the initiative to ask them on the audience’s behalf.

It was indeed an informative session with Samantha! The session ended with a few questions asked by the attendees to Samantha. Here is the Q&A:

• Mobile, today is the primary mode of information communication. But most companies have one large testing team for digital assets. Does it make sense to bifurcate testing teams into browsers, mobiles, etc.?
  Samantha: Dividing teams into different structures is a huge organizational problem. Unfortunately, I can’t answer that question because I prefer to work in smaller teams where they don’t have that.
• You’ve primarily been in the mobile testing role. Can you walk us through some of the big trends you’ve observed?
  Samantha: Web testing, I think, has been more advanced than mobile testing, so basically, the trend I’ve been seeing is that mobile testing has just gone up to speed with what web testing was like two to three years ago.

There are like a gazillion different frameworks, and they all have different issues regarding quality. I’m biased towards working on native applications because you get a lot of quality out of the box for free, like accessibility. To be able to do screenshot testing easily or things along those lines, it’s just the tools evolving.

• What are your thoughts on “Mobile app testing will dominate the future since most of the desktop OS’s are trying to emulate mobile OS”?
  Samantha: I feel the tools and technology have improved much to do a lot of emulation-based testing. You couldn’t rely solely on emulators. You had to do real device testing because you’d find bugs. In the early days of Android development, Samsung still wrote its own custom Android OS. Still, I see fewer Samsung-specific issues because it closely matches the Google version of Android.

So I think the emulation and simulation technology for mobile testing have been exceptional and have become much better over the last couple of years. In the previous few mobile apps I’ve worked on; we didn’t pick up any device-specific bugs anymore. You could get away with doing more emulation and only simulation-based testing.

• Interesting to hear your thoughts on shift left testing. More so because everyone in the world wants to move testing to the Unit/Contract/Integration side of the house vs UI/E2E testing. What are your thoughts on automating E2E tests in that regard?
  Samantha: I think it’s a misnomer because delivering software is not a linear process, and it’s an interesting model, but all models are wrong. We should try to reduce end-to-end automation tests as much as possible. They’re hard to debug, they take a long time to run, and we should be focusing on trying to build out testing and quality earlier on in the dev process because it just helps facilitate that feedback.
• You mentioned you were the first tester on the team with 14 devs. What QA: dev ratio are you targeting, and how challenging has it been starting as the first tester?
  Currently, the ratio is one to fourteen, and I’m not likely to get involved with hiring anytime soon. I was drawn to this role for being able to influence the culture of quality and testing to come in and help drive. I might get involved with hiring later on if we are growing the team. Currently, we have 14 devs, and it’s a single team. We should be trying to split that into two teams, something like a pizza-size team.

My main challenge as the first tester was getting up to speed because it is a complex product, and the founders have been working on it for ten years. There’s a lot of implicit knowledge in other people’s heads, and they’re kind of busy, and I always feel like a burden to go and ask them if they can help me figure out how to do some of this Maven stuff which sounds like it should be pretty straightforward. So, my biggest challenge has been finding time to try and get up to speed with other team members.

After the successful Testμ Conference 2022, where thousands of testers, QA professionals, and developers worldwide joined together to discuss on future of testing.

Join the testing revolution at TestMu AI Testμ Conference 2023. Register now!” – The testing revolution is happening, and you don’t want to be left behind. Join us at TestMu AI Testμ Conference 2023 and learn how to stay ahead of the curve. Register now and be a part of the revolution.