How to plan a successful quality assurance (QA) strategy

Introduction

In this blog we will be travelling on a journey to learn what a QA strategy is, why it should matter to you, what should be included in your QA strategy and how you can go about implementing it within your organisation.

I have worked in many organisations where some planning has meant that the team all steers in the same direction, ultimately this means happier customers!

What is a QA strategy?

A QA strategy is a document that highlights the quality standards that a software product should adhere to. This document is key to ensuring that all stakeholders are aligned on the quality assurance approach.

In the past I have created this document in such a way that terminology and aspects could be understood by technical and non-technical experts.

A successful quality strategy is not static; it’s a dynamic framework that evolves as project demands and technology change. One of the key aspects of maintaining a robust QA strategy is understanding the importance of regular reviews and updates. But how often should these reviews occur?

To gather industry insights on this matter, we conducted a social media poll asking professionals “How often should a Quality Assurance (QA) strategy be reviewed and updated?” The responses provided a clear preference, suggesting that a proactive approach to QA is top of mind for many in the field.

The majority of respondents believe that a QA strategy should be reviewed quarterly. This frequent reassessment allows teams to stay agile, adapting to new challenges and incorporating fresh feedback from the development lifecycle. It reflects a commitment to quality that resonates with the agile principles of iterative improvement.

Why do you need a QA strategy?

The key reason as to why it should be used is that it allows you to have a consistent set of tools, tests, roles, and expectations. This will ensure that all team members are aligned on the objective and they will have a shared vision.

The second reason is that you have a clear organizational standard and best practices. This enforces rules on the approach. For instance the best practice for writing test cases may be on a standardised tool that you are using.

One aspect that I found was that over time the QA strategy aligned all team members, invited questions and it prevented defects from being delivered in a client release. The approach has a key objective to ensure that we do not have any escaped defects.

Project managers were also big fans of the QA strategy as it allowed organizations to plan schedules, releases, and level of effort for a release. A clear strategy can be estimated more easily, and a project manager will be able to assign a level of effort more readily.

Some four key value adds that I found as a ‘Head of QA’ were:

• Functional and non-functional testing were applied as standards. Functional testing encompasses regression testing and testing key areas which were requested in customer requirements. Non-functional testing encompasses areas such as: security, performance, accessibility testing.
• Enhancement of our reputation as the quality of the product is improved.
• Reduced costs as fewer defects in the released product.
• Allowed us to get ahead of your customers and iterate on the product. If there is a clear regression test pack you are able to iterate on the product more readily and make more changes.

Note: You can use TestMu AI Cloud to test on a variety of simulators, emulators and real devices. No need to install anything, just sign up for a free account and start testing. Try TestMu AI Today!

What are the key components of a QA strategy?

When you are creating a QA strategy from scratch it can feel daunting, lets go through some key aspects to include and work through it:

• Let’s add the mission statement, this is what you are looking to achieve and what are the values. An example of a good mission statement for a mobile app is: ‘To empower businesses via our mobile app to help gain a competitive edge by providing simple yet innovative customer focused technology solutions.
• Devise your objectives and goals. These are what the customer needs and expectations are. They must be SMART:
• Specific – The goal you set should be specific, and you shouldn’t be able to misinterpret or confuse it.
• Measurable – The goal should allow you to track your progress
• Achievable – The goal needs to be realistic
• Relevant – A relevant goal relates to your values, dreams, and ambitions
• Time-bound – There needs to be a target date for completion, such as four months or one year

An example is:

SMART goal for running a marathon

Specific: I’d like to start training every day to run a marathon.
• Measurable: I will use my Apple Watch to track my training progress as my mileage increases.
• Attainable: I’ve already run a half-marathon this year, so I have a solid base-fitness level.
• Relevant: I value my health and wellness, and this goal will help me sustain that.
• Time-bound: The marathon is a year away, so I need to be ready by then.
• Acceptance criteria – confirm what requirements we are delivering against

An example of good acceptance criteria is:

As a user, I want to be able to recover the password to my account, so that I will be able to access my account in case I forgot the password.

• Roles and responsibilities – this should answer the following:
• Who is doing each task
• What skill set do we have in the team
• Deadlines for each of the tasks
• Gather progress

For example:

Test Analyst will do the QA, the skillset is manual testing and attention to detail, they need to write their test cases, run their tests within the current week and progress will be tracked in JIRA.

• Critical success features and measures – this is key as you should be tracking success of your QA strategy not just at the end of a project, but at regular intervals throughout the process.

Absolute QA testing metrics examples:

• Total number of test cases
• Number of passed test cases
• Number of failed test cases
• Number of bugs detected after releases

Defined QA testing metrics examples:

• Test effort – such as tests per bug = Total number of defects / total number of tests
• Test effectiveness
• Test coverage percentage = (Number of test cases covering the function / Total number of tests) x 100
• Testing approach – the testing approach is key as it details the ‘how’. In the next session we will go through what should be covered in a test approach so that you can create one yourself.

Note: Learn why Software Quality Assurance matters and how it can benefit your projects.

What should be in a test approach?

The key aspects that should be covered in a test approach should be well thought out and the document is interactive, therefore it should be kept updated.

Firstly it should be clear to detail the purpose and project overview – this is a clear description on the scope of work and what value it is adding. For example if we were adding a new website for selling our apples we could say: The purpose is to create a website which will allow customers to buy apples so that they can be delivered to their door.

Then we would detail what aspects are out-of-scope. This is really important as it is then clear to all stakeholders what will not be tested. In the past I have found having this section explained has meant that stakeholders have said ‘we need to test this, because of this…’

Now onto the ‘how’ – we would detail at a high level what type of testing we will do in what order. For instance using my website example to sell apples the types of testing phases would be: Unit testing > Functional testing > Regression testing > Automation testing > Accessibility testing > Performance testing.

We would then explain the test environment considerations, this is key as if we decide to use another QA it should be clear to them what test environment considerations are in scope. At the same time if the test environment needs to be configured, this section will allow them to do this. Examples of questions to answer are: What physical environments do you have? What interfaces are relevant? Are we using any test emulators? Do we need to test on any specific devices?

Another important consideration to run a test would be the test data requirements, in the past this has been very key to ensuring that we have realistic data which is used by a customer. The types of questions to ask are: Do we need to refresh the test data? Do we need to create any data from scratch? Should the data meet any regulatory requirements.

In the next session describe your testing tools and what they are used for. For example: JIRA is used to track bugs/defects and is also used to track test cases.

We would then need to describe the test phases in detail, what they are, who will be doing it and what is the exit criteria. As an example: Regression testing is ensuring changes we make do not break previously working functionality, this will be done by Adam who is a QA analyst and the exit criteria is: after running our 50 regression tests we would like to ensure that we have no critical defects.

The timescales are key, especially for stakeholders and the project manager. It would list out end dates for each of the testing phases.

Now to the last sections which would start with a distribution list on who should be updated against details listed in the test approach – this may be your project team which includes: Test Manager, Engineering Manager, Business analyst, project manager.

The last section is an audit log on who should have input and approve the testing approach. The distribution list members would generally be listed as sign-offs of the document: Test Manager, Engineering Manager, Business analyst, project manager.

How can we start to implement a QA strategy?

From experience, it was key to understand the business goals and ensure that the QA strategy is aligned with the business.

The first big learning is that you should input time to establish your quality goals, these goals should be SMART, detailing what you would like to achieve and what success looks like.

Another key learning is that building a strong QA team is underpinned by a focus on the roles and responsibilities. If the team knows what they need to do there is more chance of success.

The last bonus point that I would add is conducting a retrospective. By doing regular quality audits you will be able to correct and learn from your mistakes. As with anything it life it is good to learn.

Now that you have the QA strategy document defined it should be seen as a ‘live’ document, which is updated on a weekly basis.

Summary

The QA strategy should be at the heart of all that you do, with a strong emphasis on Software Quality Assurance. It is most effective when it is a live document that is updated weekly to ensure the highest standards are maintained.