QA Transformation Journey

What should you do?

As a Head of QA, and Test Manager in the past, I have had experience in these situations before. I will now take you on a journey of change… grab a nice drink and come on the journey with me.

Step 1: Investigate

The first step is to hold independent interviews with multiple team members to understand the problem at hand.

I would recommend asking open-ended questions like:

• What is the reason in your view for the incidents and customer complaints?
• How would you describe the culture?
• If asking questions outside the QA team:
• How would you describe the QA approach?
• How do you collaborate with the QA team?

Once the inspector gadget phase is over, we must now digest the information. I would recommend independent chats or surveys with QA, product, engineering, delivery, and customers (if possible). These chats should not be biased in any way and it should be shared by the interviewee that chats are confidential, with no names or details being shared outwards.

Step 2: Data analysis

Based on the data gathered, this data should be analyzed to then decide where to focus your attention.

For example, in my experience data can be generally focused on the following:

• Cultural issues
• Lack of ‘Assurance’
• Poor product knowledge
• Lack of collaboration between Product, Engineering, and QA

Step 3: What to do?

The next aspect is to decide what to do. I will explain some examples of what I did for each scenario, with additional examples.

1. Cultural issues
   1. Poor work-life balance

      Start to track hours worked and establish why working hours are so long. People MUST have a work-life balance. In the past, this has been due to poor estimation in sprints and also the fact that a change in mindset is needed by senior management.

   2. People are not developing

      People MUST be developing, a successful company is built around its people. I would recommend ensuring that there is a career development plan in place for each member and also ensuring there are training services available via the learning and development team.

2. Lack of assurance

   This generally can be a standard issue across most companies. Yes, we want to work in an agile way and release at pace, but at the same time, we must have some of the basics in place so that we know that we are testing the right things. If the basics are not in place it generally will lead to incidents and customer complaints. Key assurance documents that I would recommend:

   1. QA strategy – aligned to your technology strategy, what are you testing and why?
   2. Testing approach – what is your particular approach to testing on a particular project/squad?
   3. General overview on what you are testing and why? This enables collaboration with engineering, product, and stakeholders to review gaps in test cases.
3. Poor product knowledge

   In this case, without a great relationship with the product team it may mean that we are testing the wrong aspects, also that incidents/customer issues are increasing – which is bad! How can you improve your product knowledge:

   1. Get the product team to review and contribute to the test coverage in the test cases.
   2. Get the product team to review and contribute to the acceptance criteria on tickets.
   3. Meet regularly with the product team and ensure that product knowledge is shared and updated.
4. Poor collaboration with Product and Engineering

   This is a complex one, though firstly there should be a walkthrough between QA and Engineering once the code has been developed (shift left). From a product perspective, they should be reviewing test cases and sharing gaps in the test cases.

Step 4 – How do we maintain success?

We should track whether our mitigating measures have made a difference or not. Going back to our problem statement, we wanted to see whether we can reduce customer incidents and also customer complaints.

I would recommend a full post-mortem review as learning when incidents take place and reviewing whether the incident amount is going down. If they are not, do another review independently to establish the root of the issues.

Tracking success can only be done if we start to track certain metrics.

Aspects that I have found useful to track:

• Escaped defects
• Customer complaints
• Test cases that have been added to your regression suite every sprint
• Unit test coverage
• Number of times a ticket is re-tested and defects found against it