Testers nowadays are encouraged to possess multiple skills beyond the core skill of functional testing. With organisations moving towards autonomous teams, there is also a push to upskill team members to be more "T-shaped" - where the vertical bar on the T indicates depth of knowledge, and the horizontal one indicates breadth.
So how does one accelerate performance testing in autonomous squads, who may have very minimal or no experience in delivering end-to-end performance testing?
Traditional Teams vs Tribe Model Autonomous Squads
In traditional teams' organisations put together a "Testing Centre of Excellence" (TCOE) teams composed of testers specialising in testing types such as performance, automation or service virtualization. When a need for performance testing arises, an individual team will approach the TCOE for a specialist to complete the end-to-end testing.
In a Tribe Model, autonomous squads are teams that work together on specific aspects of the product. They should possess all the skills and tools needed to design, develop, test and release to production. If a performance test requirement arises, they should be able to script, execute and analyse the results with minimal support.
Approach for autonomy
Migrating squads towards autonomy with respect to performance testing is something that happens gradually over a period. Therefore, a plan is needed to carry out all the groundwork before effective automation can be achieved.
Here is a five-step approach:
1. Identify the performance test coach
The coach will be the primary person driving this initiative, so making the right choice is important. Usually it’s the responsibility of the test chapter lead to recruit the coach who will then be responsible for building the strategy for the tribe, design templates and/or standards, and drive the upskilling plan. This needs to be an experience performance engineer / tester who can mentor the team and explain performance testing to the rest of the business when required.
Also, the coach should lead the following:
- Performance risk analysis
- Test design and Code reviews
- Investigating tribe wide issues like proxy, firewall or network errors
- Results presentation, beyond reporting of transactions pass / fail & response times
- Resolving disputes when squads are not able to explain it to the business
2. Test strategy for the tribe
The Performance Test coach should start analysing the applications that are supported by the Tribe and come up with a Test strategy that consists of:
- Detailing the performance test approach
- Identifying suitable tools
- Environment setup (Cloud, in-house, or both)
- Guild formation (across tribes)
3. Design templates and define standards
This is where you define document templates for the performance test plan, workload mix, and test report, as well as tool templates for scripting and analysis. Estimation guidelines are also created.
4. Upskilling plan
Team members are split into three categories (beginner, intermediate, and advanced). A training program for each category is then designed, with a target date set to monitor progress. The Performance Test coach should work closely with individual team members in the squad and support them when required.
5. Best practice sharing
With access to the right test tools, and the guidance to implement them, squad members should:
- Start coaching new team members
- Share ideas in guild meetings
- Identify opportunities for continuous improvement
Process to accelerate performance testing
Once the right approach has been defined, the performance test coach can improve performance testing by:
1. Creating simple script templates
Most modern tools can be used to create script templates for performance testing. The test coach should design simple ones which can be easily inherited by the squads to:
- Reduce the time to create a new script
- Maintain standards and best practices
- Provide inline comments regarding each line of code
- Review easier (due to minimal changes)
- Share a common set of test data across teams
2. Adding scripts to a repository
Performance test scripts should be included as part of the tribe’s source code repository, along with the application and automation code.
The benefit of this is that it provides both developers and testers with access to the code. It also helps in maintaining versioning, and integrates well with continuous integration (CI).
3. Including performance test execution in the CI pipeline
It's not enough to just performance test the code. It should also be in the CI pipeline so every change in the code results in the tests getting re-executed.
Consider that:
- Most modern CI tools now include plugins to execute performance tests.
- The Shift Left approach helps to identify problems earlier in the lifecycle.
- If the development environment is stable, executing test after every change provides feedback at an early stage in development.
4. Generate trend reports
Trend reports compare performance test results over time, which provides better visibility into an application's performance. Most tools provide automatic trending, so every test run can be included in trending without any manual intervention.
Understanding and interpreting the results is easy, to the point where even a novice user can do it, which helps to eliminate dependencies. Trend reports can also be used during regression execution rather than relying on detailed analysis reports.
5. Result analysis
The test coach should make reporting easy by creating templates that can be extended by the squads. These should:
- Showcase only priority graphs
- Be customised to the organisation’s requirements
- Be easy to understand
- Maintain standards and best practices
Getting up to speed
With the support of the chapter lead, I had the opportunity to implement the above changes within a large financial organisation. Two dozen of their functional and automation testers, who started off with limited knowledge in performance testing, can now do it end-to-end for their squads – a few even went on to become experts in more advanced fields like performance engineering.
Are you not realising the positive benefits of performance testing? We are offering free consultations to organisations who are worried about the ability of their applications to cope with today’s changing workloads.
Just visit our Performance Risk Advice page to book a 1-to-1 session with us to understand what steps you can take to minimise your risk, and ensure new and existing systems can keep up with demand.