Skip to main content
 
nz

  • Increase Speed to Market
    Deliver quality quicker by optimising your delivery pipeline, removing bottlenecks, getting faster feedback from customers and iterating quickly.

  • Enhance Customer Experience
    Delight your customers in every digital interaction by optimising system quality and performance to provide a smooth, speedy and seamless user experience.

  • Maximise Your Investment
    Realise a positive ROI sooner and maximise your investment by focusing your energy on high-value features, reducing waste, and finding and fixing defects early.
  • The Wellington City Council (WCC) wanted to deliver quality outcomes without breaking the bank. Find out how Planit’s fast and flexible resources helped WCC achieve this goal.

this is a test Who We Are Landing Page


INSIGHTS / Case Studies

Delivering Mission Critical Performance

 30 Mar 2021 
Delivering Mission Critical Performance Delivering Mission Critical Performance
Delivering Mission Critical Performance
INSIGHTS / Case Studies

Delivering Mission Critical Performance

 30 Mar 2021 

Harnessing performance testing and engineering to unblock key business processes and boost the speed of critical business systems.

Problem

As this corporate services provider expanded their business, they inherited several legacy systems responsible for employee loyalty schemes, fleet management, and critical payroll processing and reconciliation. To streamline their technology footprint and associated costs, it became necessary to consolidate a number of these core business systems onto a single strategic platform.

A large-scale change such as this came with a high level of risk. Therefore, the company wanted to thoroughly benchmark the performance of the new platform to understand its capabilities.

They also encountered a perplexing payroll processing issue, which caused delays in payroll processing and required significant effort to manage, while evaluating the new platform. Solving this issue was crucial before moving onto the new platform, so it was necessary to replicate it in the test environment to understand why it was happening.

However, due to several constraints, including limited monitoring in the production environment, the corporate services provider’s operations team was unable to reliably replicate the problem, understand its impact, and verify any fixes. This situation highlighted a need for ongoing performance testing capability beyond the successful implementation of the platform.

Solution

The corporate services provider needed an external testing partner to assess the performance of the platform, as well as replicate the payroll issue. They selected Planit for the role due to our demonstrable track record and reputation with performance testing, as well as our large portfolio of supporting services that they could draw upon if needed.

After thoroughly analysing the in-scope systems, we concluded that a repeatable, production-like test framework leveraging an open source load test tool, such as Apache Jmeter, should be used to benchmark their current performance. This framework could also be used to test subsequent releases, recreate the payroll file processing issue, and confirm its impact on customers.

A challenge was that the company had limited performance testing experience or results for us to draw upon. This required us to work closely in partnership with them to fully understand the project’s performance requirements, business risks, and technical risks, and then develop a performance testing framework that would meet the key objectives as well as provide a solid base on which to build upon for subsequent releases.

To recreate the existing production issue, we began by carrying out an extensive analysis of the Internet Information Services (IIS) logs in conjunction with client workshops and risk assessment sessions. This enabled us to build a representative workload profile covering more than 30 user journeys across two key business critical applications running on the platform.

The resultant tests were configured to run for ten hours covering 90% of the daily traffic. In addition to closely replicating daily production traffic through the system, these extended load tests were crucial to correctly seed data for the batch performance testing, ensuring that the batch sizes and performance metrics were also comparable to production.

Extensive analysis of available log data was carried out post-execution to verify that the results observed in the test environment mirrored those seen in production. This verification provided confidence that the results observed could be used as a valid benchmark for subsequent performance tests.

Aside from the overall technical complexity and scope of functionality that needed to be accounted for, the main challenge of this project was the sheer amount of data involved. The testing had to take into account payroll processing across multiple customers and thousands of employees.

A well-defined approach to data management was critical to the successful completion of the project. Production logs had to be carefully analysed to identify the correct data, and a robust database backup/restore process was required to enable repeatable tests.

In light of their limited application performance management (APM) or analytics tools, our performance consultants worked diligently to guide the corporate services provider through the workload modelling process and verify that the test coverage was fit for purpose at each key milestone.

Database log analysis was carried out throughout the test design and execution phases to continually validate and compare it with production. The outcome of this analysis was fed back into the preparation activities to ensure that the test coverage and scenario designs were accurate.

Outcome

Our testing successfully benchmarked the performance of corporate services provider’s core system and recreated the production payroll issue. By the conclusion of the project, we had created and executed three Jmeter test plans covering 34 user journeys.

The test results also demonstrated that the payroll processing issue was responsible for the degraded performance across all payroll processing, and not just for a subset of customers. This meant that the company was able to develop the necessary fix to resolve it, saving their finance team countless hours of effort when processing the monthly payroll.

Our consultants delivered all of this by maintaining open communication with the corporate services provider’s project management team, which included notifying them early of challenges that could affect the original delivery timeline. By taking a proactive stance to mitigate issues as they arose, we were able to quickly develop and present solutions to senior management, thereby minimising any disruptions.

Based on the quality of our service, valuable insights, and savings we provided during the initial performance testing engagement, we were then retained by the company as their preferred testing partner to verify the payroll issue fixes and further determine the overall capacity of the platform.

This single engagement has expanded to a trusted partnership beyond performance testing, as we now also deliver functional testing and test automation on a growing scale.

Key Outcomes:
  • 100% increase in coverage in automated performance testing

Delivered:
  • Performance testing
  • Staff Augmentation

Technologies:
  • .NET
  • IIS Server
  • SQL Server

Tools:
  • Atlassian Jira
  • InfluxDB
  • Grafana
  • Apache Jmeter
  • Perfmon
Download Full Case Study
Performance You Need for Today

At Planit, we are experts in quality engineering and assurance. Our performance consultants can provide testing, assessments and advice to mitigate performance risks and achieve peak results.

Find out how we can help you make performance an asset, not a liability!