Accelerating Software Delivery through Test Automation
The Royal Automobile Club of Western Australia (RAC WA) is a motoring club and mutual organisation, offering motoring services and advice, insurance, travel services, finance, driver training and exclusive benefits for their members. As the independent voice for road users in WA, RAC is concerned with all aspects of road and community safety. The RAC Group comprises the Royal Automobile Club of WA (Incorporated), RAC Insurance Pty Limited, RAC Travel Services Pty Ltd, RAC Finance Ltd and RAC Security Services Pty Ltd.
RAC Insurance (hereafter RACI) had historically performed manual regression testing on their customer-facing B2C web portal for each delivery cycle. As the complexity of their regression pack increased and delivery cycle times decreased, the RACI test team needed to find a way of reducing the time required for regression testing.
Automation had been trialled previously but unsuccessfully by RACI to implement regression testing of B2C. RACI engaged Planit, who recommended Tosca based on their knowledge of RACI’s systems and delivered a successful PoC, demonstrating that Tosca could be used to automate testing of the B2C system.
B2C and its internal counterpart, PCM, are both built upon .NET MVC (web) technology stack, with an Oracle database backend - all of which are perfectly supported by chosen automation tool, Tosca.
The project involved the following:
- Analyse manual test cases
- Understand system interoperability with other systems
- Prepare valid test data
- Setup test environment
- And finally, create automation framework for 20 test cases
To develop an automated regression capability for RACI to reduce the overheads associated with carrying out manual regression testing for each release cycle of B2C. Scope was set for 20 test cases to be carried out in 30 business days, as well as ensuring stable operating environment built upon a self-provisioning test data strategy, a robust platform with repeatable and verifiable results and a low-complexity framework easy to operate by non-technical resources.
At the end of the engagement, Planit exceeded the expectations by covering 66 test cases fully automated, instead of the original 20. It was achieved by analysing existing manual test cases and utilising Tosca Test Case Design capabilities to generate test cases from a data driven table of possible permutations. This is a perfect example of leveraging advanced tool capabilities to deliver extra value to the client.
Additionally, the following activities were performed:
Shield is a core product upon which B2C and PCM are built. Arnie is a legacy system for transferring information to endpoint clients. Both were subjected to feasibility testing with Tosca, to ascertain whether future automation efforts would be applicable for scope increase.
- Continuous Integration
Tosca Execution agents were deployed for uninterrupted, unattended and out of working hours testing, leading to a maximisation of environment use and reduction of overall release time. Execution of Automated Test Suites was incorporated into the build process. With this approach effortless test results were made available instantly before a new release deployment was scheduled.
- Automation Processes
Automation testing is a discipline that bridges testing and development. Therefore, it must be under same scrutiny for process and quality as any other dev project. To that end, guidelines and firm process were established and handed over, to ensure maximum effectiveness all while maintaining technological excellence.
Charlotte Trembecki, Test Manager of RACI said:
“I just wanted to let you know how successful our POC for Tosca has been.
This is now run for every release and since the POC has been done (3 releases) a saving of 1 and half man days. That being said, it has been run more regularly than that but that is because we can now, so the online team have used it to supplement their iteration testing.
We have also added to our library since the POC, adding an additional 20 test cases as well as to our library. This has been done by our in house Senior Test Analyst who, prior to the POC, had no experience with any automation tools. He has picked up the TOSCA suite very quickly with the assistance of Planit and, whilst he still has to continue with meeting project deadlines, he has been able to maintain and add to our TOSCA suite.
It is fair to say in the three months we have had TOSCA we are very happy.”