With the release of StormRunner Load 2.3 from Hewlett Packard Enterprise (HPE), Planit went straight to the source to learn how to maximise our use of HPE's cloud-based load testing solution.
In our interview with HPE, learn more about StormRunner Load and its key features, benefits, reporting options, analytics, tips & tricks for testers, customer success stories and useful resources to help you get started.
Q1. What is HPE StormRunner Load and how does it work?
HPE: With its smarter approach to testing, HPE StormRunner Load makes it easy to plan, run, and scale your testing for web and mobile apps. StormRunner Load is a 100 percent cloud-based, load-testing service that supports web and mobile protocols. It is simultaneously simple, smart, and scalable. Getting started is as simple as 1, 2, 3.
With HPE StormRunner Load, you can initiate load testing with three simple steps. Here’s one way to create a script:
- Create a HPE TruClient script
- Design your TruClient script and add it to a load test
- Run your load test
With the self-driving test lab in StormRunner Load, there’s no need to configure load generators. The system does all the work for you in the cloud.
Q2. What are the key benefits of using HPE StormRunner Load?
- A true Software as a Service (SaaS) cloud-based offering. All you need is a web browser.
- A good fit for new Agile, cloud, web and mobile projects.
- Extremely scalable and cost effective.
- Compatible with existing VUGen and TruClient scripts.
- Open architecture-rich integration with third party open-source and proprietary tools.
Q3. What are the main features of HPE StormRunner Load for testers?
- Integration - HPE StormRunner Load has a rich selection of third party tools to integrate with, from continuous integration (CI) servers, such as Jenkins, Hudson, or TeamCity, to AWS CodePipeline, HPE ALM Octane, HPE SiteScope, Dynatrace, New Relic, HPE Network Virtualization (NV) and WebPageTest. This integration gives great flexibility and allows your Agile testing and development teams to run performance tests as part of their builds in an easy, automated manner.
- Ease of use - With the intuitive features of HPE StormRunner Load, anyone on your application delivery team can now perform mobile app or website load tests—including developers, QA specialists, and project managers.
- Speed - With the cloud-based resources of HPE StormRunner Load, you can quickly create and maintain tests to accelerate the velocity of your software development project.
- Scripting - HPE StormRunner Load allows you to leverage existing LoadRunner and Performance Center scripts, created in the best of breed scripting applications HPE VUGen and HPE TruClient, while also supporting third party open source tools such as JMeter and Gatling. In addition, HPE StormRunner Load offers the unique and innovative code based scripting tool TruAPI, a node.js based scripting framework that allows developers and testers to script in their favourite IDE.
Q4. How are test results, analytics and reporting handled in StormRunner Load?
HPE: You can view test data displayed in real time in multiple graphs or from a consolidated report. Online and offline analysis capabilities include performance trending, anomaly detection, and root cause analysis.
- Dashboard - During the test run, you can add widgets from the gallery to the dashboard to display real time data detailing the results of your load test. You can also access the results after the test has completed.
- Widget Gallery - there are a whole host of measurements that can be incorporated, including:
- Client Side Measurements such as Hits per Second or Throughput
- Transaction measurements such as Transaction Response Time [90th percentile]
- Transaction Response time [Average]
- Monitors – CPU
- Used Memory
- Disk Utilisation
- Mobile - Total Consumed Memory
- Total CPU Utilisation Percentage and
- Flex - Average Buffering Time
- RTMP over HTTP / TCP
- Reports - Each test run generates a report providing detailed statistics about that particular run. The results can be shared by either emailing the report, exporting the report to Word and PDF file formats, or using alternatives such as raw data in CSV format.
- Client side breakdown report – This report gives you the ability to view statistics that help measure the quality of the user experience on your application. This includes the HPE NV Insights Report which allows you to improve the performance of your application by identifying potentially problematic issues in your transactions and providing recommendations for optimising network traffic. In addition to this is the WebPage Test Client Breakdown report. This report helps you to measure the quality of the user experience by providing page load timing data.
Q5. When should users consider using StormRunner Load alongside other HPE testing tools?
HPE: HPE StormRunner Load is part of a suite of market-leading HPE performance testing solutions. It extends the capabilities of the widely used HPE LoadRunner and HPE Performance Center products with a cloud-only solution focused on mobile and web testing in Agile development environments. To better leverage your existing investments, your development and test teams can reuse test scripts among the different solutions in the HPE performance testing suite. LoadRunner is typically used for site-specific project based test and Performance Center for enterprise scaled projects tests where the user base is larger in size.
Q6. Can you share some customer success stories using StormRunner Load?
HPE: Some of the most comprehensive success stories using StormRunner Load include:
- A large learning science company and one of the "big three" educational publishers that provides customised educational content, software, and services for pre-K through postgraduate education - Test Digital User Experience under high WW load from browsers and mobile devices. The reason why HPE StormRunner Load won over the competition was because of price point and ramp up time. This now means the expansion of StormRunner Load from Line of Business to Center of Excellence.
- The Line of Business teams for an athletics web application needed a cloud solution for burst testing and long endurance tests. Center of Excellence were using Performance Center on SaaS, but had no influence on Line of Business. The big driver for the adopting StormRunner Load was the existing incumbent service costs. In contrast, StormRunner Load had all the ticks from a usability and integrations capabilities perspective.
- A multinational financial services corporation undertook an analysis of running load tests internally, and the spend including new hardware was easily over $3M. With the use of StormRunner Load, even with operational cost included, the customer received 75% cost savings and immediate time to market, as there was no time spent on purchasing and setting up hardware.
Q7. Do you have some useful tips and tricks for StormRunner Load testers?
- Drag and Drop - You can easily drag and drop script files to be uploaded
- Labels - Use labels to organise scripts in the repository. Labels can be nested in sub-categories. By labelling your scripts, it allows you to filter scripts by specific label or undertake a search
- Additional runtime settings - Edit or view additional runtime settings including Browser cache, HTTP timeout and Additional attributes
- Pacing time - To set pacing time in in milliseconds, enter the time in decimals. For example, 250 milliseconds = 0.250
- Suggested value - If you have run your test more than once, StormRunner Load will display a suggested percentile TRT value. Click the button to use the suggested value
- Multiple selections - Select multiple SLAs at one time and apply one or more actions to the set
Q8. Where can we learn more about StormRunner Load?
HPE: There is an extensive knowledge base when it comes to learning more about StormRunner Load. Visit the:
Special thanks to HPE staff:
Bharath Benoit, Presales Technical Consultant
Priyank Sharma, Presales Technical Consultant
Amish Prajapati, Software Account Executive
Kate Droukman, Senior Sales Engineer