In the first entry of a five-part series outlining 5 Steps to Ensure Performance, Technical Solutions Director, Joel Deutscher explains how to bypass performance issues in production and create applications which run effectively. Learn how to establish sufficient Non-Functional Requirements at the start of your project lifecycle with these insights.
Step 1 – Know What You Want
It may seem straightforward, but we have found that insufficient Non-Functional Requirements (NFR) are the single biggest cause of performance issues in production. The good news is that it's completely avoidable, and here are our tips to ensuring that you get the application you want.
So why is NFR so hard to get right? In our research we have found that performance is not well understood and often neglected due to one of the following reasons:
- Assumed or considered the "Of course" stuff
- Considered but not well defined (e.g. "I want it to be fast")
- Considered but of no actual benefit (e.g. "Must not be slower than the existing platform")
- Considered but pointless (e.g. "I want it as fast as the industry average")
- Or worst of all, they are not considered at all
How can we solve this? Timing! It's important to decide how fast your application is before it's scoped and costed. Unfortunately, statements like "As fast as industry standard" or "No worse than existing" are not good enough to ensure you end up with a high performing application. You need to be specific. Let's start by looking at a typical NFR for a website login:
The application login must be faster than 3 seconds.
While this isn't a great requirement, at least we have a starting point! We are unfortunately missing enough information to allow our vendor to deliver software that isn't what we expected. Some of the questions that remain with this type of requirement are:
- Is it acceptable if its 3 seconds with 1 user and gets slower as more users are on the website?
- Do 100% of logins have to be under 3 seconds?
- Where should this be measured? Is it at the webserver or at the client device?
- What is an acceptable error rate if any?
These questions throw enough uncertainty to allow your application to be built in a way that can't meet the 3 second requirement for login under heavy load. So what should a good NFR look like? Unfortunately, there is no silver bullet here, though it's important to populate as much information into your requirements as possible. Our example written above could be better written to remove ambiguity as:
The application login must be faster than 3 seconds 95% of the time, with a maximum response time of 5 seconds measured at the webserver when 1000 logins are performed over a 1-hour period.
Poor requirements are not unique to performance testing, and some major improvements have been seen in functional requirements using the Behaviour-Driven Development approach and its supporting Given-When-Then syntax. This same approach can also be taken to specifying performance requirements. Our login example above can be rewritten to make it easier to read using this syntax.
Scenario: Application Login
Given I am on the login page
When I submit valid authentication details 1000 times in an hour
Then I should see the message "Login Successful"
And the response should be returned within 3 seconds 95 percent of the time
And the response should not ever take more than 5 seconds to return
And the error rate should be less than 1%
Regardless of your approach, it's about finding a method that works with your teams and your development lifecycle. While it may seem obvious, making sure your requirements are considered from the beginning is the only way your teams or vendors can scope, design, build, test and deliver for what you actually want.
Step 2 – Own Your Performance →