Dynamic. Complex. Essential. Those are three words that could aptly describe the digital landscape, where opportunities are substantial and challenges are many. Platforms and devices are ever-evolving while operating systems and browsers are constantly updating and changing.

Our prime concern used to be compatibility with Internet Explorer, but this has evolved very quickly into a fragmented market. The challenge now is to support your expanded digital portfolio that includes current applications, devices and operating systems, as well as new versions and iterations that come to market more regularly.

Then there’s the need for speed for you to respond quickly to market changes to ensure your internal stakeholder requirements are met and a fantastic customer experience is delivered. This is no more obvious than in the mobile device space.

Currently, there are at least three different versions of iOS and seven of Android, with one commanding a larger market share than the other depending on the country or region. A growing number of smartphones are also accessing further connected devices with Internet of Things (IoT).

An app may have already been tested on the available devices, screen resolutions and sizes to ensure it supports your customers’ needs. The question then is, how do you stay up to date?

A fragmented landscape

In 2017 alone, there were many releases for operating systems and browsers containing changes that need to be tracked to ensure they don’t impact your production portfolio. For example, iOS 10 had four updates while Edge browser had 15.

New devices are also coming to the market all the time. In 2016, Samsung released 31 different devices while LG released 19, with many different resolutions, screen sizes and functionality that must be considered.

Research by Homestead found that 68 per cent of users will leave a website due to an unintuitive user interface, which highlights the important of regularly maintaining and testing your platform after release. Additionally, Dimensional Research discovered that 53 per cent of users will uninstall a mobile app that crashes, freezes or has errors, while 80 per cent will attempt to use a problematic mobile app just three times or less.

From a performance perspective, DoubleClick by Google discovered that 53 per cent of mobile site visits are abandoned if it takes over three seconds to load. Dimensional Research also saw that 49 per cent of users expect a mobile app to respond in two seconds or less, while 36 per cent stopped using a mobile app due to heavy battery usage, once again highlighting that performance is at the forefront of a user’s mind.

With Gartner predicting that 70 per cent of software interactions will occur on mobile devices by 2022, apps will only grow in usage and importance. A case study conducted by the Commonwealth Bank in Australia backs up this forecast, with 53 per cent of transactions already being made via digital devices and 80 per cent of logins made via mobile devices.

If you have applications and websites that have not been updated recently, there are several factors to consider. This includes the business’s demands and your customers’ lofty expectations, keeping pace with what they require.

You also need to be aware of what new technologies are being brought out that could potentially be used to support your digital portfolio. Additionally, you must track the evolving digital landscape of platforms, operating systems, browsers and devices.

Adopting the right approach

Organisation who deliver their business through digital platforms will need to have a formal approach to mitigating the impact of external changes that are not under their control. To that end, there are three approaches to assuring a product: production monitoring, proactive testing in production, and pre-release beta testing.

Production monitoring is currently the most used approach when organisations rely on third parties for releases. There’s no formal testing done and problems identified by end users post-release are tracked in an issue management system.

This is a low-cost approach due to no real investment in testing, and may suit non-critical or complex applications or minor changes with low risk. However, all issues will only be discovered in production, there’s a potential for business and customer impact, and no ability to look at non-functional requirements (NFR) such as performance or security.

Once a release has been made, you can do testing in production. If a new device or updated operating system is released, you will have planned and implemented a regression suite.

It’s another low-cost approach, with issues found quickly in production and the customer impact is minimised. It is suited for non-critical or complex applications with minor changes, though issues are only found in production, there’s a potential customer impact, and limited NFR testing.

With software releases, a beta version is usually available to test in pre-production. With proactive beta testing, issues can be identified before the release goes live.

This method is suited for critical and complex applications, and major version upgrades, and you can test functional and non-functional requirements in your environment. However, it is more expensive than the other approaches and requires significant support from the organisation.

Now that you have a choice from these three options, how do you define, deliver, and implement a maintenance test strategy? Find out by watching the full recording of our Digital Change webinar on YouTube or by reaching out to our Digital QA team.