While major software development project problems are often the stuff of headlines, it seems they are more the rule than the exception – and we are getting worse at them.
According to Planit Sofware Testing’s annual Software Testing Index, only 42 percent of software projects started in the past two years were delivered on time, on budget, and in line with their original scope. This result was a decline from 2009 when 49 percent of projects were reported as being delivered optimally. The Index found that projects were more likely to run over time, cost more than was estimated, or experience changes to more than 25 percent of their original scope.
Planit managing director Chris Carter said the major causes of these problems were poor definition and gathering of project requirements.
“We seem to have this inability to firstly to define what the requirements are adequately, and when we have defined them, to actually stick with them,” Carter said. “And we are not very good at project estimation from a budget and a time perspective. And we don’t set realistic expectations – we are almost setting ourselves up to fail.”
Not surprisingly then 64 percent of respondents said that improving requirements gathering would make a significant positive impact on project outcomes.
When a project came under pressure respondents nominated that they were more likely to extend the deadline or even increase the budget rather than reduce their focus on quality processes such as testing. Only 1 percent of projects were reported as failing entirely.
Now in its fourth year, the Planit Software Testing Index gathered responses from 235 individuals with roles related to software testing in organisations across Australian and New Zealand, with strong representation from the Software/IT, Financial Services, Government and Telecommunications sectors. Other findings included a significant rise in the number of organisations that use Agile software development methodologies, which overtook the more traditional Waterfall model of sequential software development process. Almost half of all organisations now use Agile, although often in parallel with more traditional methodologies.
The Index also found that despite the role that testing can play in improving software project outcomes, attitudes towards software testing declined in many organisations. The number of respondents that rated it as a critical element in producing reliable software fell from 56 percent in 2009 to 45 percent now.
“A further 22 percent of our respondents consider software testing to be strategically important to a project’s success,” Carter says. “But we now have 19 percent of our respondents who are saying that testing is a necessary evil, and we also have a large number of respondents who are thinking that testing is either a cost to be minimised or not a major priority in their organisation.”
Carter said he was pleased however to see that two thirds of respondents were business-focused when it came to making their business case for software testing, with 40 percent nominating that it leads to enhanced customer satisfaction and builds customer loyalty.
On average, testing accounted for 19 percent of project budgets, behind only the actual development phase in terms of budget allocation.
“With 19 percent of the budget you should be able to deliver a quality product,” Carter said. “The justifications for a reasonable budget for software testing to deliver quality are well presented and appear to be reasonable well received by executive management.”
He also said respondents’ description of project conditions was generally positive, making the negative project outcomes even more puzzling.
“One would have expected with the project conditions being reported that the project outcomes would have been fairly positive, but we’ve found that in 2010 the project outcomes have been worse than 2009,” he said.