Why Half of Software Development Projects Fail

Oct 7, 2022

Studies show a shocking 70% of digital transformation projects fail (Sources: Boston Consulting Group and McKinsey).

Another study conducted by Info-Tech Research Group shows that 70% of those failures are due to issues with requirements. If we take these two statistics together, we can infer that 49% of digital transformation projects will fail due to requirements. That’s almost half!

These studies ring true with our experience of over 200 digital transformation projects. Requirement gathering, or lack of, is very often the cause of many problems when projects are implemented.

We have tried to highlight some areas that commonly cause issues so that you are forewarned when you undertake a digital transformation project.

Why almost half of software development projects fail

(Ab)use of agile methodology.

Agile methodologies are great. They can be used to speed up development of projects and potentially get things delivered quickly. However we have seen a worrying trend where people, with restrained budgets and timescales, use Agile Methodologies to avoid any upfront requirements capture.

The story goes that we don’t need to define upfront requirements as “it’s agile”. Some companies can handle a very low requirements approach such as a small start-up team with a strong vision of the product they are developing. However, for companies who do work on clients’ behalf or internal IT departments who need to fulfil the needs of internal stakeholders, we would highly recommend some detailed requirements capture. Companies usually need to budget for projects and estimate timescales so a lack of upfront requirements can mean no clear budget or timescale for the project.

Overruns in budget are a common reason for project failure as the company runs out of money for the project.

Missing requirements

Gathering a complete set of requirements for a project is difficult. We are only human and stakeholders and the implementation team can miss things. Failure to properly gather requirements means some requirements are missed, creating two problems : –

The project can take longer to deliver than expected
Costs can increase beyond those budgeted

Missing requirements are generally discovered during project implementation. As projects run people begin to notice things that have been missed. This can cause unplanned tasks and in the worst cases missing requirements, once discovered, can interfere with documented requirements causing rework. According to a study by Info-Tech Research Group 50% of re-work was as a direct result of requirements issues. That’s a lot of wasted money and effort.

You can never guarantee that you will not miss requirements but a structured process and structured questions can help to reveal areas of the project requirements that may be otherwise forgotten. Where appropriate we find that showing stakeholders wireframes or prototypes of proposed systems helps to focus the stakeholder on what will be delivered.

Lack of detail in requirements

Related somewhat to missing requirements, a lack of detail in requirements can cause problems. Whilst this may be considered similar to missing requirements there can sometimes be a complacency that all the requirements have been captured and agreed with the business stakeholders yet due to a lack of detail or clarity the estimate and timescales for the project can be breached because of the extra work involved.

Whilst we would not consider this as bad as missing requirements, the lack of detail can suddenly start adding effort to each part of the system. For example, if the need to store and list a client was detailed only at a high level then the details such as requiring a photo and video of the client or having some sort of credit check done on the client might add additional effort and cost not anticipated when timescales and budgets were set.

In the example above if your project does require a client to be modelled then noting the data stored against the client (e.g. photo) and actions performed on the client (e.g. credit check) will help tie down requirements at a better level of detail than just specifying that the system will have clients.

Not enough attention paid to non-functional requirements

Not paying enough attention to non-functional requirements can trip up a project implementation. For example, what are accessibility requirements, security requirements, reliability and performance requirements. What browser versions are you targeting for a web site or what platform are you targeting for a mobile application. We know of one company who outsourced a company web application which was delivered successfully. Unfortunately, no one told the outsource company that the client standard technology was React and the web app was delivered written in Angular. The client could not support the application so ended up having to get the application rewritten at their cost.

Non-functional requirements are a common area where inexperienced business analysts and stakeholders can fail to specify important areas of the system. Again, this can affect budgets and timescales.

Ensuring that structured questions are asked in relation to non-functional requirements can really help stakeholders and the implementation team think about these areas of the system and start to specify these areas.

To us, it is clear, and studies back this up, that the requirements gathering process causes many of the issues associated with failed projects.

With our industry experience helping a range of clients we have grown weary of seeing projects fail, overrun within the industry, or not achieve their goals because of poor requirements. We believe the industry needs more process and structure in requirements gathering and, as a result, we have developed Requiment. Our product helps you through a set of guided questions and serves as your virtual business analyst or, at least , a business analysis assistant.