Despite the Agile Manifesto being over 21 years old, empirical research on its industry impact has been limited.
However, a recent study has shed light on the effectiveness of Agile practices, revealing that 65% of software projects adopting Agile requirements engineering practices fail to deliver on time, within budget, and to a high standard of quality. In stark contrast, projects using a new approach called Impact Engineering only fail 10% of the time.
This groundbreaking research was conducted for a new book, Impact Engineering, which presents an alternative methodology that significantly outperforms traditional Agile practices. The study involved 600 software engineers, with 250 based in the UK and 350 in the USA. Fieldwork was carried out from May 3rd to May 7th, 2024, by J.L. Partners, a member of the British Polling Council.
Key Findings:
- Projects using Agile practices are 268% more likely to fail than those using Impact Engineering.
- Projects with a documented specification or requirements before development begins are 50% more likely to succeed.
- Clear requirements before starting development increase success rates by 97%.
- Projects that avoid significant requirements changes late in the development process are 7% more likely to succeed.
- Psychological safety for engineers to discuss and address problems increases project success by 87%.
- Real-world problem-based requirements boost success rates by 54%.
The statistical significance of the data is compelling, equating to the probability of rolling a number six consecutively six times on a fair six-sided dice on the first attempt.
Cost Savings Potential: Adopting Impact Engineering could save an estimated $115 billion USD annually on wasted R&D spending in the USA and approximately £7 billion GBP annually for British taxpayers on failed government digital change projects.
Agile Practices Under Scrutiny: Three key practices of the Agile Manifesto—working software over comprehensive documentation, customer collaboration over contract negotiation, and responding to change over following a plan—are called into question by this research. The study shows that a robust requirements engineering process and the ability to address problems promptly are crucial for project success.
Psychological Safety and Burnout: The study also highlights the importance of psychological safety for software engineers. Projects where engineers feel safe to discuss and address problems are significantly more successful. Additionally, burnout remains a critical issue, with 83% of software engineers previously reporting high workload as a primary cause.
Historical Context: The research follows public awareness of catastrophic software failures, including the infamous Horizon IT system scandal in the UK. This early large-scale Agile project, which used Rapid Application Development, faced severe technical issues due to the lack of a robust requirements engineering process, leading to what has been described as the largest miscarriage of justice in British history.
Transatlantic Differences: The study reveals a troubling disparity between the US and UK, with UK engineers 13% less likely to feel they can discuss and address problems. This difference highlights a significant cultural gap in engineering practices between the two countries.
Author Insights: Dr. Junade Ali, author of Impact Engineering, emphasizes the need to question the widespread adherence to Agile practices: “With 65% of projects adopting Agile practices failing to be delivered on time, it’s time to question Agile’s cult following. Our research has shown that what matters when it comes to delivering high-quality software on time and within budget is a robust requirements engineering process and having the psychological safety to discuss and solve problems when they emerge, whilst taking steps to prevent developer burnout. This is fundamental to the philosophy of Impact Engineering.”
Impact Engineering is now available on Amazon in Kindle eBook and paperback formats. The business novel is based on real-world case studies of personal and organizational transformations using the Impact Engineering methodology, alongside a chapter detailing the scientific basis of the approach.
