Rethinking Software Estimation: Embrace Probabilistic Forecasting for Agile Success

Published on
3 minute read

One of the most significant challenges I encounter in the realm of software projects is the perennial struggle with estimation. It’s a topic that seems to generate endless debate and confusion. Many organisations feel compelled to engage in some form of estimation, whether that’s through absolute estimation—which often leads to errors—or by employing relative estimation tools. The reality, however, is that we often feel we need these estimates to chart our course forward. But do we really?

In my experience, what we truly need is the ability to forecast into the future with a degree of probability. When it comes to software delivery, where variance is high and uncertainty reigns, achieving accuracy in our predictions is nearly impossible. If you find yourself in a situation where more than 50% of your project is unknown, you can forget about precise predictions. Instead, we should focus on probabilistic forecasting.

So, what does this look like in practice? Rather than attempting to estimate individual items in our backlog, we can utilise forecasting simulation tools to project our capabilities based on historical data. For instance, we might say, “In the next 30 days, we can deliver 50 items with an 85% likelihood.” This approach allows for the inherent unpredictability of software development while freeing our teams from the burden of extensive estimation processes.

Here are a few key takeaways from this approach:

  • Embrace Probabilities: Instead of fixating on exact numbers, focus on probabilities. This shift in mindset can alleviate the pressure on teams and foster a more realistic outlook on project timelines.

  • Reduce Batch Size: While we may not be estimating individual items, it’s still crucial for teams to analyse their work and break it down into smaller, manageable chunks. This practice not only aids in understanding the deliverables but also enhances our ability to execute effectively.

  • Build Confidence: In this new paradigm, we may not have traditional estimates, but we can still cultivate confidence in our forecasts. I often encourage teams to aim for an 85% confidence level. Given that around 70% of startups fail, this benchmark provides a pragmatic approach to assessing our likelihood of success.

  • Assess Your Confidence Level: Ask yourself, “What’s my confidence level in delivering this project?” As long as it exceeds 70%, you’re likely on solid ground. This perspective allows for a more flexible and adaptive approach to project management.

In conclusion, while the traditional methods of estimation may no longer serve us well in the fast-paced world of software development, we can still navigate the complexities of our projects with confidence. By embracing probabilistic forecasting and focusing on understanding our work, we can create a more agile and responsive environment that ultimately leads to better outcomes. Remember, it’s not about having all the answers; it’s about having a clear path forward, even amidst uncertainty.

One of the big struggles within the context of software projects is estimation. Most organizations try and do some form of estimation. Either they’re using absolute estimation, so we’re talking errors here, or they’re using some kind of relative estimation tool. It’s really difficult because we feel like we need estimates to understand what we’re going to do. We actually don’t.

But what we do need is the ability to forecast out into the future with some kind of probability. Right? So when we’re talking about software delivery, where we have really high variance, it’s still really difficult. There is no tool when you have more than 50% unknown that you can use to accurately predict the future. Right? Accuracy is not a thing you can have.

So what’s the best we can get? Right? We want to be looking at probabilities and probabilistic forecasting. So what we tend to do is use some kind of forecasting simulation tool to look forward into the future based on what actually happened. So we’re never actually effectively estimating an individual item within our backlog. We’re just saying that we can deliver, let’s say, in the next 30 days, we’ll deliver another 50 items with an 85% likelihood that that’s going to happen.

That leaves wiggle room for things not happening the way you think, and it doesn’t require the people on your teams to spend a significant amount of time estimating. It doesn’t require any actual numbers to come out of the team. It still requires the people doing the work to analyse the work and break it down into smaller chunks. Right? That’s reducing the batch size of each individual item. That is still required. That is something you want to spend time on because it builds understanding within the context of what it is that we’re delivering.

Right? We want to understand the thing we’re delivering, and that understanding is the main impactor of our ability to execute on those things. So in this new world, there isn’t really estimation, but we still have confidence, and we put an actual probability on our confidence. I usually encourage teams and departments and organizations to look at about an 85% confidence level, being that what 70% of all startups fail. So it probably relates a little bit to 70% of all ideas are going to be unsuccessful.

Right? So what’s your confidence level of being able to deliver? As long as it’s higher than 70%, it’s probably a good bet, and we can figure it out from there.

Forecasting Estimation Software Development Decision Theory Product Delivery Pragmatic Thinking Decision Making

Connect with Martin Hinshelwood

If you've made it this far, it's worth connecting with our principal consultant and coach, Martin Hinshelwood, for a 30-minute 'ask me anything' call.

Our Happy Clients​

We partner with businesses across diverse industries, including finance, insurance, healthcare, pharmaceuticals, technology, engineering, transportation, hospitality, entertainment, legal, government, and military sectors.​

Kongsberg Maritime Logo
Philips Logo
Ericson Logo
Hubtel Ghana Logo
Boxit Document Solutions Logo
Emerson Process Management Logo
Higher Education Statistics Agency Logo
Slaughter and May Logo
Brandes Investment Partners L.P. Logo
DFDS Logo
Slicedbread Logo
Epic Games Logo
Workday Logo

CR2

YearUp.org Logo
ALS Life Sciences Logo
Bistech Logo
SuperControl Logo
Washington Department of Transport Logo
Nottingham County Council Logo
Washington Department of Enterprise Services Logo
Ghana Police Service Logo
New Hampshire Supreme Court Logo
Royal Air Force Logo
Higher Education Statistics Agency Logo
Lockheed Martin Logo
SuperControl Logo
Hubtel Ghana Logo
Milliman Logo
New Signature Logo