If you’re building products at any scale—whether you’re a small startup or a sprawling enterprise—one thing has become abundantly clear to me over the years: you simply cannot make good decisions without evidence. I’ve seen too many teams, and too many organisations, fall into the trap of “gut feel” decision-making. It’s seductive, but it’s also a fast track to mediocrity. Evidence-based management isn’t just a buzzword; it’s the foundation for real, sustainable improvement.
Now, evidence can take many forms. At the business level, it might be hypothesis-driven experiments. At the delivery level, it’s all about flow metrics—lead time, cycle time, work in progress, and so on. The challenge is always the same: how do we get the right data, and how do we make it visible in a way that actually helps us improve?
The Visibility Imperative
One of the most fundamental lessons I’ve learned—sometimes the hard way—is that without visibility, you’re flying blind. If you can’t see what’s happening, you can’t change it. And if you can’t see the impact of your changes, you’re just guessing. This is why telemetry is so critical—not just telemetry from your product, but telemetry from your process.
Azure DevOps, for all its quirks, is actually a fantastic system for gathering this kind of telemetry. Every value of every variable in your work items is collected and stored. Add a custom field? No problem—the full history is there. This means you can pull and analyse any data you like, in any way you like. That’s powerful.
Out-of-the-Box: The Good, the Bad, and the Rudimentary
Let’s be honest: the visualisations built into Azure DevOps are a bit basic. Microsoft’s approach is to provide the data layer, the API layer, and a few “table stakes” visualisations. The expectation is that partners and customers will build the value-adds that are specific to their needs.
- Lead Time & Cycle Time: Out of the box, you get these metrics, plus a bit of work in process. On your boards (let’s call them Kanban boards, though that’s a stretch), you can set WIP limits for each column. They’re not enforced—if you go over, the column just turns red. There’s no system-wide limit, which is a bit of a miss.
- Averages: The dashboards show average cycle and lead times. Personally, I’m not a fan of averages. I want to see the 85th percentile, the spread, the outliers. But Azure DevOps picks one metric and sticks with it.
- Cumulative Flow: You get a basic cumulative flow diagram. It’s straightforward, and it’s free.
All of this is built on a rich data foundation. Microsoft is opinionated about when work starts and finishes, but all the underlying data is there. You can access it via APIs, or load it into PowerBI and build whatever graphs you want. If you’re a large organisation with an internal engineering team, you can build custom boards, controls, and dashboards, and share them across the company. If you’re small, you probably don’t want to do that—you want a vendor solution that just works.
If you want to go beyond the basics (and you should), there are two tools I recommend:
Flow Viz
- Pre-built PowerBI reports that connect to Azure DevOps.
- Pulls your data and displays all the key flow metrics.
- Free to use—download, install, and you’re off.
- Great for teams who want more insight without building everything from scratch.
Actionable Agile Metrics for Predictability
- A plugin for Azure DevOps—interactive, JavaScript-based, and your data stays in your system.
- Costs about $20 per user per month, but worth every penny in my experience.
- Provides rich visualisations: lead time, cycle time, percentiles, WIP overlays, and more.
- My favourite feature: the work-in-progress view with percentile overlays, so you can instantly spot items at risk.
Both tools give you far more than the out-of-the-box experience. You get deeper insights, more actionable data, and a much clearer picture of your flow.
If you’re a developer and prefer the command line, Ben’s Azure DevOps Admin Tools are worth a look. It’s a CLI toolkit that pulls most of the data you need—cycle time, lead time, graphs, and more. It’s not as user-friendly for non-developers, but it gets the job done if you like to script your way to insight.
Making the Invisible Visible
With all these options, it can feel a bit overwhelming. But the goal is always the same: make your work visible. Move from gut feel to data-driven delivery. When you can see your flow, your bottlenecks, your risks—you can actually do something about them. You can experiment, measure, and improve with confidence.
If you’re ready to stop guessing and start making decisions based on real evidence, let’s talk. I’ve helped teams of all sizes make their work visible and their delivery more predictable. It’s not magic—it’s just good practice, backed by the right data.
Let’s make your work visible, and your outcomes better.
If you’re building products at any scale, whether it’s small scale or large scale, it’s really important to make decisions based on actual evidence. And that evidence can come in the form of hypothesis-driven practices at the business level, at the value level, or it can come all the way down to flow metrics at the delivery level. And that’s one thing that I would probably say Azure DevOps is a little bit rudimentary in that space out of the box, but we can absolutely get those things and augment those things.
So one of the fundamental understandings that I’ve gained over many years of building software and helping teams build software is without visibility, you can’t see what’s going on. And without being able to see what’s going on, you can’t make effective changes. And you also can’t see the impact and effect of those changes on the way that you work.
So we need telemetry. We need data. Some of that telemetry comes from your product, right? Building open telemetry into your product, collecting specific and deliberate telemetry from your product. But we can also use telemetry on the way that we work and Azure DevOps is a fantastic system for gathering that telemetry.
So in Azure DevOps, every value of every variable that ever was inside of your work items is collected and stored. Okay. So it doesn’t matter what field it is. If you add a custom field, no problem. The full history of that field is available within the context. So that allows you to pull and collect any sort of data you like in any way you like. Okay. That’s the powerful part, right? All of that data is there.
The visualizations that are built into Azure DevOps, they’re a little bit rudimentary, but that’s fundamentally Microsoft’s premise, right? Microsoft’s premise is we’re not going to build everything for you. We’re going to build the data layer, the API layer, and then we’ll build some examples of here’s what you can do with that data that meet the table stakes. We’ll build the table stakes and then let partners build the value adds that are specific to your organization.
So I work with customers that have a large internal ecosystem of custom boards, controls, dashboards that they build internally and ship around the organization. So they have an engineering team who builds augmentations for Azure DevOps. They support other teams who build augmentations for themselves in Azure DevOps and allow those to be shared across the organization. And Azure DevOps has capabilities for that as well, right? That’s for people that are really big. If you’re really big, you can absolutely do that. If you’re really small, you don’t want to have to do that. You want some vendor to do that for you and you absolutely have that capability.
So there’s a couple of things that you can do within the context of Azure DevOps. So out of the box, it does have lead time and cycle time. Okay. And there’s a little bit of work in process. So, at the top of your columns on your boards, the board, I guess they’re called Kanban boards, right? But they’re not really, they’re just boards. On the top of the boards, they have a WIP limit. It is not enforced. It will just go red if you go over the limit. And the limit is for each column. They don’t have a system limit, right? So, that’s a limitation. They don’t have a system limit, just a limit for each column.
You can add cycle time and lead time to the dashboards and it will give you a hover over view of your average, just the average cycle time and lead time. I don’t like the average. I like the 85th percentile and it should have all of them but it doesn’t. So that’s what I mean by sometimes it’s a little bit basic and rudimentary. It’s opinionated. They pick one thing. So they have that but you can hover over individual items and see where they are and then they have a cumulative flow, right, which is again straightforward, just visualizes the cumulative flow. That’s out of the box, you get that for free.
Microsoft is opinionated on when work items start and when work items finish. That’s beyond the scope of this but we can definitely talk about that in another video. But all of that data that they use to make that is stored in the system. So you can access the APIs and access the PowerBI, the analytics. So everything can be loaded into PowerBI and build whatever graphs you want. Right? So that’s one. But if you’re small enough, you don’t want to build it yourself.
I recommend two, there’s two tools that I definitely recommend taking a look at. One is called Flow Viz, and that’s effectively some pre-built PowerBI reports that you can load into your PowerBI environment. It connects to Azure DevOps and pulls your data and displays all of the cool stuff. And it’s free, right? Flow Viz gives you that capability. Great. Go try it out. Download it, install it.
The other one is actionable agile metrics for predictability. So actionable agile is a plugin, an interactive JavaScript-based, no data goes anywhere but your system plugin, but it’s $20 per user per month. I recommend that one, too. That’s what I use because I just want an easy visual indication that I can just add a plugin to Azure DevOps and go spelunk people’s data.
So both of those tools give you much more accurate—accurate is probably the wrong word—but much better, much more features, many more features for lead time, cycle time, for percentiles, for seeing your work in process, seeing your flow. I particularly like in actionable agile it has a view which is work in progress and you can see the overlay of your percentiles on your work in process. So you can see the things that are most likely to be in danger within that context.
So using Azure DevOps you get access to all of the data and some visualizations. Then bringing in some third party tools on top of that, you can make it really light up and create any sort of data visualizations you want.
If you’re a coder, Ben also has a, I think it’s called the Azure DevOps admin tools. He has a little command line toolkit that you can run that will get you most of the data as well and give you your cycle time, lead time, build your graphs, all of those things from the command line. So if you’re a developer, you can use that, but it’s not as friendly for everybody else that might be using the system.
With all of these options, it can be a little bit difficult to navigate. So if you’re ready to move from making decisions based on gut feel to more of a data-driven delivery story, give me a call and let’s make your work visible.
