Unlocking Engineering Excellence: How Azure DevOps Transforms Traceability, Transparency, and the Developer Experience

Azure DevOps isn’t just a tool set. It’s designed as a platform for excellence. It’s designed as a platform engineering capability, something that you can use within that context. And the reason I mean Azure DevOps is the brainchild, is probably the way you describe that. The visionary behind Azure DevOps was a gentleman called Sam Gubenheimr and he’d been around for quite some time in the ALM application life cycle management space.

And she envisaged this world where we as builders of products, software engineers, managers, all the people involved in building products were able to see what’s going on really well. I want to be able to see if I’m a manager, product owner, if I’m in product management, and I want to be able to see where my feature is, what state it’s in, not only are the bits finished, but in what environments those bits have gotten to, right? So if you’re doing a ring-based deployment model, are they still in the canary environments or have they moved up to the preview environment or are they in production? How do I know, how do I get that visibility without just going and asking people?

I want to be able to see that information so that I can ask more interesting questions than “where’s my stuff,” right? “Where’s my stuff” is not an interesting question.

So his idea was to create a system like that and Brian Harry in the good old days was the product unit manager. He was the delivery manager for that vision. And a lot of those things that we would like Azure DevOps to do are in there. They’re in there and they’re usable and they really bring capabilities for our developers that perhaps you don’t even realize are there. Right?

I’ve been using Azure DevOps since 2006, 2007, when the preview was launched. So, very long time. And Azure DevOps has a bunch of capabilities that most people have no idea are in there. So, for example, you can get full traceability from the work that you’re doing, the work items that you’re working on, all the way down through to the lines of code that were changed to fulfill that capability. And that can be at the task level, at the backlog item level, at the feature level, or at the epic level. And Azure DevOps has that hierarchy because it does this clever thing that as long as your developers have to do something, they have to associate the code that they are working on with the work item. So they tag it in the comment: “This is what I’m working on” or “this closes this item,” right? And then the automated build system inside of Azure DevOps picks up all of that awesome data and then it goes and writes onto the work items. They have a hidden field called the integrated in field. They go right into those work items. This is the build number, right, all the way up the chain.

So then the epic will always have the latest build number that it has things in. And then you drill down to the next level and you’ve got a set of features and each of those features have the latest build number that has stuff from that feature in it. And then all the way down and you can see with that build number, you can then look at your environments and see, well, where is this version? What version is in what environment?

Now, that would be a pain in the butt if we had to manually go figure out all that stuff. So, nicely, you can open up the environment and you can basically say what features are in this environment, what code has been changed in this environment from the previous environment, or because you’ve got a build that’s deployed, a build ID, and Azure DevOps is able to do those diffing between any two builds, right? What’s the difference between these in features, in source code changes?

So think about the value proposition of that from a business perspective. You might be looking at risk analysis. You might be looking at, well, we’re about to do a new release of this version. This is the version we were on before. What’s changed between these two? Here’s all of the tasks, all of the backlog items, the stories, all of the features, all of the epics that have been impacted by these changes. And then for the technical folks, here’s a list of all of the changes, all of the code changes that have been made. Audit problems solved, right? All of your audit problems are solved with that story. And that’s just one of the capabilities inside of Azure DevOps that’s actually, they’re called the DevOps features. They’re there specifically to support a DevOps story.

So this integrates the Azure boards for planning with the Azure repos for source control and Azure pipelines for automated builds and maintains a level of traceability across all of these items. Then you bring in things like the test tools. So hopefully you’re doing automated tests. Automated tests are already associated with your build, right? Because you’re running them within the context of your build. But then you can also bring in manual tests, right? There’s two types of, I’m going to be flippant and say there’s two types of manual tests. There’s the old school manual tests which should really be automations that are test scripts, test cases, right? You have a test case, you work through it. There’s reasons why you might definitely have test cases for sure. I can think of a couple of scenarios. I might mention one.

And then you’ve got things like exploratory testing where you’re looking through the system and you want to find it. And both of those are supported by the test tools in Azure DevOps. Although one of them is completely free. You can use the feedback tool for free. You’ve probably not seen that tool. It’s a feedback tool and it plugs into your browser and lets you record and create tests. It’s awesome. Validate tests as well.

So those tests are run against a specific version of your product, right? So now when you create a bug, the bug is associated with the version of the product that you’re testing. You know what code’s changed between that version and a previous version that didn’t have the bug. It’s much easier for the developers to find the bug and fix it. Once the bug is fixed, the bug has the build number on it that tells you which build it’s been fixed in. So you don’t have that problem of the testers finding that bug again because they can see that that bug has been fixed in a newer build than the one they’re testing right now. So it’s been fixed, it’s just not fixed in the version we’re using and we can see that information.

This reduces what I would call bug pingpong, right? The developers say it’s fixed, the testers say it’s not. It is fixed but also it’s not, right? So it reduces that big bug ping pong. It increases the levels of traceability. It increases the levels of transparency between the people that are working on the product. Ideally you’ve just got everybody working together in one team but the realities of lots of organizations are not there yet. We want to work towards that but we’re not there. So how do we do that in the interim?

And it has this really compelling story for the test tools, a valid story for the test tools. This is my agile story for the test tools: you’re going to find a problem. You’re using the feedback tool on the app and you find a problem, right? So you provide feedback and that feedback is, or a user provides feedback and then a tester needs to go in and validate that feedback. So they go in and create the steps to reproduce in a test case, right? So that’s a test case work item type. You test case with a set of steps. You can automatically generate that from the feedback tool but test case in steps. And then you run that test case, collecting whatever data you want around that, and create the bug.

So the bug is associated with a test case that proves that that bug exists. So when the developers go to work on that bug, they can see the test case, they can see the test steps, they can run those test steps to validate that they fixed the bug, right? So you’ve got that full life cycle and then when they close off the bug, this is not bug ping pong, it goes back to the tester and the tester says it’s still there. They’ve run the test case, the bug is gone. Give that back to the testers and the testers can then validate in an environment where the bug is fixed so they don’t false positive “it’s still broken.”

Those types of traceability features are really powerful. Those are just examples, a couple of examples of things within the tool that allow you, if you’re using the tools, if you’re delivering engineering excellence within the context of the tools, if you build a story around platform engineering where you’re talking to the developers and the coders and the testers and management and everybody about the capabilities of the tools, and building a developer experience. What is the product developer experience for your engineers within your organization? You can really pull out and simplify the story that you have with traceability and observability around your building of the product. Same as you would within the context of your product with telemetry coming from your product. This is telemetry and data and transparency coming from your engineering platform, which in this case we’re talking about Azure DevOps.

You can also integrate that with GitHub as well. So you can have your code on GitHub and do all these cool things within the context of Azure DevOps integrated with GitHub. So if you want a platform that you can build across your organization that improves your developers’ experience and encourages developers to use those features, you need to tell them about those features and explain them and educate and they’ll want to use the platform. They’ll want to use those features. DevOps.