Engineering Excellence: Why Perfection Is the Wrong Goal
Engineering excellence isn’t about perfection. I see this misconception time and again, especially in larger organisations with multiple teams. There’s this persistent belief that before we can move to a new way of working, we must understand every facet, every choice, every possible outcome. The urge is to make a big, detailed plan—one that covers all the bases before we even take the first step.
But here’s the reality: as soon as you start implementing anything—whether it’s with people, systems, or software—the needs you discover are different from the ones you anticipated at the outset. The plan you so carefully crafted is almost immediately rendered obsolete by the first bit of feedback you receive. That’s not to say planning is pointless. Far from it. We absolutely need planning. But the plan itself, every little detail, is the irrelevant part.
There’s a quote often attributed to Eisenhower: “Plans are irrelevant, but planning is everything.” It’s the act of planning—the discussions, the exploration of options, the sharing of information—that matters. The plan itself? It rarely survives first contact with reality. Sun Tzu said it best: “No plan survives contact with the enemy.” In our world, the “enemy” is the complex, ever-changing nature of software development and organisational change.
If your goal is engineering excellence, stop chasing perfection. Instead, focus on instilling modern software engineering ideas that are, at their core, continuous:
- Continuous delivery
- Continuous quality
- Continuous adaptation
- Continuous learning
It’s this continuous, emergent nature that matters. The knowledge you need, the challenges you’ll face, and the solutions you’ll discover will all emerge over time. Our job is to create a system that adapts as new knowledge emerges, so we can deliver working products and high-quality output. The goal isn’t “agility” for its own sake, but being fast, nimble, and able to deliver for your customers. That’s what agility really means.
Common Barriers to Modern Engineering Practices
Every time I work with customers, we talk about practices that enable this kind of excellence. Take trunk-based development, for example. I was discussing this just yesterday, and the customer immediately listed five or six reasons why it wouldn’t work for them. “We can’t do it that fast,” they said. “Our process is too manual.”
My response is always the same: Why are those things blocking your ability to adopt modern engineering practices? Where did those constraints come from, and how can you fix them? Sometimes, the answer is legacy products or manual infrastructure that will take time to modernise. If you’re still manually building out your infrastructure or have manual steps in your build process, the idea of trunk-based development can seem daunting. The friction is real.
But if you want fast feedback, you have to get to your customers as quickly as possible—with high quality. Delivering rubbish just gets you rubbish feedback. We want to deliver high-quality, usable, working product continuously, as quickly and as often as possible.
What You Need for Engineering Excellence
To achieve this, you need a few foundational capabilities:
- Clean code: You can’t move fast with poor quality code. If you’re starting with a mess, that’s your first area to address.
- Observability: You need to see what’s happening in your system, both from a user perspective and internally. If you don’t have telemetry or observability, you’re flying blind.
- CI/CD (Continuous Integration/Continuous Delivery): Automate your build and deployment processes. Manual steps are friction.
- Automated testing: The more you can test automatically, the faster you can get feedback. Long-running, manual, or brittle tests are a bottleneck.
Think of it like the tracking on your car. If your wheels are even slightly misaligned, your tyres wear out faster, costing you more money in the long run. Pay a little now to fix the alignment, and your tyres last much longer. The same is true for your software. Fix the “tracking”—clean up your code, add observability, automate your builds and tests—and you’ll go further, faster, and with less pain.
A Real-World Example: Azure DevOps
Let me give you a concrete example. The Azure DevOps team started with a massive legacy product. The code quality was, by their own admission, not great. Testing was mostly long-running system tests—think Selenium, but with their own system. It took 48 to 72 hours to run the full test suite. That’s simply not good enough. If it takes that long to find out you’ve broken something, the developer has lost context and has to relearn everything, wasting massive cognitive effort.
So, what did they do? They chipped away at the problem, sprint by sprint, over four years—about 80 sprints. They paid back a little technical debt each time. The result? They went from 48-72 hours to just three and a half minutes to run the entire test suite, including a full environment and integration tests, on a developer’s local machine. That’s the power of shortening feedback loops.
Shortening Feedback Loops: The Heart of Agile and DevOps
This is the fundamental premise of both DevOps and Agile: shorten the feedback loops.
- DevOps: Shortens feedback loops in engineering.
- Agile: Shortens feedback loops in business.
If you want your teams to build better and deliver faster, start with engineering excellence. Not perfection. Not a grand, unchangeable plan. But a system that continuously adapts, learns, and improves. That’s how you deliver real value to your customers—and that’s what true agility looks like.
Meta Description:
Discover why engineering excellence isn’t about perfection, but about continuous improvement, clean code, and shortening feedback loops. Learn how to overcome common barriers and deliver real value with modern Agile and DevOps practices.
Engineering excellence isn’t about perfection, right? A lot of teams and organizations, especially organizations with more than one team, try. They have this ethos. Oh, we’re going to move to a new way of doing things and I want to understand all of the facets of that before we move. I want to understand all of the choices we’re going to make. I want to understand why we’re making all those choices. I want to make a big plan of what we’re going to do. And that’s not reality. That’s not how, when you actually start implementing things. And this is true for implementing things with people, implementing things in systems, implementing things in software.
The needs that you discover are different than the needs that you thought of right at the beginning. So there’s not a huge amount of value in planning out all the things at the start only to have that plan destroyed. So we still need planning. Absolutely. I’m not saying shoot from the hip and make stuff up. That’s definitely not what I’m saying. The plan itself, that every little detail is the irrelevant part.
I think there was a US president from years ago. Might have been a general or a US president, might have been Eisenhower, said, plans are irrelevant. Planning is everything. I think it was a World War II thing, right? It’s about the planning, the discussions, the what do we think’s going to happen? What do we want to try and do? What are our options when these different things happen? And discussing them and disseminating that information, but the plan itself is irrelevant.
I think isn’t there a Sun Tzu? Maybe no plan survives contact with the enemy, right? As soon as you start doing something in your organization, as soon as you start following the plan, it’s irrelevant, right? The first step, the feedback from that first step in your plan probably destroys the entire plan. So if your goal is to move towards engineering excellence, it’s not about perfection. It’s about instilling those ideas, modern software engineering ideas of continuous, right? Usually you stick continuous in front of everything. Continuous delivery, continuous quality, continuous adaptation, continuous learning. It’s that continuous or emergent nature that the things you’re going to need, the knowledge you’re going to have to learn, the things you’re going to find out are going to emerge over time.
So we need to create a system within which that emergent knowledge adapts the system as it goes to what is happening. So that we end up getting, delivering working products, getting high quality output. And although the goal is not agility, being fast and nimble and delivering for your customers is, right? And that can be a definition of agility for sure.
So there are loads of tools and capabilities, practices that are known to enable this. And every time I work with a lot of customers and we talk about some of these practices. I was talking yesterday about trunk-based development and the customer came back with five or six reasons why trunk-based development wouldn’t work for them, right? Or we couldn’t do it that fast, but you know, it could be this long.
And my immediate response was, well, why are those things blocking your ability to do that modern engineering practice? Where did those things come from and how do you fix them, right? And some of it could be a long-term fix. We’ve caused some legacy products and it’s going to take time to get to that point. Right? If you’re currently manually building out your infrastructure for your products, if you’re currently doing manual portions in your build process, right? It’s not an automated CI/CD process. There’s stuff somebody has to go do in between, in different spots in order to make it work. Then it can be really daunting to have this idea of, well, we’re going to do trunk-based development. We’re going to have a topic branch spin-off. It’s only going to exist for a few hours. It’s going to have a full environment. We’re going to run all our tests and then it’s going to come into main in a few hours. How is that even possible, right? Because of the frictions that are holding us back.
But if we want fast feedback, right, then we have to get to our customers as quickly as possible with high quality, right? Not delivering them crap because then the feedback will be it’s crap, right? We want to deliver high quality, usable, working product on a continuous, as quickly as possible, as often as possible basis. And for that, we’re going to need to have clean code, right? You can’t do this with poor quality code. And you might be starting with poor quality code. So there’s something to go work on. You need observability in the way you do things and you may not have observability today. You may not have telemetry coming through your product. You may not have those things built in that allow you to see what’s going on, the knowledge that you gain on how users are using your system but also what’s happening in your system while users are using it. It can be invaluable, right?
You’ve effectively got brittle, I’m going to say poorly architected, but I don’t mean that as a direct criticism. Maybe atrophied code. It’s not usually a deliberate effort. We want to build bad code and we want to build bad products and we’re bad people because we built bad products. That’s not what we’re talking about. It’s something that happens slowly over time. It’s like a drift. It’s like when your tires on your car are not aligned properly, when the tracking is not aligned properly, right? Maybe it’s just a little bit out. Maybe it’s just a degree. That could be quite a lot actually. Maybe it’s just a fraction of a degree out. And that means the wear on your tires is unnatural, right? And you maybe have to replace your tires more often. So, it’s costing you more money. And you’re trying to figure out why is this costing me more money? Well, and it’s £100 a tire and you’re doing that every, let’s say every year you’re having to replace these tires. You’re like, what’s going on?
But if you paid £40 to get the tracking aligned, then we don’t have that problem. Maybe our tires last a lot longer. Maybe they last 50,000 miles and someone, that’s actually a lot for tires. Maybe they last 20,000 miles instead of 5,000 miles, right? So that’s the type of thing we’re talking about. How do we fix the tracking on our software, right? Well, we need clean code. We need observability. We need CI/CD ideas. We need to automatically build our product. We need automated testing. We need to test as much as possible. This is that whole idea that fixing the tracking is the idea of also shifting left. You might have heard that terminology, but shift left is about moving things as close to the developers writing the code to figuring out that it doesn’t work.
I always use the example of the Azure DevOps team because they started with a massive legacy product. Very low, I’m going to say low code quality. I think they probably agree with that. Not great code quality. And very poor testing capability. Most of the testing was long-running system tests, like things like Selenium tests today, but they had their own system. And that’s just not good enough anymore because they take too long to run. It takes too long to find out, for the developer to find out that they’ve broken something. And if it takes too long, they’ve lost the context. And if they’ve lost the context, they have to relearn that context. And that’s a massive cognitive effort that could have been used to work on other things, right?
So we want fast tests and they were, I think they were either 48 or 72 hours to run their full test suite against the stuff they were building as Azure DevOps. Not in Azure DevOps actually, Azure DevOps being building that product. And they spent a bunch of effort. It took four years because they’re doing a little bit at a time, right? They’re paying back a little bit of that friction every sprint. And over four years, which I think is 80 sprints. They did three week sprints, so you can probably figure it out. But over those 80 sprints, they paid back a little bit of time and they went from 48 to 72 hours to find out whether they’d broken anything to three and a half minutes.
So a developer can run the entire test suite for the product in three and a half minutes on their local workstation. And that includes having a full environment running to be able to run those types of, not just the fast unit tests, but that includes some integration tests as well. And that is what makes things better. That’s what makes things faster. The quicker, shortening the feedback loops. This is the fundamental premise of DevOps, of agile, is shortening the feedback loops. DevOps is about shortening the feedback loops in engineering. Agile is about shortening the feedback loops in business, right?
So if you want your teams to build better and deliver faster, you need to start with engineering excellence.
