In my experience working with organisations of all shapes and sizes, I see a recurring pattern that undermines engineering excellence: teams are still testing quality in, rather than building it in from the start. This isn’t just a technical quirk—it’s a fundamental flaw that ripples through your entire delivery process, inflating costs, slowing feedback, and eroding trust in your product.
Let’s be clear: when you rely on testers to catch issues after the fact, you’re effectively giving engineers permission to say, “It’s fine, QA will catch it.” But the further a defect travels from the engineer’s keyboard, the more expensive and disruptive it becomes to fix. Finding a bug in production is the worst-case scenario, but even waiting until QA validation—after code has already polluted your main branch—means you’re introducing unnecessary friction and risk.
I’m currently working with a customer who exemplifies this. Their workflow pushes code into a preview environment for testers to validate, but by that point, the code is already in main. If a problem is found, it’s another branch, another pull request, another round of changes. Yes, testers are essential, but the ideal is that they find nothing. We want to catch issues as early as possible—ideally, before the code ever leaves the engineer’s hands.
How do we achieve this? It comes down to a few core practices:
- Automation: Automate everything you can—builds, tests, deployments.
- Integration: Continuous integration (CI) and continuous delivery (CD) are non-negotiable.
- Test-Driven Development (TDD): TDD isn’t a testing strategy; it’s an architectural and design discipline. It ensures your code is testable, focused, and does what you expect.
- Static Code Analysis: Don’t try to boil the ocean by turning on every rule in a legacy codebase. Instead, enable specific checks, fix them incrementally, and make those warnings part of your developers’ daily feedback loop.
- Code Review Policies: I favour trunk-based development, where long code reviews are the enemy. The goal is to get changes into main (or trunk) as quickly as possible, automating approvals unless there’s a real infraction. I even use AI checks to enforce team policies—sometimes overzealous, but often invaluable.
But even with all this, catching issues at the pull request stage is still too late. We need to shift left—move quality checks as close to the engineer as possible, and as early as possible.
The Azure DevOps Example: A Real-World Shift Left
One of my favourite stories comes from the Azure DevOps team. When they moved from a two-year release cycle to a three-week cadence, they hit a wall: their full regression suite (think selenium-style system tests) took 24 to 48 hours to run. Imagine waiting two days to find out if your change broke something! Developers’ changes were batched into rolling builds, so if you were unlucky, it could be nearly four days before you got feedback.
This is the epitome of testing quality in, not building it in. Those long-running tests are lagging indicators—by the time you get feedback, the context is lost, and the cost to fix is high.
The Azure DevOps team had around 36,000 of these system tests. Over four years, they methodically refactored, moving tests down the pyramid: from slow, brittle system tests to fast, reliable unit tests using stubs, mocks, and solid engineering principles. The result? They went from 48 hours for 36,000 tests to three and a half minutes for 80,000 unit tests—delivering the same level of confidence, but orders of magnitude faster.
This is what building quality in looks like. It’s not a one-off fix; it’s a sustained investment in shortening feedback loops and empowering engineers to own quality.
What Does This Mean for Your Team?
- Move tests as far left as possible: The closer to the engineer, the better.
- Shorten feedback loops: Identify bottlenecks—what’s taking too long? How can you make it faster?
- Automate relentlessly: Manual steps are opportunities for delay and error.
- Incremental improvement: Don’t try to fix everything at once. Tackle one static analysis rule, one flaky test, one slow build at a time.
- Empower engineers: Quality is everyone’s responsibility, not just QA’s.
This isn’t just theory. Azure DevOps itself is a tool built by a team that lived this journey, for teams who are on the same path. Whether you’re a four-person startup or a 15,000-strong enterprise with a 350GB git repo, these principles scale.
On one of my current projects, I’ve got the time from cutting code to automated build with a pull request down to about a minute and a half. Another minute and a half to preview, and another to production. That’s a feedback loop of under five minutes from code to live. The result? We catch mistakes early, fix them fast, and build trust with every release.
The Bottom Line
If you want to build quality into your process—rather than testing it in after the fact—start by designing an engineering workflow that puts feedback in the hands of your engineers, as early and as often as possible. That’s the heart of agile, Scrum, and DevOps. It’s not just about tools or ceremonies; it’s about creating a culture where quality is built in, not bolted on.
Meta Description:
Discover why building quality in—not testing it in—is essential for engineering excellence. Learn practical strategies for shifting left, shortening feedback loops, and empowering your team to deliver high-quality software, faster.
In many organizations that I work with, teams test quality in, they don’t build it into their product. And that has a very specific impact on engineering excellence within your organization.
So when you test quality in, you’re effectively allowing engineers to say, “Well, it’s okay, the testers will catch that in the next piece.” But the problem with that is the further you get from the engineer writing the code, the more expensive it is to fix, right? With production, finding something in production being the most expensive thing to fix. So even waiting until QA validation, where you’ve already gone through, you might have—working with a customer at the moment—and they have, working in a team environment. They then push that into a preview environment. And that preview environment is where the testers run. But that’s already in main. It’s already polluting main by the time the testers are getting a hold of the content.
So if we find a problem there, that’s another branch, another pull request, another change coming into the system. Now, we do need those testers to find those problems if we’ve got them, but we would rather those testers didn’t find any problems. We would rather find them sooner. And in order to do that, you need automation, you need integration, you need continuous build, you need CI/CD, you need all of those things.
So, great way to validate your architecture right from the start that you’re making code that is testable, that is just the code that you need, that it does what you expect it to do is TDD. Test-driven development is not a testing strategy. It’s not a replacement for QA. TDD is an architectural and design strategy. But you can also add on to that things like static code analysis, right?
And although you don’t want to take an existing legacy app and turn on all the static code analysis because you’ll just get a million errors and error overload and nobody will want to do anything with it, what you can do is turn on specific things and go fix those things through your product. Do a little refactor every now and again to get rid of one thing. Okay, now that’s turned on. If people get a warning, it’s because they’re doing something that we’ve gone through and fixed in our codebase. And then keep doing it a little bit at a time, turning something on in static analysis that you want to make sure that your engineers aren’t doing, fixing it throughout your application, and then having that enabled so that developers get those warnings.
Also, code review policies. Now, although—and this is where I favor trunk-based development—and long code reviews are the antithesis of trunk-based development. You want to get things into that trunk as quickly as possible. Whatever you call it—maybe main, might be trunk—but whatever you call it, you want to get things in there as quickly as possible. So you want to automate as much as possible.
I use pull requests as part of the code review policies to apply the policies, and those policies are generally automated. So most of the time, a pull request is automatically approved and automatically merged into the system, and it’s only not merged if there are infractions which need to be fixed. And one of those checks that I do have is an AI check. So AI is checking the code that we’re writing against the policies that we’ve written as a team, and flagging things that don’t meet those policies, right? So then we have to respond to those things and either we ignore them, which sometimes we do because it’s a little bit overzealous sometimes, but a lot of the times it catches things.
But it means that we don’t stop at the pull request unless there’s an actual thing that we need to go look at, right? An interrupt. So that gives us that balance between getting things into main as quickly as possible and validating that main is the right thing, that it’s good, that it’s high quality. And for your team, you would need to decide what that level is. But even catching things going into main at the pull request level is too late. We want to be catching them at the engineer level.
Shift left. Get as many things that you’re doing as close to the software engineers and as fast as possible.
I love the story of the Azure DevOps team when they were building Azure DevOps, when they first moved from a two-yearly release cycle to a three-week release cycle. One of their biggest blockers was that their full regression, their tests, their equivalent of selenium tests, were taking 24 to 48 hours to tell them whether they’d been successful. You can only run so many of them at once. So now you’re doing rolling builds and bulking up developers’ changes into this rolling test enterprise, which means that it could be 48 hours for a developer to find out that they’ve got a problem, or it could be more, right? It could be double that if you just get a change in just after the rolling build finished. It picked up the changes before yours, and now you’ve got to wait for that 48 hours to finish, and then your changes are picked up, and a further 48 hours before you find out that you’ve got something wrong.
So those types of long-running tests are not building quality in, they’re testing quality in still. That’s a lagging, severely lagging indicator from the engineer’s perspective. So they had 36,000—something like that, something like 36,000—of these long-running system tests that they would run. And they worked over a long period of time. It was over four years, paying back a little bit at a time to slowly move those tests from long-running system tests to fast-running unit tests with stubbing out, with mocking, with all of the tools that you would expect to use at that level. And some of that is using—if you build products with solid principles, with good engineering practices, then it makes it more testable, easier to do those things with.
So they started doing that, and over that four years they took it from 36 hours—48 hours, sorry—for 36,000 tests to three and a half minutes for 80,000 unit tests, and that gave them the same level of security. I’m going to put it that way. They designed it that way. Gave them the same level of security that they had with the other tests, the same level of confidence that they’d not broken anything, that they were doing the right thing, that they had that in there. So that means that fewer things were caught by that long-running tests, and eventually they could just be removed, and they no longer do that long, long step of tests because they’re confident in their shift-left test infrastructure.
That’s what we all need to do. We all need to move these tests as far left as we can, and then we need to figure out which of these things are taking too long and figure out for our application and our context, how do we shorten those feedback loops. And that’s the fundamental premise of agile, fundamental premise of scrum, fundamental premise of DevOps within all of these contexts.
Azure DevOps is a tool designed by a team that went through this journey for teams that go through and have gone through this journey. So it provides capabilities that enable those practices and enables those practices at scale. And the Windows team use this. I think it’s four and a half thousand software engineers, 15,000 people working on a git repository that’s over 350 gig at the tip. Right? These are massive projects, all the way down to, “I’ve got a little repo, I’m doing a little build, and everything can be done in a few minutes.”
I have a project I’m working on at the moment that my time from cutting code to automated build with a pull request is about a minute and a half. And then if that’s approved, another minute and a half into preview. If that’s good, another minute and a half into production. Right? That really shortens that feedback loop. You have those repeated cycles, figuring out earlier that we’ve messed it up.
So if you want to build quality into your process rather than testing it in, let’s help you design an engineering workflow that works for
