https://nkdagility.com/resources/XhPHa1RXrWo copied to clipboard!

Product Development

tech·nic·al·ly agile

Agile is Not New: A Journey Back to the Basics

Explore Agile’s rich history and its evolution in this insightful video by Martin Hinshelwood, revealing how to reclaim its true value in modern business. 🌍💡

Published on 10 October 2024

Written by Martin Hinshelwood

6 minute read

https://nkdagility.com/resources/XhPHa1RXrWo

Agile isn’t some trendy buzzword that just came out of nowhere. In fact, it’s far from new. It’s a concept that has roots stretching back centuries. Before the Industrial Revolution, small, local businesses—like your neighborhood cobbler—knew their customers on a personal level. They crafted products based on the unique needs and preferences of each individual. This personal connection, built on understanding and empathy, was central to how work got done.

The Loss of Human Connection in Business

Back then, businesses weren’t about mass production. They were about understanding the needs of individuals.
Your local cobbler didn’t just make shoes; they made your shoes because they knew your story.
As businesses scaled up, that personal connection faded, replaced by efficiency and standardization.

Many blame Frederick Winslow Taylor for this shift. He wasn’t trying to strip the humanity out of work; he aimed for efficiency. But the way his ideas were interpreted led to work environments that felt mechanical and dehumanized.

The Impact of Taylorism on Modern Work

Taylor’s ideas focused on productivity and breaking work into smaller tasks.
While it revolutionized industries, it came with a cost: the loss of human connection in the workplace.
As companies adopted these principles, they moved further from the personalized, adaptive ways of working that had existed before.

Agile, in many ways, is a return to those roots—a response to the rigidity that Taylorism brought. It’s about figuring things out as you go, learning from mistakes, and adapting to change. This approach mirrors how humans have navigated challenges for centuries.

The Evolutionary Story of Adaptation

Humans have always adapted to their environment, learning by doing.
Those who adapted to new tools, like farming equipment, thrived.
Those who resisted change struggled to survive.

This idea of adapting and evolving isn’t just about survival; it’s the foundation of Agile. It’s how we’ve always gotten things done—experimenting, failing, learning, and improving. So, while Agile might sound like a modern trend, it’s really just a new name for an age-old approach.

Agile’s Resurgence in the ’90s: The Birth of XP

The ’90s marked a turning point for Agile, especially in the software industry. The traditional ways of building software were failing—teams were drowning in complexity, struggling with waste, and producing low-quality products. Enter Extreme Programming (XP), which brought Agile principles into the spotlight.

Why XP and Agile Gained Traction

High levels of waste: Traditional methods were inefficient.
Hateful work environments: Many, like Ken Schwaber, described it as “hateful work.” Employees, managers, and even leaders were frustrated.
Complexity was out of control: Teams were trying to control chaos, often with disastrous results—like trying to tame a wild lion.

I still remember starting out in the software industry, hearing about these so-called Death March projects. Everyone knew these projects were doomed from the start—yet we were expected to push through because “that’s just how it is.”

The Challenge of Death March Projects

Dead on arrival: Projects that everyone knows won’t succeed.
Unrealistic expectations: Timelines, budgets, and feature lists that were never achievable.
Motivation killer: Team morale plummets when everyone knows they’re working on a losing battle.

It was in this environment of frustration and failure that the Agile Manifesto emerged in 2001, bringing a new way of working that focused on people, empathy, and collaboration.

The Agile Manifesto: A New Way of Working

The Agile Manifesto didn’t lay down strict rules. Instead, it suggested a new way of thinking—a shift from rigid processes to a focus on human interactions and adapting to change. It aimed to bring back the social aspects of work that Mary Parker Follett spoke about nearly a century earlier.

Key Principles of the Agile Manifesto

Embrace complexity: Accept that you can’t control everything.
Focus on people: Build products with empathy and a deep understanding of user needs.
Iterate and adapt: Learn from feedback and keep improving.

The intent was to create an environment where teams could thrive, build valuable products, and adapt quickly to change. However, there was an underlying assumption that has since proven problematic.

The Competence Gap in Agile Roles

When the Agile movement took off, it assumed a certain level of craftsmanship and competence from its participants. But the demand for skilled engineers, product managers, and leaders who understood Agile far exceeded the supply. Even today, this competence gap remains.

Why Competence Matters in Agile

Engineering: It’s not just about writing code; it’s about delivering value.
Product Management: Understanding user needs and translating them into valuable features.
Leadership: Guiding teams through uncertainty, focusing on outcomes over outputs.

When I entered the workforce in 2001, there was a rush to fill these roles, but the focus wasn’t on quality. As a result, many organizations hired people who didn’t fully grasp what Agile was supposed to achieve. The outcome? A watered-down version of Agile that didn’t live up to its potential.

The Competence Crisis: A Struggle for Value

Today, many organizations are facing a competence crisis. They’ve invested in Agile practices, hired Scrum Masters and coaches, but they aren’t seeing the return on investment they expected.

The Symptoms of the Competence Crisis

High salaries, low impact: Companies pay top dollar for Agile roles but don’t see the results.
Lack of leadership skills: Many leaders in Agile roles don’t understand how to guide teams through complexity.
Misaligned expectations: Organizations expect immediate results, but Agile requires patience and continuous improvement.

This gap between expectation and reality is forcing many companies to rethink their Agile journey. They’re starting to see that competence is the missing ingredient, and without it, their Agile transformation is likely to fail.

The Commercialization of Agile: A Reset is Coming

What started as a movement to bring back humanity and adaptability to work has, in many ways, become a commercialized, watered-down version of itself. But I believe a reset is on the horizon—a return to the original principles of Agile, where competence, empathy, and adaptability take center stage.

Signs of the Reset

Organizations demanding more: They want value for their investment and are no longer satisfied with superficial Agile transformations.
A focus on real skills: Companies are starting to prioritize competence in their hiring and training.
Back to basics: A shift towards pragmatic Agile that focuses on what truly works rather than following a rigid framework.

Agile isn’t just about ceremonies and rituals. It’s about creating environments where teams can experiment, learn, and adapt. It’s about bringing back that human connection that we lost in the drive for efficiency. And most importantly, it’s about being ready to evolve—because that’s what humans have always done.

Final Thoughts: Building a Competent Agile Future

Agile isn’t new, but our understanding of it continues to evolve. As we move forward, it’s critical that we focus on the competence of those driving Agile transformations. Here’s my advice for anyone on this journey:

Invest in continuous learning: Whether you’re a Scrum Master, engineer, or product manager, keep building your skills.
Embrace feedback: The best Agile practitioners are those who listen, adapt, and improve.
Stay true to the principles: Agile is about people, not processes. Always keep that in mind.

The future of Agile lies in our ability to learn from the past, adapt to the present, and build a path forward that’s grounded in competence, empathy, and a genuine desire to create value. 🚀 Ready to embrace the next chapter of Agile? It’s time to reset, refocus, and build a future that stays true to the heart of what Agile is all about.

Agile is not a new thing by any means. One could absolutely argue that it is a very old thing. Prior to the Industrial Revolution, although it’s not totally to do with that, things were done in small, Mom and Pop shops all over the world. Your local cobbler made your shoes, and your local cobbler understood fundamentally your needs as an individual because they knew who you were. They knew your life story, and they created goods just for you. As we scaled up, we lost that human connection between things, and a lot of people blame Frederick Winslow Taylor. He’s not strictly at fault; it wasn’t his intent to dehumanise the work environment. It was just the outcome of the practices that he pushed and how they were interpreted.

Agile is a thing that has always been true for most of human existence. The way you get stuff done is you figure it out. If you don’t know how to do something, you go try some stuff and figure out what works best and what doesn’t. Ultimately, it’s an evolutionary story because the people in the world that were able to do things better perhaps survived, and those that were not able to do it did not. Those that adapted to farming and new farm equipment and were able to increase the yield of the crop prospered. Those that weren’t able to do that or couldn’t figure it out did not. That’s fundamentally how evolutionary theory works and how humans interact with the world around us. If we’re successful, we do well at something; if we don’t do well, we’re not.

So, agile’s not a new thing, but even as far back as the late 1900s, they were talking about harnessing the social aspects of work in order to maximise our ability to deliver value. If you want to look up some of the theory and stories around that, it’s Maryanne Fetter you want to be looking up. She was a contemporary and perhaps a little bit in opposition to Frederick Winslow Taylor at the time, and her work was largely lost by the 1930s. However, there was really a resurgence of this idea. The thing with good ideas is they’ll keep coming back even if we forget them.

In the ’90s, XP came along in the software world because it was very obvious that it wasn’t working. We weren’t able to successfully and repeatedly build high-quality, valuable products without copious amounts of waste. We had lots of unhappy people. I think Ken Schwaber talks about it as “hateful work.” We hated the companies we worked for; we hated the work that we were doing. The managers hated the work that they were doing. Everybody just hated it because we were dealing with complexity, and everything would always go wrong. When you try to control chaos, much like a lion tamer, you might get eaten. In fact, the likelihood is that you’re going to get eaten at some point if you keep trying to do it that way, and that happened in project after project.

When I started out in the software industry, right when I graduated university, there was talk of this idea of a “death march.” A death march project is a project that every single person on it already understands before we hardly started that it is dead on arrival. It’s not going anywhere; it’s not going to be successful. We’re not going to be able to do this. It’s not going to cost what they think it’s going to cost; it’s not going to be done in that time; it’s not going to have the features that they think it’s going to have. But we have to do it anyway because that’s what we’re paid to do, and somebody bean counter higher up in the organisation has made the decision.

This resurgence culminated in the ’90s with XP, which was really the start of the bigger resurgence, although it had been mulling around before that. Then Scrum eventually culminated in the Agile Manifesto in 2001. The manifesto really defined the principles for a new way of working, one that focused on dealing with complexity and the unknown with empathy and with connection to people, bringing that social aspect that Maryanne Fetter wrote about nearly 100 years before.

However, it was predicated on competence. There was an assumption of the inherent craftsmanship and competence of the participants that we were going to build on. However, the demand, especially when I graduated university in 2000 and started my first job in 2001, for engineers, product managers, and leaders that had both the competence and understood this new way of working far outstripped the supply in the market. It still does. Part of the reason it still does is that we didn’t focus on the competence of who we needed to do these things to make these changes and fundamentally support them from a leadership perspective, from an engineering perspective, and from our goals and discovery.

Because we didn’t focus on that level of competence, it’s largely starting to fall apart. That realisation is not quite like the dot-com bubble bursting; that was a money thing. This is a competence thing. Of the hundreds of thousands of people that are currently in roles where they’re expected to either participate in craft or oversee, the level of competence is a tiny percentage of people in that space. It’s starting to show in organisations where they are looking for what they are getting for their money because we’re paying perhaps a lot of high-price salaries for engineers, product managers, and leadership. We’re not getting the return on investment that we’re expecting, and we need to get the return on investment; otherwise, it’s not worth spending that money. That’s how business functions in our world.

If we can’t justify the cost, we’re going to have to get rid of the thing that’s costing us, and at the moment, it looks like it’s not the engineers and not the product managers, but it’s the leaders, the Scrum Masters, the coaches—those folks that just don’t necessarily have the level of competence that’s demanded to fulfil the role and be successful. They may have had enough to convince the people that were hiring them that didn’t necessarily understand what they needed, but as it turns out, they couldn’t do it.

It’s systemic at the moment, and that’s what we’re seeing in the industry. There’s a wide-ranging general reset of our understanding of what these competencies are, and most people are coming up lacking. That’s kind of what we intended to do when we were originally talking about agile back in 2001. The outcome has not been that; it’s been a commercialised, watered-down lip service-based result, and that’s where we’re at right now.

Competence Agile Philosophy Agile Product Management Software Development Agile Planning Business Agility Pragmatic Thinking Agile Values and Principles Agile Transformation Agile Leadership … 7 more Continuous Learning Organisational Change Organisational Culture Sociotechnical Systems Complexity Thinking Market Adaptability Personal

Connect with Martin Hinshelwood

If you've made it this far, it's worth connecting with our principal consultant and coach, Martin Hinshelwood, for a 30-minute 'ask me anything' call.

Our Happy Clients

We partner with businesses across diverse industries, including finance, insurance, healthcare, pharmaceuticals, technology, engineering, transportation, hospitality, entertainment, legal, government, and military sectors.

CR2