When I first encountered Azure DevOps back in 2006, it wasn’t even called Azure DevOps. Back then, it was known as Visual Studio Team Services (VSTS), and before that, Team Foundation Server (TFS). Over the years, the name has changed several times, but the core intent has remained consistent: empowering teams to enhance their capabilities through powerful tools. In this blog post, I want to take you through my journey with Azure DevOps, from my early days as a software engineer to becoming a Microsoft MVP and a DevOps consultant. I’ll share personal experiences, lessons learned, and practical advice on leveraging Azure DevOps to its full potential.
Early Experience with Team Foundation Server (TFS)
A Humble Beginning
My journey with Azure DevOps began in the mid-2000s when I was a software engineer plugging away at building products. Back then, TFS was my gateway into the world of DevOps. It was a tool designed to help teams improve their processes, increase their capabilities, and streamline their workflows.
Challenges and Growth
However, TFS, in its early days, struggled to extend its reach beyond the Microsoft ecosystem. It was primarily designed for teams deeply entrenched in Microsoft technologies, limiting its versatility. Despite these challenges, I found TFS to be an invaluable tool, and my work with it played a significant role in earning my Microsoft MVP status. This recognition came from my efforts in building plugins and working with the TFS APIs, which laid the foundation for my future endeavors in the DevOps space.
Becoming a Microsoft MVP
Recognition Through Contribution
Gaining recognition as a Microsoft MVP was a pivotal moment in my career. My contributions to the TFS ecosystem, particularly in API development and consulting, earned me this prestigious title. I had the opportunity to work with teams at Merrill Lynch and Aggreko, where I helped them implement automated builds, streamline work item tracking, and integrate various tools to enhance their capabilities.
Real-World Applications
In these roles, I wasn’t just writing code; I was helping teams understand the importance of DevOps practices and how to effectively utilize TFS to achieve their goals. I built visualization tools, automated processes, and played a key role in driving the adoption of DevOps practices within these organizations.
Significant Projects and Custom Adapters
The Power of API Work
After earning my MVP, I moved to the United States and worked as a DevOps consultant, primarily within the Microsoft stack. During this time, I was heavily involved in API work, which included complex migrations and custom adapter development. One particularly challenging project involved migrating data from TFS to other systems using an old Microsoft tool with an adapter model. This experience highlighted the importance of flexibility in DevOps tools and the need for custom solutions to address unique challenges.
Creating Custom Solutions
One of my notable achievements during this period was creating a custom migrator for Test Track Pro into TFS. This project demonstrated the importance of understanding both the technical and business aspects of DevOps. It wasn’t just about moving data; it was about ensuring that the migration process aligned with the organization’s goals and workflows.
Consolidation and Optimization
As my career progressed, I started my own business and took on projects that required deep expertise in DevOps and migration strategies. One such project involved a company called Slumber, which needed to consolidate its tools and processes. Slumber’s environment was a mishmash of tools and engineering processes, and they needed to streamline their operations by adopting TFS as their primary tool.
The Role of PowerShell
In this project, I initially built a series of PowerShell scripts to prototype the migration process. PowerShell was instrumental in rapidly iterating on solutions, despite its limitations in terms of stability and scalability. These scripts allowed us to migrate data, change process templates, and move data between fields. However, as the project progressed, it became clear that a more robust and pluggable solution was needed.
Two Approaches to Migration
The migration landscape in Azure DevOps is complex, with two primary approaches to consider:
Microsoft’s Built-in Migration Tools: These tools are limited but powerful, designed to help organizations move their on-premise TFS environments to Azure DevOps in the cloud. If your environment is relatively up-to-date, this is often the simplest and most reliable method.
Custom Migration Tools (My Solution): For organizations with unique needs or outdated environments, my custom migration tools offer a more flexible solution. These tools allow for selective migration of work items and other data, providing the necessary adaptability for complex scenarios.
Real-World Case Studies
One of the most significant projects I’ve worked on involved migrating 2.7 terabytes of data for Slumber, making them one of the largest Azure DevOps customers outside of Microsoft. This massive migration was a testament to the power of Microsoft’s tools when used correctly. On the other end of the spectrum, I’ve also handled smaller migrations, where the process was completed in a matter of hours, showcasing the scalability of Azure DevOps.
Practical Advice for Migration
Planning and Execution
When planning a migration to Azure DevOps, it’s crucial to:
Assess your environment: Determine the current state of your TFS environment, including the version and any dependencies.
Choose the right migration path: Decide whether to use Microsoft’s built-in tools or opt for a custom solution.
Prepare for complexity: Be aware that migrations often involve multiple stages, especially if you’re dealing with older versions of TFS.
Custom Solutions for Complex Needs
For organizations with unique requirements, such as partial migrations or splitting projects, custom tools like the ones I’ve developed can be invaluable. These tools allow for a more granular approach, enabling organizations to move only the necessary components while maintaining the integrity of their data.
Training and Consulting Services
Empowering Teams
In addition to providing migration tools, I offer training and consulting services to help organizations make the most of Azure DevOps. Whether it’s a one-off migration or ongoing support, my goal is to empower teams to leverage the full capabilities of Azure DevOps.
Training: I provide customized training sessions for teams, ensuring they have the skills needed to manage their environments effectively.
Consulting: For organizations facing complex challenges, I offer consulting services to help them navigate the intricacies of Azure DevOps and develop tailored solutions.
Real-World Applications
One of the most rewarding aspects of my work is seeing the impact of these tools and services in real-world scenarios. For example, I recently completed a migration of 880,000 work items for a customer, enabling their audit department to seamlessly transition to a new system. This project not only demonstrated the scalability of Azure DevOps but also underscored the importance of having the right expertise on hand for such critical tasks.
Conclusion and Ongoing Support
A Continuous Journey
As both a user and consultant for Azure DevOps, I’ve witnessed the platform’s evolution and its impact on organizations of all sizes. While my work has increasingly shifted to GitHub, I still hold a deep appreciation for the capabilities Azure DevOps offers. However, as powerful as these tools are, there are always gaps that require custom solutions or expert guidance.
Final Thoughts
Whether you’re planning a migration, seeking to optimize your DevOps processes, or simply need advice on the best approach for your organization, remember that Azure DevOps is a versatile platform with the potential to transform how you work. But like any tool, its effectiveness depends on how well you understand and leverage its capabilities. So, take the time to explore your options, seek out the right expertise, and make informed decisions that align with your business goals. By sharing my journey and insights, I hope to inspire others to explore the full potential of Azure DevOps and embrace the continuous journey of improvement that DevOps embodies. 🚀
So I first encountered Azure DevOps back in 2006, so that’s quite some time ago. It wasn’t called Azure DevOps back then; it was Visual Studio Team Services back in the day. It’s gone through many name changes since then, but effectively it’s not the same service, but it’s the same intent from the service.
Azure DevOps is something that I’ve been using since then, and I was a lowly software engineer, plugging away building some products. Azure DevOps was really my route into DevOps, into helping teams become better at what they do, increase their capabilities, and use tools to help them better their capabilities. Team Foundation Server was intending to be that tool. I think it largely struggled to do anything outside of the Microsoft space until they moved much further on to Azure DevOps, the cloud version.
That early product was actually how I got my Microsoft MVP as well, which was building plugins and playing around with the APIs for TFS. I worked with a number of teams, some teams at Merrill Lynch and then some teams at Agreco. In both circumstances, while also doing the work, I helped those teams understand how to do automated builds, how to use the work item tracking, and how to link all the things together. I built some visualisation tools against the APIs, so I’ve really been plugging away at the Azure DevOps APIs since 2006.
I know where a lot of the bodies are buried in the APIs, which is why when I think what happened, I got my MVP, I moved to the US, and I worked as a DevOps consultant almost exclusively in the Microsoft stack for three years. I was doing a lot of Azure DevOps work. We did a lot of API work; there were some migrations. There used to be this old tool that was totally crazy to work with. I’m trying to remember what it was called. Microsoft created it originally, but really it was just a big code base, and it allowed you to migrate data from TFS to other TFS. It had this kind of adapter model so that you could get from other systems into TFS; that was the idea.
I think I’m the only one—I would hedge; I’m certainly the only one that published creating their own adapter for it. I created a migrator for Test Track Pro into TFS back in the day when Test Track Pro was even a thing, and maybe it is still a thing; I’ve just not heard about it in 15 or 20 years. That experience of creating that adapter really led me, when I arrived in the UK, to start my own business. One of my first gigs was with a company called Slumber, and Slumber was consolidating. They needed a lot of DevOps help to make the most out of the tools that they had. They were using the tools really well; it was just a little bit convoluted.
They wanted to take—they’re the sort of organisation where everybody does everything, right? So there’s every tool you can think of, every engineering process; they have everything. They wanted to consolidate down into using at least a single source control tool with work item tracking, and the tool they picked was TFS, Team Foundation Server. I helped them migrate something, and they had a standard process template, so I helped them migrate something like four to five hundred teams from what they were doing before into this new model and then move them from server to server.
Originally, I built a bunch of PowerShell scripts. That’s when you’re prototyping; scripting languages are the best way to run your thought processes. You’re effectively writing down your thought process and then running it and saying, “Yeah, that kind of works.” If I select this bit, that kind of works, and if I do this, that kind of works. So it’s a really good way to do some fast prototyping. It’s incredibly error-prone and buggy as heck, but it’s a good way to do those things.
I built a bunch of PowerShell tools that I had, I think at one point, four or five people else in the organisation running those PowerShells to start migrating data around, changing process templates, moving data around in fields, all of those things. Then I started looking at, “Well, really, this needs to be a little bit more solid; this needs to be a little bit more capable; it needs to be something that’s a little bit more pluggable.” I built the first version of the Azure DevOps migration tools.
There are kind of two ways to do migrations in Azure DevOps. One is really limited but really powerful; the other one is mine. The one that’s really limited and really powerful is Microsoft’s own migration tools. Obviously, they want as many people as possible to move into the cloud. They don’t really want to be supporting and maintaining Team Foundation Server or Azure DevOps Server, the on-prem version of the product.
They want people to move up to the cloud, so they’ve built a tool that if you get onto almost the latest version of Azure DevOps on-prem, you can effectively package up your database, give it to them. There’s a whole process for that, but package up your database. It’s not as easy as it sounds. Package up your process, your database, give it to them, and they’ll import it into Azure DevOps.
So if you’re trying to get into Azure DevOps, the easiest, most high-fidelity, most productionised way to do that is to have Microsoft do it. They provide a bunch of tools. I’ve done this for a bunch of customers over the years, tons and tons of customers. The biggest one was the Slumber engagement, where I think it was 2.7 or 2.8 terabytes was the collection that we moved up to Azure DevOps. It was pretty ginormous. I think they became the third largest customer using Azure DevOps outside of Microsoft in the service.
That was a huge amount of work, but lots of smaller ones are much easier. I did one for a customer where we did it in about an hour. Their database was small enough; they were on the latest version; they had control of all the servers, the passwords, all the things. It was small enough package; we just sequenced it up, and then they looked at the result and went, “Yep, that’s good; let’s go with that,” and we turned off the on-prem version.
I have personally done a whole bunch of migrations. I have a team that helps with that. We did one recently that was more complicated than it sounds. You’re not just taking the Azure DevOps database and putting it in the cloud. Microsoft have to process it. You have to be on a version that’s supported by that process, which is the absolute latest version and one service pack back.
This means that if you’re sitting with a local Team Foundation Server and you’re on TFS 2010, 2012, 2015, or whatever else was there, you have to upgrade it first, which means that you might need to upgrade SQL Server, you might need to upgrade the operating system, you might have to do all of those things. We did one recently for a customer that was TFS 2010, I think they were on 2010, all the way up to Azure DevOps.
There were three distinct migrations we had to do because you’ve got to upgrade from 2010 to 2015. Then you’ve got to go from 2015 to, I think you can get to the latest version from 2015 up to the latest version of TFS. Then you have to run and validate all the things that you need to do to go to the cloud. Then you can take your environment into the cloud, and that can go one of two ways. You can just take the database if it’s small enough; if it’s under 150 meg, you can just give them the database that pack, and it’ll work.
Otherwise, you have to set up an Azure environment; you have to install the SQL Server, upload data, and then get Microsoft to do stuff with it. That is the easiest, simplest, and most productionised route, but it’s not what a lot of people can or want to do. Lots of companies have lots of circumstances that are outwith the bounds of that. They’ve got 100 teams, and only five teams want to go to the cloud. They’ve got other teams that can’t for whatever reason, and that then adds complexity to that mix.
They’ve got teams that want to go just now and other teams that want to go later. Those are the most common ones for that push up into Azure DevOps. But also, you’ve got companies that sell products. If you’ve got five products in your company and one of those products is being sold, it’s probably inside of the same TFS or Azure DevOps, and you want to move that piece over to somewhere else because you’re selling it to somebody, and they’re going to take the history and all of the stuff that goes along with it.
Microsoft provide no support for any of those scenarios. If you’ve got 50 projects and you want one, you’ve got projects across five collections, and you want one collection at one count. Or I’ve got, I don’t know, I created one big massive project, and I really should have created two projects. These are things that are just not fundamentally supported by the tool or in any productionised way by Microsoft.
What my tool that I created does is it allows you to migrate pieces of your whole thing to another location. We do our query-based, so it’s predominantly focused on work items, although there’s lots of other tools in there. It’s like a Swiss Army knife, and people are adding new tools all the time. But the main tool that you use is work item migration because that’s the thing that’s just not out there.
You’re basically saying, “Here’s a query; I want these work items, and I want them recreated over here,” and the tool goes and recreates them. There are lots of limitations, lots of caveats, lots of things around that, but that’s effectively the idea. I quite often get asked to help customers in a variety of different ways. Sometimes they want to do the migration if they’ve got loads of migrations to do.
I’ve got people out there that have been using my tools for years, and it makes sense if you’re going to use the tools. Big enterprise customers find quite often they need to move data from one place to another. They want to consolidate; they want to split, so they spin up a group of people, probably the administrators of Azure DevOps for their company, that they want to spin up on the tool and use the tool.
I provide them with a bunch of support. Sometimes it’s just support calls; sometimes it’s training. They want to train some people on using the tool. But more often than not, with lots of organisations, it’s a one-off thing. We’re going here; we want this to happen, and we just want it to happen. We don’t want to have to—we don’t need that skill in-house. If it’s not your core business, you’re not doing it all the time, and you don’t need it to support your core business, then that’s when you outsource.
I did a big migration recently of about 80,000 work items for a customer that wanted an audit trail. They’ve bought another company; they’ve not got their hands on all of that company’s stuff yet. There’s still all the legal stuff that has to happen in between, but they want a copy of all the work items and all the code so that their audit department can then go through all of that and validate and check and match things up.
Yes, this is the right thing, and all this is missing, and that’s missing. I don’t know what they’re doing right, but they wanted that copy, so they don’t want to mess around having their own people learn a tool and then do it, so we just did it for them. It probably took eight or nine days to migrate actual 24 hours running on it to migrate that number of work items because it’s a lot.
But then after it’s done, they have that copy in the new location, and they can do their own thing. Sometimes we do the migrations; sometimes we help support the migrations; sometimes we help train people, and sometimes we just consult. I quite often get calls from folks that just need some advice on how to do something in Azure DevOps.
These big infrastructure ideas that people have—the business decides they want to consolidate stuff, they want to split stuff, they want to move stuff around—like, what does that mean for us? What are our options? What are our capabilities? I quite often have discussions with customers on how that all goes together, and sometimes they use my services to help them, and sometimes they have enough information to go do it on their own, or it’s easy enough that it’s just following the docs from Microsoft.
So as both a user of Azure DevOps, which I have been for many years, although not so much anymore, I do a lot of my work in GitHub now. But as a user of Azure DevOps, I really appreciate the capabilities that it has. But then there are some gaps that you need somebody to fill, and sometimes you want to spin it up yourself, and sometimes you need to kind of get somebody in to help you out. So that’s kind of my experience with Azure DevOps.
