Azure DevOps Migration Tools

v14.4.7

Azure DevOps Migration Tools

Overview

Azure DevOps Migration Tools

Article

  

16 minutes to read

  

2024-02-06 08:31:18 -0600

GitHub release Build on VSTS

The Azure DevOps Migration Tools allow you to bulk edit and migrate data between Team Projects on both Microsoft Team Foundation Server (TFS) and Azure DevOps Services. Take a look at the documentation to find out how. This project is published as code on GitHub as well as a Azure DevOps Migration Tools on Chocolatey.

Ask Questions in GitHub Discussions

Some Data from the last 30 days (as of 06/04/2023)

Catagory Metric Notes
Work Item Revisions 14m A single Work Item may have many revisions that we need to migrate
Average Work item Migration Time 35s Work Item (inlcudes all revisions, links, and attachements for the work item)
RelatedLinkCount 5m Each work item may have many links or none.
Git Commit Links 480k  
Attachments 252.37k Total number of attachments migrated
Test Suits 52k total suits migrated
Test Cases Mapped 800k Total test cases mapped into Suits
Migration Run Ave 14 minutes Includes dry-runs as well.
Migration Run Total 19bn Seconds Thats 316m hours or 13m days of run time in the last 30 days.

INFO: This tool was developed to support the scenarios below, and the edge cases that have been encountered by the 30+ contributors from around the Azure DevOps community. You should be comfortable with the TFS/Azure DevOps object model, as well as debugging code in Visual Studio. Community support is available through GitHub; Paid support is available through our recommended consultants as well as our contributors and many DevOps consultants around the world.

What can you do with this tool?

  • Migrate Work Items, TestPlans & Suits, Teams, Shared Queries, & Pipelines from one Team Project to another
  • Migrate Work Items, TestPlans & Suits, Teams, Shared Queries, & Pipelines from one Organisation to another
  • Bulk edit of Work Items across an entire Project.

What you should not do with this tool?

While this tool can do the things below there are other tools and ways provided by MSFT that are way better at doing these perticular jobs!

  • Change Azure AD Tenant - If you are wanting to move your entire Org from one Azure AD tenant to another then you should follow the Microsoft documentation to Change your organization connection to a different Azure AD
  • Migrate your entire Collection to Azure DevOps Services - If you want to move your entire Collection from local TFS/Azure DevOps then you should use the Azure DevOps Migration Service provided by Microsoft. (If you need to do a Proces Template change before the migration then you can use our tool to facilitate that.)

What versions of Azure DevOps & TFS do you support?

  • Work Item Migration Supports all versions of TFS 2013+ and all versions of Azure DevOps (Many tools also work all the way back to TFS 2010 but are not supported)
  • You can move from any Tfs/AzureDevOps source to any Tfs/AzureDevOps target.
  • Process Template migration only supports XML based Projects

Typical Uses of this tool

  • Merge many projects into a single project
  • Split one project into many projects
  • Assistance in changing Process Templates
  • Bulk edit of Work Items
  • Migration of Test Suites & Test Plans
  • new Migration of Builds & Pipelines
  • Migrate from one Language version of TFS / Azure Devops to another (new v9.0)

Extras

This is a preview version of both the documentation and the Azure DevOps Migration Tools.

External Walkthroughs and Reviews

Getting the Tools

There are two ways to get these tools:

Support

  1. Question & Discussion - The first place to look for usage, configuration, and general help.
  2. Issues on Github - If you have identified a bug and have logs then please raise an issue.

Professional Support

You can get free support from the community above and on social media on a best effort basis if folks are available. If you are looking for paid support naked Agility with Martin Hinshelwood & Co has a number of experts, many of whom contribute to this project, that can help. Find out how we can help you with your migration and book a free consultation to discuss how we can make things easier.

We use these tools with our customers, and for fun, to do real world migrations on a daily basis and we can:

  • Consult with your internal folks who need help and guidance in running the tooling.
  • Make changes to the tool to support your needs; all additions are committed to the main repo.
  • Run the migration for you:- you would need to pay for the hours that we would spend baby-sitting the running migrations

## Details

These tools are build by naked Agility Limited’s DevOps & Agility consultants to do real world migrations on a daily basis. We always work in Azure DevOps Services on https://dev.azure.com/nkdagility/migration-tools/ with code in GitHub and publish as a chocolatey package that pulls from Github Releases.

Public Issues GitHub Issues
Builds & Releases Azure Pipelines
Releases Output Github Releases
Documentation Github Pages

Watch the Video Overview to get you started in 30 minutes. This tool is complicated and its not always easy to discover what you need to do.

Processors (v1 Architecture)

There are a number processors that can be used to migrate, or process, different sorts of data in different ways. Which one is right for you depends on the situation at hand. Most of these processors need to be run in order. If you try to migrate work items before you have migrated Area and Iterations then bang you need to go back.

Name Status Target Usage
CreateTeamFolders alpha Shared Queries

Creates folders in Sared Queries for each Team

ExportProfilePictureFromADContext alpha Profiles

Downloads corporate images and updates TFS/Azure DevOps profiles

ExportTeamList missng XML code comments missng XML code comments

missng XML code comments

ExportUsersForMapping missng XML code comments missng XML code comments

missng XML code comments

FakeProcessor missng XML code comments missng XML code comments

Note: this is only for internal usage. Don’t use this in your configurations.

FixGitCommitLinks missng XML code comments missng XML code comments

missng XML code comments

ImportProfilePictureContext alpha Profiles

Downloads corporate images and updates TFS/Azure DevOps profiles

TeamMigrationContext preview Teams

Migrates Teams and Team Settings: This should be run after NodeStructuresMigrationConfig and before all other processors.

TestConfigurationsMigrationContext Beta Suites & Plans

This processor can migrate test configuration. This should be run before LinkMigrationConfig.

TestPlansAndSuitesMigrationContext Beta Suites & Plans

Rebuilds Suits and plans for Test Cases migrated using the WorkItemMigration

TestVariablesMigrationContext Beta Suites & Plans

This processor can migrate test variables that are defined in the test plans / suites. This must run before TestPlansAndSuitesMigrationConfig.

WorkItemDelete ready WorkItem

The WorkItemDelete processor allows you to delete any amount of work items that meet the query. DANGER: This is not a recoverable action and should be use with extream caution.

WorkItemMigrationContext ready Work Items

WorkItemMigrationConfig is the main processor used to Migrate Work Items, Links, and Attachments. Use WorkItemMigrationConfig to configure.

WorkItemPostProcessingContext preview Work Items

Reapply field mappings after a migration. Does not migtate Work Items, only reapplied changes to filed mappings.

WorkItemQueryMigrationContext preview Shared Queries

This processor can migrate queries for work items. Only shared queries are included. Personal queries can’t migrate with this tool.

WorkItemUpdate missng XML code comments WorkItem

This processor allows you to make changes in place where we load from teh Target and update the Target. This is used for bulk updates with the most common reason being a process template change.

WorkItemUpdateAreasAsTagsContext Beta Work Item

A common issue with older TFS/Azure DevOps instances is the proliferation of Area Paths. With the use of Area Path for Teams and the addition of the Node Name column option these extensive tag hierarchies should instad be moved to tags.

Processors (v2 Architecture) [ PREVIEW ]

These are experimental processors that should replace those above. We are interested in feedback of the new format of the config, as well as the functionality.

The new processor configuration is designed to allow the Migration Tools to support different Source and targets than just TFS/Azure DevOps, and we moved the Endpoints to the processor to allow both Object Model & REST in different processors.

Name Status Target Usage
AzureDevOpsPipelineProcessor Beta Pipelines

Azure DevOps Processor that migrates Taskgroups, Build- and Release Pipelines.

ProcessDefinitionProcessor Beta Pipelines

Process definition processor used to keep processes between two orgs in sync

TfsAreaAndIterationProcessor Beta Work Items

The TfsAreaAndIterationProcessor migrates all of the Area nd Iteraion paths.

TfsSharedQueryProcessor Beta Queries

The TfsSharedQueryProcessor enabled you to migrate queries from one locatio nto another.

TfsTeamSettingsProcessor Beta Teams

Native TFS Processor, does not work with any other Endpoints.

WorkItemTrackingProcessor missng XML code comments missng XML code comments

This processor is intended, with the aid of ProcessorEnrichers, to allow the migration of Work Items between two Endpoints.

Field Maps

By default, when you are moving from source to target the system will map all of the fields that exist in source to the same field in the target. This is the behavior if the FieldMaps section is not present in the configuration file.

However sometimes you want to move data to another field, or use a regex to parse out just the bits that you want. To help we have built a number of mapping tools that should give you the versatility you need.

Name Status Target Usage
FieldClearMapConfig ready Work Item

Allows you to set an already populated field to Null. This will only work with fields that support null.

FieldLiteralMapConfig ready Work Item Field

Sets a field on the target to b a specific value.

FieldMergeMapConfig ready Work Item Field

Ever wanted to merge two or three fields? This mapping will let you do just that.

FieldSkipMapConfig ready Work Item

Allows you to skip populating an existing field. Value in target with be reset to its OriginalValue.

FieldtoFieldMapConfig ready Work Item Field

Just want to map one field to another? This is the one for you.

FieldtoFieldMultiMapConfig ready Work Item Field

Want to setup a bunch of field maps in a single go. Use this shortcut!

FieldtoTagMapConfig ready Work Item Field

Want to take a field and convert its value to a tag? Done…

FieldValueMapConfig ready Work Item Field

Need to map not just the field but also values? This is the default value mapper.

FieldValuetoTagMapConfig ready Work Item Field

Need to create a Tag based on a field value? Just create a regex match and choose how to populate the target.

MultiValueConditionalMapConfig ready Work Item Field

??? If you know how to use this please send a PR :)

RegexFieldMapConfig ready Work Item Field

I just need that bit of a field… need to send “2016.2” to two fields, one for year and one for release? Done.

TreeToTagMapConfig ready Work Item Field

Need to clear out those nasty Area tree hierarchies? This creates Tags for each node in the Area Path…

Code (TFVC)

There are no good tools for migrating TFVC code. All of them suffer from “time-dilation” as one can’t control the dates of the Check-ins. While you can use tools like TaskTop, TFS Integration Tools, or others there is no support. We prefer to recommend that you use Git-TFS and migrate from TFVC to Git with full history and branches. If your resulting repository is too large we recommend creating a full clone of TFVC for posterity, and then create a limited branch clone with limited history for work.

Code (Git)

When moving Git repos between Projects and Accounts you are able to maintain full history. However you will need to have all of the links to

  • FixGitCommitLinks - (obsolete) [this is now fixed on the fly if git is migrated first) Allows you to fix the migrated Git commit hooks (and thus external links) to point to the new repository in the target project. If the source and target repository names are the same, this will work out of the box. If the target repository has a different name, you can specify that name via the “TargetRepository” property.

Build & Releases

When you import a build or release definition into Azure DevOps you need to fill out some of the data to allow that new definition to be saved. Things like connections and other require fields usually don’t have matching GUIDS and need manual intervention. For builds & Releases we recommend that you use the built in Export/Import tools provided in the UI to move then to a new Team Project.

Why does this exist

The main migration tool for TFS has always been the TFS Integration Tools which is sadly no longer supported. Indeed it only loosely supported versions of TFS after 2010 and had a lot of bugs. It was very difficult to configure and run. This tool hopes to solve some of that by providing support for TFS 2015 and Visual Studio Team Services (VSTS).

It solves:

  • Supports all currently supported version of TFS
  • Supports Azure DevOps Services
  • Migrates work items from one instance of TFS or Azure DevOps to another
  • Bulk edits fields in place for both TFS and Azure DevOps Services
  • Being able to migrate Test Plans an Suits from one Team Project to another
  • Being able to migrate Teams from one Team Project to another

Contributing

If you wish to contribute then feel free to fork this repository and submit a pull request. If you would like to join the team please contact.

This project is primarily managed and maintained on Visual Studio Team Services and code checked into MASTER is automatically synched between VAzure DevOps and GitHub. There is no hidden published code, but not all branches are published.

If you want to sync your GitHub repository then check out Open-source with Azure DevOps or TFS and Github for better DevOps .

Events for the Team:

  • Weekly Architectural Review - Thursday 2100 GMT

If you want to be added to the community Team then please fill out this form and request access

FAQ

Check out the FAQ pages

Primary Contributors & Consultants

  • Martin Hinshelwood, naked Agility Ltd - @MrHinsh is the founder of the Azure DevOps Migration Tools is available worldwide to help organisations plan and enact their migration efforts. You can contact him through naked Agility Ltd..
  • Wes MacDonald, LIKE 10 INC. - @wesmacdonald is a DevOps Consultant located in Toronto, Canada. You can reach out to him via LIKE 10 INC..
  • Ove Bastiansen - @ovebastiansen is a DevOps consultant living in Oslo, Norway, but working worldwide in todays remote friendly world. You can reach him via his GitHub profile https://github.com/ovebastiansen.
  • Gordon Beeming - @DevStarOps is a DevOps Specialist in Durban, South Africa working at Derivco. You can reach him via his profile page that links to all social media.
  • Richard Hundhausen - @rhundhausen is an Azure DevOps consultant living in the United States and working at Accentient

Terms

naked Agility Limited & our contributors creates and maintains the “Azure DevOps Migration Tools” project under its terms of business and allows full access to the source code for customers and the general public.