Azure DevOps Migration Tools

v16.0.4

Azure DevOps Migration Tools allow you to migrate Teams, Backlogs, Tasks, Test Cases, and Plans & Suits from one Project to another in Azure DevOps / TFS both within the same Organisation, and between Organisations.

Home
 › 
Learn
 › 
Overview
 › 
Reference
Overview

Reference

Article

  

20 minutes to read

  

2024-10-03 07:48:10 +0000

Azure DevOps Migration Tools are mainly powered by configuration which allows you to control most aspects of the execution flow. This page will guide you through the configuration options available to you.

Creating a Configuration File

The easyest way to get started with the configruation is to create both a reference file, thats one with everything, and a minimal file that will get you started using the tool.

Creating a Reference File

If you run devlopsmigration init --options Reference -c configuration-ref.json the output should be a file with all posible options using the Sample data.

Creating a Minimal File

To get a file that you can use its best to start with a minimal file. If you run devlopsmigration init --options Basic --overwrite thsi will create that minimal file.

Other options

Right now we support:

  • Reference - Create the reference file with everything
  • *Basic WorkItemTracking* - Create a minimal file with just the WorkItemTracking processor enabled
  • PipelineProcessor - Create a minimal file with just the PipelineProcessor enabled

Note: Azure DevOps Migration Tools do not ship with internal default configuration and will not function without one.

To create your config file just type devlopsmigration init in the directory that you unzipped the tools and a minimal configuration.json configuration file will be created. Modify this as you need.

Note that the generated file show all the possible options, you configuration file will probably only need a subset of those shown.

Global configuration

The global configuration created by the init command look something like this:

{
  "Serilog": {
    "MinimumLevel": "Debug"
  },
  "MigrationTools": {
    "Version": "16.0",
    "Endpoints": {
      "Source": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/nkdagility-preview/",
        "Project": "migrationSource1",
        "AllowCrossProjectLinking": false,
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "",
          "NetworkCredentials": {
            "UserName": "",
            "Password": "",
            "Domain": ""
          }
        },
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      },
      "Target": {
        "EndpointType": "TfsTeamProjectEndpoint",
        "Collection": "https://dev.azure.com/nkdagility-preview/",
        "Project": "migrationTest5",
        "TfsVersion": "AzureDevOps",
        "Authentication": {
          "AuthenticationMode": "AccessToken",
          "AccessToken": "",
          "NetworkCredentials": {
            "UserName": "",
            "Password": "",
            "Domain": ""
          }
        },
        "ReflectedWorkItemIdField": "nkdScrum.ReflectedWorkItemId",
        "AllowCrossProjectLinking": false,
        "LanguageMaps": {
          "AreaPath": "Area",
          "IterationPath": "Iteration"
        }
      }
    },
    "CommonTools": {
      "WorkItemTypeMappingTool": {
        "Enabled": true,
        "Mappings": {
          "User Story": "Product Backlog Item"
        }
      },
      "StringManipulatorTool": {
        "Enabled": true,
        "MaxStringLength": 1000000,
        "Manipulators": [
          {
            "$type": "RegexStringManipulator",
            "Enabled": true,
            "Pattern": "[^( -~)\n\r\t]+",
            "Replacement": "",
            "Description": "Remove invalid characters from the end of the string"
          }
        ]
      },
      "TfsAttachmentTool": {
        "RefName": "TfsAttachmentTool",
        "Enabled": true,
        "ExportBasePath": "c:\\temp\\WorkItemAttachmentExport",
        "MaxRevisions": 480000000
      },
      "TfsChangeSetMappingTool": {
        "Enabled": true,
        "File": "C:\\temp\\ChangeSetMappingFile.json"
      },
      "FieldMappingTool": {
        "Enabled": true,
        "FieldMaps": [
          {
            "FieldMapType": "FieldtoFieldMap",
            "ApplyTo": [ "SomeWorkItemType" ],
            "sourceField": "System.AcceptanceCriteria",
            "targetField": "System.AcceptanceCriteria2"

          },
          {
            "FieldMapType": "FieldtoFieldMap",
            "ApplyTo": [ "SomeWorkItemType" ],
            "sourceField": "System.Description",
            "targetField": "System.Description2"

          }
        ]
      },
      "GitRepoMappingTool": {
        "Enabled": true,
        "Mappings": {
          "Source Repo Name": "Target Repo Name"
        }
      },
      "TfsNodeStructureTool": {
        "Enabled": true,
        "Areas": {
          "Filters": [ " *\\Team 1,*\\Team 1\\**" ],
          "Mappings": {
            "^Skypoint Cloud([\\\\]?.*)$": "MigrationTest5$1",
            "^7473924d-c47f-4089-8f5c-077c728b576e([\\\\]?.*)$": "MigrationTest5$1"
          }
        },
        "Iterations": {
          "Filters": [],
          "Mappings": {
            "^Skypoint Cloud([\\\\]?.*)$": "MigrationTest5$1",
            "^7473924d-c47f-4089-8f5c-077c728b576e([\\\\]?.*)$": "MigrationTest5$1"
          }
        },
        "ShouldCreateMissingRevisionPaths": true,
        "ReplicateAllExistingNodes": true
      },
      "TfsRevisionManagerTool": {
        "Enabled": true,
        "ReplayRevisions": true,
        "MaxRevisions": 0
      },
      "TfsTeamSettingsTool": {
        "Enabled": true,
        "MigrateTeamSettings": true,
        "UpdateTeamSettings": true,
        "MigrateTeamCapacities": true,
        "Teams": [ "Team 1", "Team 2" ]
      }
    },
    "Processors": [
      {
        "ProcessorType": "TfsWorkItemMigrationProcessor",
        "Enabled": true,
        "UpdateCreatedDate": true,
        "UpdateCreatedBy": true,
        "WIQLQuery": "SELECT [System.Id] FROM WorkItems WHERE [System.TeamProject] = @TeamProject AND [System.WorkItemType] NOT IN ('Test Suite', 'Test Plan','Shared Steps','Shared Parameter','Feedback Request') ORDER BY [System.ChangedDate] desc",
        "FixHtmlAttachmentLinks": false,
        "WorkItemCreateRetryLimit": 5,
        "FilterWorkItemsThatAlreadyExistInTarget": false,
        "PauseAfterEachWorkItem": false,
        "AttachRevisionHistory": false,
        "GenerateMigrationComment": true,
        "WorkItemIDs": [ 12 ],
        "MaxGracefulFailures": 0,
        "SkipRevisionWithInvalidIterationPath": false,
        "SkipRevisionWithInvalidAreaPath": false
      }
    ]
  }
}

Anatomy of the Configuration File

Endpoints

Name Status Target Usage
AzureDevOpsEndpoint missing XML code comments missing XML code comments

missing XML code comments

FileSystemWorkItemEndpoint missing XML code comments missing XML code comments

missing XML code comments

TfsEndpoint missing XML code comments missing XML code comments

missing XML code comments

TfsTeamProjectEndpoint missing XML code comments missing XML code comments

missing XML code comments

TfsTeamSettingsEndpoint missing XML code comments missing XML code comments

missing XML code comments

TfsWorkItemEndpoint missing XML code comments missing XML code comments

missing XML code comments

Processors

Name Status Target Usage
AzureDevOpsPipelineProcessor Beta Pipelines

Azure DevOps Processor that migrates Taskgroups, Build- and Release Pipelines.

KeepOutboundLinkTargetProcessor missing XML code comments missing XML code comments

missing XML code comments

OutboundLinkCheckingProcessor missing XML code comments missing XML code comments

missing XML code comments

ProcessDefinitionProcessor Beta Pipelines

Process definition processor used to keep processes between two orgs in sync

TfsExportProfilePictureFromADProcessor alpha Profiles

Downloads corporate images and updates TFS/Azure DevOps profiles

TfsExportUsersForMappingProcessor ready Work Items

ExportUsersForMappingContext is a tool used to create a starter mapping file for users between the source and target systems. Use ExportUsersForMappingConfig to configure.

TfsImportProfilePictureProcessor alpha Profiles

Downloads corporate images and updates TFS/Azure DevOps profiles

TfsSharedQueryProcessor Beta Queries

The TfsSharedQueryProcessor enabled you to migrate queries from one locatio nto another.

TfsTeamSettingsProcessor Beta Teams

Native TFS Processor, does not work with any other Endpoints.

TfsTestConfigurationsMigrationProcessor Beta Suites & Plans

This processor can migrate test configuration. This should be run before LinkMigrationConfig.

TfsTestPlansAndSuitesMigrationProcessor Beta Suites & Plans

Rebuilds Suits and plans for Test Cases migrated using the WorkItemMigration

TfsTestVariablesMigrationProcessor Beta Suites & Plans

This processor can migrate test variables that are defined in the test plans / suites. This must run before TestPlansAndSuitesMigrationConfig.

TfsWorkItemBulkEditProcessor missing XML code comments WorkItem

This processor allows you to make changes in place where we load from teh Target and update the Target. This is used for bulk updates with the most common reason being a process template change.

TfsWorkItemDeleteProcessor ready WorkItem

The WorkItemDelete processor allows you to delete any amount of work items that meet the query. DANGER: This is not a recoverable action and should be use with extream caution.

TfsWorkItemMigrationProcessor ready Work Items

WorkItemMigrationConfig is the main processor used to Migrate Work Items, Links, and Attachments. Use WorkItemMigrationConfig to configure.

TfsWorkItemOverwriteAreasAsTagsProcessor Beta Work Item

A common issue with older TFS/Azure DevOps instances is the proliferation of Area Paths. With the use of Area Path for Teams and the addition of the Node Name column option these extensive tag hierarchies should instad be moved to tags.

TfsWorkItemOverwriteProcessor preview Work Items

Reapply field mappings after a migration. Does not migtate Work Items, only reapplied changes to filed mappings.

WorkItemTrackingProcessor missing XML code comments missing XML code comments

This processor is intended, with the aid of ProcessorEnrichers, to allow the migration of Work Items between two Endpoints.

Common Tools

Name Status Target Usage
FieldMappingTool missing XML code comments missing XML code comments

missing XML code comments

StringManipulatorTool missing XML code comments missing XML code comments

Used to process the String fields of a work item. This is useful for cleaning up data. It will limit fields to a max length and apply regex replacements based on what is configured. Each regex replacement is applied in order and can be enabled or disabled.

TfsAttachmentTool missing XML code comments missing XML code comments

missing XML code comments

TfsChangeSetMappingTool missing XML code comments missing XML code comments

missing XML code comments

TfsEmbededImagesTool missing XML code comments missing XML code comments

missing XML code comments

TfsGitRepositoryTool missing XML code comments missing XML code comments

missing XML code comments

TfsNodeStructureTool missing XML code comments missing XML code comments

The TfsNodeStructureToolEnricher is used to create missing nodes in the target project. To configure it add a TfsNodeStructureToolOptions section to CommonEnrichersConfig in the config file. Otherwise defaults will be applied.

TfsRevisionManagerTool missing XML code comments missing XML code comments

The TfsRevisionManagerTool manipulates the revisions of a work item to reduce the number of revisions that are migrated.

TfsTeamSettingsTool missing XML code comments missing XML code comments

The TfsUserMappingTool is used to map users from the source to the target system. Run it with the ExportUsersForMappingContext to create a mapping file then with WorkItemMigrationContext to use the mapping file to update the users in the target system as you migrate the work items.

TfsUserMappingTool missing XML code comments missing XML code comments

The TfsUserMappingTool is used to map users from the source to the target system. Run it with the ExportUsersForMappingContext to create a mapping file then with WorkItemMigrationContext to use the mapping file to update the users in the target system as you migrate the work items.

TfsValidateRequiredFieldTool missing XML code comments missing XML code comments

missing XML code comments

TfsWorkItemEmbededLinkTool missing XML code comments missing XML code comments

missing XML code comments

TfsWorkItemLinkTool missing XML code comments missing XML code comments

missing XML code comments

WorkItemTypeMappingTool missing XML code comments missing XML code comments

Used to process the String fields of a work item. This is useful for cleaning up data. It will limit fields to a max length and apply regex replacements based on what is configured. Each regex replacement is applied in order and can be enabled or disabled.

And the description of the available options are:

TelemetryEnableTrace

Allows you to submit trace to Application Insights to allow the development team to diagnose any issues that may be found. If you are submitting a support ticket then please include the Session GUID found in your log file for that run. This will help us find the problem.

Note: All exceptions that you encounter will surface inside of Visual Studio as the developers are working on the source. This will make sure that they tackle issues as they arise.

Source & Target

Both the Source and Target entries hold the collection URL and the Team Project name that you are connecting to. The Source is where the tool will read the data to migrate. The Target is where the tool will write the data.

For multi Language support you can add the name used in the source and target for both ‘Area’ and ‘Iteration’. This allows a migration from one language version of TFS / Azure DevOps to another.

ReflectedWorkItemIdField

This is the field that will be used to store the state for the migration . See Server Configuration

CommonEnrichersConfig

This configuration allows to set the configuration for some enrichers at the global level, and the configuration can then be re-used in processors. Currently supported only by TfsNodeStructureOption, to be re-used in WorkItemMigrationConfig and TestPlansAndSuitesMigrationConfig.

WorkItemMigrationConfig

You can specify BasePaths for Areas/Iterations to migrate. The area/iteration has to start with that string to be eligible for migration. E.g. BasePath = “Product\Area\Path1”

With existing areas: “Product\Area\Path1\TestArea” “SomeOtherProduct\Area\Path1\TestArea” “Product\OtherArea\Path1\TestArea”

only the first one matches the BasePath “Product\Area\Path1” and would be migrated, the other ones are ignored.

Field Maps

There are a number of field maps available for when you need to change the data as you are processing it. These mappings work for both in place bulk edit, and for project to project migrations.

"FieldMaps": [
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.MultiValueConditionalMapConfig",
      "WorkItemTypeName": "*",
      "sourceFieldsAndValues": {
        "Field1": "Value1",
        "Field2": "Value2"
      },
      "targetFieldsAndValues": {
        "Field1": "Value1",
        "Field2": "Value2"
      }
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldBlankMapConfig",
      "WorkItemTypeName": "*",
      "targetField": "TfsMigrationTool.ReflectedWorkItemId"
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "System.State",
      "targetField": "System.State",
      "defaultValue": "New",
      "valueMapping": {
        "Approved": "New",
        "New": "New",
        "Committed": "Active",
        "In Progress": "Active",
        "To Do": "New",
        "Done": "Closed"
      }
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldtoFieldMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "Microsoft.VSTS.Common.BacklogPriority",
      "targetField": "Microsoft.VSTS.Common.StackRank"
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldtoFieldMultiMapConfig",
      "WorkItemTypeName": "*",
      "SourceToTargetMappings": {
        "SourceField1": "TargetField1",
        "SourceField2": "TargetField2"
      }
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldtoTagMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "System.State",
      "formatExpression": "ScrumState:{0}"
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldMergeMapConfig",
      "WorkItemTypeName": "*",
      "sourceField1": "System.Description",
      "sourceField2": "Microsoft.VSTS.Common.AcceptanceCriteria",
      "targetField": "System.Description",
      "formatExpression": "{0} <br/><br/><h3>Acceptance Criteria</h3>{1}",
      "doneMatch": "##DONE##"
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.RegexFieldMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "COMPANY.PRODUCT.Release",
      "targetField": "COMPANY.DEVISION.MinorReleaseVersion",
      "pattern": "PRODUCT \\d{4}.(\\d{1})",
      "replacement": "$1"
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValuetoTagMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "Microsoft.VSTS.CMMI.Blocked",
      "pattern": "Yes",
      "formatExpression": "{0}"
    },
    {
      "$type": "VstsSyncMigrator.Engine.Configuration.FieldMap.TreeToTagMapConfig",
      "WorkItemTypeName": "*",
      "toSkip": 3,
      "timeTravel": 1
    }
  ],

Iteration Maps and Area Maps

These two configuration elements apply after the NodeBasePaths selector, i.e. only on Areas and Iterations that have been selected for migration. They allow to change the area path, respectively the iteration path, of migrated work items.

These remapping rules are applied both while creating path nodes in the target project and when migrating work items.

These remapping rules are applied with a higher priority than the PrefixProjectToNodes option. This means that if no declared rule matches the path and the PrefixProjectToNodes option is enabled, then the old behavior is used.

The syntax is a dictionary of regular expressions and the replacement text.

Warning: These follow the .net regular expression language. The key in the dictionary is a regular expression search pattern, while the value is a regular expression replacement pattern. It is therefore possible to use back-references in the replacement string.

Warning: Special characters in the acceptation of regular expressions and json both need to be escaped. For a key, this means, for example, that a literal backslash must be escaped for the regular expression language \\ and each of these backslashes must then be escaped for the json encoding: \\\\. In the replacement string, a literal $ must be escaped with an additional $ if it is followed by a number (due to the special meaning in regular expression replacement strings), while a backslash must be escaped (\\) due to the special meaning in json.

Advice: To avoid unexpected results, always match terminating backslashes in the search pattern and replacement string: if a search pattern ends with a backslash, you should also put one in the replacement string, and if the search pattern does not include a terminating backslash, then none should be included in the replacement string.

Examples explained
"IterationMaps": {
  "^OriginalProject\\\\Path1(?=\\\\Sprint 2022)": "TargetProject\\AnotherPath\\NewTeam",
  "^OriginalProject\\\\Path1(?=\\\\Sprint 2020)": "TargetProject\\AnotherPath\\Archives\\Sprints 2020",
  "^OriginalProject\\\\Path2": "TargetProject\\YetAnotherPath\\Path2",
},
"AreaMaps": {
  "^OriginalProject\\\\(DescopeThis|DescopeThat)": "TargetProject\\Archive\\Descoped\\",
  "^OriginalProject\\\\(?!DescopeThis|DescopeThat)": "TargetProject\\NewArea\\",
}
  • "^OriginalProject\\\\Path1(?=\\\\Sprint 2022)": "TargetProject\\AnotherPath\\NewTeam",

    In an iteration path, OriginalProject\Path1 found at the beginning of the path, when followed by \Sprint 2022, will be replaced by TargetProject\AnotherPath\NewTeam.

    OriginalProject\Path1\Sprint 2022\Sprint 01 will become TargetProject\AnotherPath\NewTeam\Sprint 2022\Sprint 01 but OriginalProject\Path1\Sprint 2020\Sprint 03 will not be transformed by this rule.

  • "^OriginalProject\\\\Path1(?=\\\\Sprint 2020)": "TargetProject\\AnotherPath\\Archives\\Sprints 2020",

    In an iteration path, OriginalProject\Path1 found at the beginning of the path, when followed by \Sprint 2020, will be replaced by TargetProject\AnotherPath\Archives\\Sprints 2020.

    OriginalProject\Path1\Sprint 2020\Sprint 01 will become TargetProject\AnotherPath\Archives\Sprint 2020\Sprint 01 but OriginalProject\Path1\Sprint 2021\Sprint 03 will not be transformed by this rule.

  • "^OriginalProject\\\\Path2": "TargetProject\\YetAnotherPath\\Path2",

    In an iteration path, OriginalProject\Path2 will be replaced by TargetProject\YetAnotherPath\Path2.

  • "^OriginalProject\\\\(DescopeThis|DescopeThat)": "TargetProject\\Archive\\Descoped\\",

    In an area path, OriginalProject\ found at the beginning of the path, when followed by either DescopeThis or DescopeThat will be replaced by TargetProject\Archive\Descoped\.

    OriginalProject\DescopeThis\Area will be transformed to TargetProject\Archive\Descoped\DescopeThis\Area. OriginalProject\DescopeThat\Product will be transformed to TargetProject\Archive\Descoped\DescopeThat\Product.

  • "^OriginalProject\\\\(?!DescopeThis|DescopeThat)": "TargetProject\\NewArea\\",

    In an area path, OriginalProject\ found at the beginning of the path will be replaced by TargetProject\NewArea\ unless it is followed by DescopeThis or DescopeThat.

    OriginalProject\ValidArea\ would be replaced by TargetProject\NewArea\ValidArea\ but OriginalProject\DescopeThis would not be modified by this rule.

...
Getting Support

Community Support

Question & Discussion - The first place to look for usage, configuration, and general help.

Commercial Support

We provide training, ad-hoc support, and full service migrations through Azure DevOps Migration Services