One Team Project Collection to rule them all – Consolidating Team Projects

Audience

Everyone

Following on from last weeks successful Upgrading TFS 2010 to TFS 2012 with VSS Migration and Process Template consolidation I finished off the last of the 20-30 Team Project Process Template migration/upgrades to a customised form on the Visual Studio Scrum 2.0. We only added a couple of fields, but we also defined a process and strategy for use of the Template within the organisation.

image
Figure: Consolidate to a single Team Project Collection

As pert of the process of getting everything onto a single Process Template, namely the Visual Studio Scrum 2.0, we identified that the customer should also move to a single Team Project Collection and more than that, into a single Team Project.

Why you might ask?

As I have often mentioned before there are a number of reasons that you would want to be on a single Team Project Collection and there are also reasons why you do not. You should have separate Team Project Collections when:

  • There is not nor will there ever be a relationship between source or work items between Collections
  • You want to  move source from one location to another
  • You have more than 300 Team Projects

In this case the things that we do want to do are:

  • Link Work Items from one project to another
  • Move work items from one project to another

There is only one tool for this job and that is the TFS Integration Platform. While the TFS Integration Platform provides lost of migration options I am interested only in the TFS 2012 –> TFS 2012 variety which greatly simplifies the configuration. In addition to that I am migrating from a Visual Studio Scrum 2.0 Process Template to… well… a Visual Studio Scrum 2.0 Template. I just have 20-30 Team Projects to migrate. The key here is to get the mappings correct.

Installing TFS Integration Platform with Visual Studio Team Explorer 2012

As soon as I downloaded the TFS Integration platform I ran into a bug where the registry key is not found and the Wix installer does not look for the right Key for the RC. The key it is looking for is only present if a developer edition of Visual Studio is installed.

screenie99
Figure: TFS Integration Tools – Issue: “This tool requires the TFS client object model”

Luckily Jahangeer Mohammed road to the rescue within 4 hours of me reporting the issue with the root cause and Willy posted the solution the very next day. That’s ALM Ranger efficiency… customer was very impressed (although Neno would have a fit over the offending registry key)

Running the TFS Integration Platform for the first time on TFS 2012

The first time you run the TFS Integration Platform on any computer you will get a message asking you to add the account that it is running under to a particular group. You can either add it manually or click “Yes” to get it to do it itself…quickly followed by cursing as you need to be in Administrator mode. Restart… Yes… all is good with the world.

I will get into the configuration in just a minute, but first lets solve the very first error message you will run into.

image
Figure: TFS WIT bypass-rule submission is enabled

You may not be using an account that is in the “Team Foundation Server Service Accounts” group or the equivalent Collection group. No wait… you can’t just go an add it. This is a special group that does not allow you to populate it through the UI. You can however view it and all of the accounts that you use for your Build Agents, Build Controllers and other bits and bobs will all be in this list already. So how to add your TFS Integration Platform account?

image
Figure: Updating the TFS Security group

You use our old friend the command line. There is an application called TfsSecurity that will allow you to add an account directly to that group.

tfssecurity /g+ "Team Foundation Service Accounts" n:domainusername ALLOW /server:http://myserver:8080/tfs

Now you have that sorted you are ready to rock…

Migrating Source Control using the TFS Integration Platform

As we are trying to fold 30 Team Projects across 4 Team Project Collections into a single Team Project on a single Team Project Collection we need to be very specific with the mappings. We are also really lucky that there are no relationships between Work items and Version Control which simplifies the process as we can do them separately.

There are a ridicules number of options here and the rangers have many, many documented options that will allow you to do whatever you want. There is however something interesting that you can do and that we wanted to try… You can move individual branches to new locations while maintaining the relationships. This works really well on codebases that do not have a lot of complex branches or deletes. If you have many deletes outside of the scope, or you have a lot of sub branches of code branches into your solution from outside of the scope of a team project you can run into a few problems.

image
Figure: Complicated mappings will not always work

You can skip to the end to see all of the problems and solutions, but if you are doing these complicated mappings you may have issues getting a distinct mapping due to how the TFS Integration Platform creates mappings, but the solution is to ditch the mappings and the root of $/TeamProjectA/ to the single sub folder of  $TeamProjectB/TeamProjectA/ so that everything is just mapped at that level. This solved all of those problems and you can always do the rearranging later.

Regardless of this problem we needed to map all of our existing Team Projects to a new folder underneath a single existing Team Project on our new Collection. We are mapping each Team Project individually as that is how the TFS Integration Platform is setup so we need to create a Migration Configuration for each Team Project.

We actually tried the complex rearrange mapping on all of the projects and called a “destroy” on the migrated source for all of the ones that failed and redid them at the root.

tf destroy $/somefolder/somesubfolder/ /collection:http://mytfsserver:8080/tfs/tfs01

Figure: Destroying Migrated data to start again

image
Figure: A successful mapping

Make sure that you check the source after you bring it across for consistency as even if the Platform tells you all is well, it could well be lying and we did find a couple of glitches (again detailed below).

Effectively you just go through each of your source Team Projects, mapping them to the destination Team Project and keep going until you are done. The time it will take will depend on the amount of source code and the number of changes, but the source code will be fairly fast.

There is a neat trick that works pretty well if you don’t have any differences, except for the target folder, for your source to bring everything across from a Collection.

image
Figure: Just because your pick a Team Project does not mean that is the root

In this case I want to bring everything across from all of the Team Projects in the Collection to a folder under the new Team Project. Now, you need to initially select a Team Project as the source but I then changed it to “$/” to just bring everything across to its new home of “$/NewTeamProejct/Department/” and have it create folders of the same name as the old Team Projects as plain folders.

image
Figure: Runs through like a dream

I had initially worried that it work choke at the Team Project boundary, but I have checked that everything is coming across properly and for a change it is operating exactly as you would expect given the settings.

So, apart from the 7 team projects that I have a block on to see if a fix comes through from the Product Team I have successfully consolidated the Source from

Migrating Work Items using the TFS Integration Platform

simplesBecause we have already completed the Process Template consolidation we only have one Process Template mapping to work about… or do we. If you used method #7 for migrating  your process template you will be left with extra fields that are not part of the core template. Make sure that the destination Team Project has the same work item type definitions as the source Team Project and you will have no problems Smile

In your configuration file, because we have the same work item types on each end, you can just map all of the Left Work Item Types to all of the Right Work Item Types…

 

Figure: Work Item Mappings

On top of that they only real issue is that you need to enable WIT Bypass rules submission which tells the TFS Integration Platform to use the Woeb Services directly and not the API’s. Why, you might ask…. well we need to be able to write historical states that do not actually exist in the workflow so that we can maintain the history.

  
    
    
    
  
   

Figure: Enabling the WIT Bypass rules submission

Enabling this will allow us to seamlessly do the migration of Work Item data. If we did have links between Source Control and Work Items we can handle that by either migrating both Source and Work Items in the same session or we can add the relationships later.

A final configuration would include some mapping or transformation of the Area Path so that we can find our merged work items later. The TFS Integration Platform will create any Area Paths that don’t already exist.

< ?xml version="1.0" encoding="utf-16"?>

  
    
  
  
  
    
      
        
          
          
          
        
        
          
        
        
      
      
        
          
          
          
        
        
          
        
        
      
    
    
      
        
        
          
            
              
              
                
              
              
                
                  
                    
                  
                  
                    
                      
                      
                    
                  
                
              
              
            
          
          
        
        
          
            
            
          
        
      
    
    
      
      
    
    
    
    
      
    
    
      
      
    
  

Figure: Configuration for Merging Team Projects

image
Figure: Save your configuration ready to rumble

Once you have migrated your work items you will see some invalid data, but this is due to the Process Template consolidation that we have already completed. All you have to do is fix the visible data before you save and all is well…

image
Figure: Some work items will be invalid

And that finishes of migrating your data…

Troubleshooting

There are a couple of errors and exceptions that while outside the norm were a common occurrence for our migration:

References

There were a number of references that I used as part of this engagement.

Conclusion

While it is difficult to migrate data or move data around in Team Foundation Server it is not insurmountable and mist issues can be solved if you can accept the trade-offs.

  • Are you interested in consolidating to fewer Team Projects?
  • Did you mistakenly create multiple Team Project Collections?

If so then give us a call and we can help you fix it…

Upcoming Training Opportunities

These are the next five classes we have, and you can check out our full public schedule of classes.

Timezone:
Live Virtual Professional Scrum Master Online on 29th May 2023
Virtual
29 May-1 May, 2023
09:30-13:30 BST
4 half-days
Live Virtual Professional Scrum Product Owner online 5th June 2023
Virtual
5-8 Jun, 2023
09:30-13:30 BST
4 half-days
Live Virtual PAL Evidence-Based Management Online on 19th June 2023
Virtual
19-20 Jun, 2023
09:00-13:00 BST
2 half-days
APS 19th June 2023
Virtual
19-22 Jun, 2023
09:00-13:00 EDT
4 half-days

We can deliver any of our courses as private in-house training over Microsoft Teams & Mural. We also recommend training based on your accountabilities or role, you can go directly to recommended courses for Scrum MastersProduct OwnersDevelopers and Agile Leaders.

Create a conversation around this article

Share on Facebook
Share on Twitter
Share on Linkdin

Related Courses

No items found

Read more

Martin Hinshelwood nkdAgility.com
https://youtu.be/_fFs-0GL1CA Why do you encourage people to follow a certification path in their career journey? I would encourage you to follow a scrum certification path for the same reason that people go to university. The same reason that people do any course or certification. It gets you a foot in …
Martin Hinshelwood nkdAgility.com
https://youtu.be/QGXlCm_B5zA What will you learn on the PSM II course? There are two main things that most scrum masters will learn on the PSM II or Advanced Professional Scrum Master course. That they haven’t been working effectively as a scrum master to date. That they are there to empower and …
Martin Hinshelwood nkdAgility.com
In Scrum Events across the world, I hear repeated the phrase “that’s how agile works” when describing behaviours that are both unprofessional and the very opposite of an agile mindset. These behaviours will inhibit agility and are a result of a lack of understanding of the underlying principles. We need …
Martin Hinshelwood nkdAgility.com
https://youtu.be/klBiNFvxuy0 What is the most common Aha moment people have in a scrum course? It depends on the scrum course they are attending. The content presented in the Applying Professional Scrum (APS) course leads to very different epiphanies when compared to the content presented on an Advanced Professional Scrum Master …

OUR TEAM

We believe that every company deserves high quality software delivered on a regular cadence that meets its customers needs. Our goal is to help you reduce your cycle time, improve your time to market, and minimise any organisational friction in achieving your goals.

naked Agility Limited is a professional company that offers training, coaching, mentoring, and facilitation to help people and teams evolve, integrate, and continuously improve.

We recognise the positive impact that a happy AND motivated workforce, that has purpose, has on client experience. We help change mindsets towards a people-first culture where everyone encourages others to learn and grow. The resulting divergent thinking leads to many different ideas and opportunities for the success of the organisation.