Create a Build vNext build definition

I am going to show how to create a Build vNext build definition on VSO. Microsoft recently announced the creation of a brand new build system for TFS and VSO at the Connect event last year. This new build system will eventually replace the current one and be much more modular and friendly. Happily I am in the early adopter program and the product team just made an early alfa of the service available for that program and I have been giving it a spin.

Download Team Foundation Server 2015 today

Microsoft has released a CTP of TFS 2015 that includes the vNext build system. You can download TFS 2015 and try it out today. Remember that this is not a go-live version and you should not install it in production.

Now that we have configured a Build vNext agent we can get on with the job of creating a build. I had hoped that for my demo at NDC London last month that I would have been able to use this but it took an extra month for the product team to get the Alfa ready. This is really just part of the realities of software development that we can’t know how long something will take until its done.

Create a vNext Build on VSO

There are a number of out-of-the-box templates available and I believe that there will be more by launch. For my FabirikamFiber site I will be using the standard “Visual Studio” definition.

clip_image002

You will get dropped out at Build tab of the “New Visual Studio definition 1” definition that has been created for you. Here you can add new tasks and configure them, which we will discuss in a moment.

clip_image003

This has not yet been saved, so the first thing I am going to do is save my detention with a little more memorable name.

clip_image004

Back at the “Task” list you can click “Add new task” to get a list of all of the available tasks. This is probably pretty close to the list of tasks that we will see initially when the preview becomes more generally available and it is extensive:

  • Android Build – Run an Android build using Gradle and optionally start the emulator for unit tests.

    clip_image005

  • CMake – Cross platform build system. I have never used it but it really does sound handy.

    clip_image006

  • Cmd Script – Run a Windows cmd or batch script and optionally allow it to change the environment

    clip_image007

  • Jake – Javascript build tool, similar to Make or Rake. Built to work with Node.js

    clip_image008

  • MSBuild – Build with MSBuild; In the pre-2010 builds everything was done in MSBuild and in 2010+ (XAML Builds) these build types were only supported in the legacy build template, “UpgradeTemnplate.xaml”. You do not need Visual Studio installed to execute this, but your compilation might.

    clip_image009

  • Visual Studio Build – This build is executed through Visual Studio and should run in the same way that it would locally.

    clip_image010

  • VSTest – You can run tests using the Visual Studio Test Runner. This runner will load tests from any framework that has a test adapter so it supports; MS Test, jUnit, xUnit, mbUnit, and others.

    clip_image011

  • Xcode Build – This task allows you to build an Xcode project with the xcodebuild tool. Microsoft has had a new strategy for a while to support everyone else’s stuff and as they release new versions of their products is it becoming more and more obvious that this is no longer a case of lip service.

    clip_image012

  • PowerShell – Need to run a PowerShell? In TFS 2013 the build workflows were simplified to allow PowerShell both post and pre build as well as post and pre-test. Now you can insert a task any place you like in the build process. PowerShell will let you do anything from moving files around to manipulating the build numbers.

    clip_image013

  • Process Runner – Is the logic that you need to run wrapped up in an executable? Use the Process Runner to execute any executable process.

    clip_image014

  • Azure Cloud Service Deployment via PowerShell – Just like the old Xaml templates to do the same job here is a pre-configured PowerShell command to do the deployment. You can always create a customer PowerShell is you need it, but this is a helper.

    clip_image015

  • Azure PowerShell – Do you ever feel the need to run some PowerShell on one of your servers as part of a build? I don’t, as I use Release Management for environmental bits, but if you have an immature build process you may need this.

    clip_image016

  • Azure Web Site Deployment via PowerShell – I just want to deploy my website to azure! Well here you you go.

    clip_image017

This is just the list that is available in the preview, which is alfa. The plan, as I currently understand it, is to make this extensible so that you can create any sort of tasks that you like and have them listed in the system. At the moment however there is no way to do this and I am not sure when this will happen.

clip_image018

The second tab is the “Options” tab. This is a list of general options that applicable to any build that you would run. Here we can select:

  • MultiConfiguration – This allows us to build more than one configuration as part of the same pass. You can even elect to run them in parallel. You just enter the same name that you have in the configuration pick-list in Visual Studio to either build more than one, or something other than the default. I have used this before to have websites built locally use different parameters than those built on the server. That way I can have my local developer setting built by default, and then have things like the connection string replaced at compilation time with a generic “__connectionStrong__” value for Release management.
  • Copy to Staging Folder – In previous versions of Team Build this was more or less a non-optional step as the VS Test platform needed all of the files copied to the staging location for testing. Now you can control both what files are picked up and where to. This will be very useful for output that is not the traditional “bin” output.
  • Create Build Drop – You might ask why you would not want to have a build drop created but I have seen this for teams with large output and many CI’s. I may not need all the output. The traditional option here is to drop everything onto a network share with a UNC path. However TFS 2012 introduced the concept of a “Server drop”. This is where the build output is stored as a zip files inside TFS. The advantage to this is that you have one location to backup as the build output is an organisational asset. I prefer server drops every time.

clip_image019

Currently the new Team Build vNext system only support Git as the source repository, but you can also select Github and point to any repository that you like. Although I have not tried it I believe that it will support any Git repository that uses the same protocols as GitHub. Nice…

Info: VSO Build vNext has built in support for GitHub builds

clip_image020

In addition to where to get the source code from you may need to create some custom variables that will be available as part of the build. These would be available in your scripts and any other location. There are some enforced variables and most are configurable.

You may use this to inject an option like “HardenPlatfrorm” that takes a Boolean option and you can then use that in your scripts and commands to choose wither you apply your application hardening techniques for piracy protection. Allows you to easily turn it off and on per build…

clip_image021

At the moment the only trigger available is “Continuous Integration” or CI. I would expect in the future to maintain the ability to Schedule builds as well and I really like the fact that you select it with a check-box rather than a radio-button. That bodes well for the future.

clip_image022

“General” configuration contains the last few nuggets of usefulness. First is the pre-defined default queue and maybe a description for the build. I am hoping that we can dynamically update the build number from within the tasks as this is what the majority of my customers want. It just makes sense, to me, to be able to have the build number reflect the version of the code that is being built.

For TFVC I normally change the build number to be something like “1.3.$(YY)$(DayofYear)$(rev:.r)” which would produce something like “1.3.15001.1” as the build number. I would then have a PowerShell that stripped that build number and used it for the DLL’s and the Nuget packages that are produced as part of the build. This works great when we have a single build definition per branch.

However, for Git we need something a little different. With Git builds we build all branches (potentially) with the same build definition. So here we want to have a file checked into the branch that has the filter above so that it can be different per branch. That means that the first thing that we need to do as part of the build is to have a PowerShell execute that reads that file and changes the build name in TFS to match. We will see if that is possible in the new build system as it was hard, but not impossible, in the old one.

clip_image023

Finally, a much requested feature. Most folks want to do a little bit of audit on their build configuration as from some reason folks go in there and change stuff. Now you will know not only who did the change but what they changed. Just above the entry is a little “Diff” button…

clip_image024

…Where you get a full diff of the configuration changes that were made between edits. This makes it really easy to see what was changed and the comment should give you an idea of why. Here you can see that I added a trigger of type 2 (CI).

clip_image025

You now have a build definition configured and you can queue a new build. You can see the Associated commits and other information coming in immediately and there is a new real-time (kinda) command window where it shows the status of the build as it executes.

Conclusion

All in all I am very impressed with the current system. My only issue, as you see the build failed above, is that I get a PowerShell version miss match during execution. This may be due to me using Windows Server Technical Preview as by build platform and I have reached out to the awesome build guys to find out what the issue is. In the mean time I will likely build out a Server 2012 R2 to do more testing…

  • AimeR

    MSBuild – Build with MSBuild; this is equivalent to the old legacy builds from pre-2010 days.

    The pre-2010 build definitions used MSBuild scripts to get the source code, run tests and a whole lot of other stuff, which I would suspect would be in conflict with the statement that the new build system only supports Git as a source control provider.

    It sounds like this option is the 2010-2013 default build template. It would be odd not to have that somewhere in there (those templates DO NOT use Visual Studio to build your projects). It sounds much more logical for the TFS guys to have dropped support for pre-2010 templates rather than basic MSBuild calls.

  • Pingback: Dew Drop – March 5, 2015 (#1968) | Morning Dew()

  • Philippe

    These evolutions, longly awaited, are very promising! I hope it will easy a lot of things with tfs…
    And I hope it will be possible to trigger a build based on the result of a previous build. A feature highly desired!!!

    >However TFS 2012 introduced the concept of a “Server drop”. This is
    where the build output is stored as a zip files inside TFS. The
    advantage to this is that you have one location to backup as the build
    output is an organisational asset. I prefer server drops every time.

    One of the multiple bad practices that Tfs encourage by letting the user check in binaries files in source control 🙁 As a consequence, your database become huge and is difficult to backup, store,…
    A notion that is difficult to understand by tfs users because “that’s possible…” And that’s like that that you end up with huge workspaces to checkout 🙁

    I’m pretty sure that you do that because “Release management” push you to do that. That’s why I highly prefer “Octopus Deploy” (even if it’s not a Microsoft software) because it enforce you to use good practicies like building nuget packages and store them OUTSIDE of Tfs. This software really worth the try and id MUCH more easier and powerfull than “Release Management”…

    To return on the original subject, with git and the new build system, Tfs will at last replace (or propose an alternative) to its two worst pieces. Will become interesting (for those like my that are obliged to work with that tool!)

    • @pmiossec:disqus , the “Server Drop” is not in source control. You are thinking of the “Store drop in Version Control” option that shipped with TFS 2012 but was quickly surpassed by the “Server Drop” location in VSO and shipped in 2013.
      Server Drops are not stored in Source Control and instead are stored in a separate un-versioned repository inside of TFS. Its like a network share, except that it is backed up along with the rest of your organisational assets. This would be the recommended place to store your drops.
      I like Nuget packages too (not always for deployment) and at Connect() Microsoft announced that TFS would have a built in Nuget server. As an aside, the Nuget packages on the server will likely also be stored in that un-versioned store that was specifically designed for loose (ish) files.

    • LECOQ Vincent

      Bonjour Philippe,

      Je cherche à te contacter suite au commentaire que tu avais posté sur korben.info au sujet d’un concours raspberry pi organisé par microsoft.
      J’ai bien participé, enregistré mon site et reçu la confirmation. Cependant Strictement aucune nouvelle depuis la fin du concours et je ne trouve aucun moyen de les contacter pour savoir ce qui se passe.

      As tu complété la procédure de ton coté ? si oui as tu eu des retours ?

      Merci.

  • Pingback: Using the Build vNext capabilities and demands system | naked ALM - Experts in ALM, TFS & lean-agile with Scrum()

  • Alon Amsalem

    This looks very much like TeamCity build configuration style

  • Venkata Sai

    Hi Martin
    I am looking for some guidance on building a consolidated build definition using Build vNext like we do in XAML build system using master template i.e.Invoking other builds as part of this build.Would you be able to guide me to any article that you have already written?

    • You would do all of that in PowerShell. I will be building this out for a customer that does 40k builds a month on a single product in Q4. I will be posting on all of my findings and troubles 🙂

      • Venkata Sai

        Hi Martin,
        I am creating a Build definition for a solution which has both web apps and Cloud services.I wanted to create a single build step and individual deployment for each component.I am using Visualstudio Build step.But I am facing some issues with MS Build parameters as they are different for bothweb apps and cloud services.Is there any way to achieve that?

  • Dominic Perreault

    What’s really missing is to be able to interact with the build past the Staging and Dropping build dont you think?

    It’s way easy to launch scripts during the build process but it doesn’t seem to be a way to launch scripts after the build is dropped in the DropFolder. For deployement for exemple…

    Just the way TfvcTemplate.12.2.xaml would allow with the Pre and PostDrop powershell scripts.

    Have you found a way to do so?

    • The TfvcTemplate.12.2 allows post-compile and post-test, but no post Drop. You should really look at Release Management for deployment through a pipeline and only use a deployment from build for your Development Teams CD. You get RM with VSO is you are deploying to Azure.

      The new build system in VSO and TFS 2015 will come with an integrated set of deployment tools and capabilities that were announced at Build.

      • Dominic Perreault

        The TfvcTemplate.12.xaml only allows Pre/Post Build and Pre/Post Test while the TfvcTemplate.12.2.xaml added the Pre/Post Drop exactly for using RM with the vNext Release Template which are triggered from a REST API. We are using RM since it was named InRelease from InCycle we kinda know a lot about it 🙂

        We needed to change the build process template from 12 to 12.2 to use the PostDrop sequence because the build process “ReleaseTfvcTemplate.12.xaml” shipped with RM 2013 Update 1 or 2 was only a an “Agent Based” deployement scenario. When we moved from agent based scenarios to Powershell based deployement we searched for a new way to trigger the release attached to the build sequence.

        Here’s what we found: http://blogs.msdn.com/b/visualstudioalm/archive/2014/10/10/trigger-release-from-build-with-release-management-for-visual-studio-2013-update-3.aspx

  • Jaans

    Just gave TFS2015 Build vnext a go with VS Online. Deployed an agent and all seemed to go well until the Restore of Nuget packages failed. The failure was due to custom/private package source not being picked up from the installed VS configuration, so those packages could not be found.

    How do I specify the additional package sources for NuGet to use when using an agent?

    • Jaans

      OK It seems that the package sources configured for NuGet.config is stored per user account, e.g. c:Users<>AppDataRoamingNuGetNuGet.config

      My issue is I’m using the system account, so the following path is what you’ll need:
      C:WindowsSysWOW64configsystemprofileAppDataRoamingNuGetNuGet.Config

      PS: This info was very hard to come by!

  • Peter

    I am wondering, if there is a discription of the variables where I can get build information like the changeset number (and other things) the agent is building to make this information available in my program.

  • Matthew Cole

    Hi Martin,
    Have just upgraded an on-prem tfs to 2015, and configured the new builds, but it appears I have ‘broken’ reporting. none of the new builds are showing in our SSRS Reports for the build quality, or build success over time? Any Ideas where I may have messed up?

    • The data from the new build system does not end up in the Data Warehouse, and thus not in the cube either. You need to go through the REST API for this…

      • Matthew Cole

        ahh, that would explain it then.
        Do you know if MS have any plans to include said data in to the warehouse and hence the cube? (because i know you have contacts)

        • I have no special insight on this subject but based on history to date, I don’t believe that there will be any future investments in the current data warehouse and cube. There has been no investment in new data in there since 2010 and that smacks of a feature in maintenance mode. The team seam to be focusing on surfacing data via a fast and efficient REST API. I would focus my efforts on data retrieval from the Rest API…

          • xxxdevxxx

            Well, i don’t really understand how they can make a distinction between OLAP and a REST Query that is servered by the collection database.

  • Allen

    How can I get Build status(succeed or failed) in Build Task such as command line Task?Are there any Build variable to get this status?I have a task to do something that is different when status is succeed or failed. Thanks a lot.

    • Allan, if you go to the Variables tab on your build there is a link at the top that takes you to a list of all available build variables.

      • Allen

        Thanks