{ blog: 'Gøran Sveia Kvarv' }: 2012

Thursday, October 11, 2012

Continuous Delivery with psake and TeamCity - Visualizing a pipeline REVISED

The series so far:

In one of my pervious post, Continuous Delivery with psake and TeamCity - Visualizing a pipeline I wrote about how you could sync build numbers between build configuration. In that post I suggested to create a file that contained the build number which would be published to the artifact repository in TeamCity and used by dependent build configurations to set the build number. That was not exactly an optimal solution.
In Continuous Delivery with psake and TeamCity - Time for Some Refactoring I talked about how you could use the inherited build number from the “parent” build configuration and pass that into the build script and then set the build number in TeamCity. This was certainly a better solution, but it would be better to just let TeamCity handle setting the build number. To fix this, go to General Settings in the dependent build configuration and put dep.btX.build.number in the “Build number format” text box.

Actually I have been doing it this way for a long time now, so I should have written this post much sooner. Sorry.

Monday, February 13, 2012

Continuous Delivery with psake and TeamCity - Config Templates

The series so far:

When we are deploying we always do some config transforms so that the application is configured for the right environment. In Nant we got xmlpoke, in msbuild 4.0 we got XmlPoke and there's also web.config transforms in .Net 4.0 which has been transformed to a general config transformation tool. In psake there isn’t something like this available, however as James Kovacks himself says, it’s very easy to do with PowerShell. I’ve created a little PowerShell script called Replace-Tokens, which you can find on GitHub, that does the work. Given a file some.template:

hello #{world}

and a file some.data.ps1:

@{
    world = 'world'
}

When we run:

replace_tokens_with_file "some.data.ps1" "some.template" "output.txt"

The output is the file output.txt:

hello world

The main idea here is that we pass in a hash table who's keys will be located in the template file and swapped by the value. We can also pass in a hash table directly instead of a file. So referring to my former post, if we were to change the app.config of the application when deploying to Test and Acceptance Test we could use the script and do the transformation. Change the test.ps1 properties file to:

$database_server = "Server 2"

$app_config_data = @{
    "database_connection_string" = "Data Source=$database_server;Initial Catalog=App;Integrated security = true";
}

Then change the deploy task of deploy.ps1 to:

task deploy -depends set_build_number{
    Write-Output "deploying to $env => database is deployed to $database_server"
    $app_path = "$build_artifacts_dir\Debug\ContinuousDelivery.WpfApplication"
    $config_file = "ContinuousDelivery.WpfApplication.exe.config"
    replace_tokens $app_config_data "$configs_dir\$config_file.template" "$app_path\$config_file"
}

Note that we pass in the $app_config_data from the test environment properties file. Now we just need to create a template of the app.config called ContinuousDelivery.WpfApplication.exe.config.template and place in in folder called configs under the build folder:

<?xml version="1.0"?>
<configuration>
  <connectionStrings>
    <add name="database" connectionString="#{database_connection_string}"/>
  </connectionStrings>
</configuration>

This will overwrite the app.config based on the values found in the environment specific properties file when we deploy. Download the code from GitHub.

Tuesday, February 7, 2012

Continuous Delivery with psake and TeamCity - Time for Some Refactoring

Refactoring is not just something you do with code. It’s also something you do with the build scripts and build process. Today we will split up our build script in nested builds and do some changes to how we pass on the build number to the next build configuration in the pipeline.

My previous installments in this series:

First up, I want to change how I pass in the task to run and the environment to the build script. It looked something like this for the CI configuration:

&  {$host.UI.RawUI.BufferSize = new-object System.Management.Automation.Host.Size(512,50); .\tools\psake\psake .\build\build.ps1 ci -parameters @{env='ci}}

See how ci gets passed in, both for the task and the env parameter. I want to change this so the build script gets these values from TeamCity through Configuration Parameters. By doing that it’s a little bit easier to see and change these values. Let’s define some configuration parameters. Go to Build Parameters and add this:

Change the PowerShell build step to:

&  {$host.UI.RawUI.BufferSize = new-object System.Management.Automation.Host.Size(512,50); .\tools\psake\psake .\build\build.ps1 %task% -parameters @{env='%environment%'}}

TeamCity let’s you access the parameters that you have defined by typing %parameter_name%. Do the same changes for the Test and Acceptance Test build configuration.

Next we’ll do something similar to pass in the build number from CI to Test and Acceptance Test. When you set up a dependent build you get access to all the build parameters passed and set by the build a build configuration is dependent on. So if C is dependent on B and B is dependent on A, C gets access to both A’s and B’s build parameters. So the build number of CI gets passed to Test and Acceptance Test as %dep.bt2.build.number% where bt2 is the build configuration id of CI. For Test and Acceptance Test add the following build parameter:

and change the PowerShell build step to:

&  {$host.UI.RawUI.BufferSize = new-object System.Management.Automation.Host.Size(512,50); .\tools\psake\psake .\build\build.ps1 %task% -parameters @{env='%environment%'; build_number='%build_number%'}}

The final thing I want to change how we call the task deploy and set_build_number. Both these tasks are relevant only to deployment so we’ll move them to a separate file called deploy.ps1:

$framework = '4.0'

include .\..\tools\psake\teamcity.ps1

task default -depends deploy

task deploy -depends set_build_number{
    Write-Output "deploying to $env => database is deployed to $database_server"
}

task set_build_number {
    TeamCity-SetBuildNumber $build_number
}

and then we’ll add a new deploy task in build.ps1 which calls the the deploy build script so that we get nested builds, a feature in psake. Here we also pass along all the parameters and properties so that the nested build can access the same variables:

task deploy {
    invoke-psake .\deploy.ps1 -properties $properties -parameters $parameters
}

Remember to always refactor your build scripts and build process. It’s just as important as code refactorings :)

Download the bits from GitHub.

Sunday, January 29, 2012

Continuous Delivery with psake and TeamCity - Environment Configuration Management

Happy New Year! I’ll start of the year by adding another post to the series about continuous delivery with psake and TeamCity. So far we’ve covered:

Today we’ll look at how to take care of configuration settings when we are deploying to different environments.

Let’s say that we have set up three environments; “CI”, “Test” and “Acceptance Test”:

Each of these environments need to have specific settings, for example which server to deploy a database to.

Powershell has the concept of dot sourcing. Basically it lets you include a PowerShell script from within another script and then you get access to all those variables and methods you defined in the script you included. We’ll use this feature to load an environment specific file into our build script based on an environment parameter that we pass into the build.

Modify the build script called build.ps1 from the previous post and add the highlighted line to the properties section:

properties {
    $base_dir = resolve-path .\..
    $build_dir = "$base_dir\build"
    $properties_dir = "$build_dir\properties"
    $source_dir = "$base_dir\src"
    $build_artifacts_dir = "$base_dir\build_artifacts"
    $tools_dir = "$base_dir\tools"
    $config = "Debug"
    $test_dir = "$build_artifacts_dir\$config\tests"
    
    . "$properties_dir\$env.ps1"
}

Next we’ll add three environment specific files to the path “.\build\properties”:

# ci.ps1
$database_server = "Server 1"

# test.ps1
$database_server = "Server 2"

# acceptance_test.ps1
$database_server = "Server 3"

One of these files will be loaded when we pass in the env parameter to the build script from TeamCity.

Copy the Test build configuration and call it Acceptance Test. Remember to republish the artifacts downloaded in the Test build configuration and make Acceptance Test dependent on the last pinned build of Test:

Change the Powershell build step (see picture below) in TeamCity and add

-parameters @{env='ci'} to the CI build configuration,
-parameters @{env='test'} to the Test build configuration and
-parameters @{env='acceptance_test'} to the Acceptance Test build configuration.

Let’s modify the deploy task in build.ps1 and print out some information about the environment:

task deploy -depends set_build_number{
    Write-Output "deploying to $env => database is deployed to $database_server"
}

and also add the following to the compile task:

Write-Output "Integrating database changes for $env at $database_server"

to simulate that we integrate database changes. Since we defined the variable database_server in the environment specific files we can now access it in the build file.

We should now be able to verify in the build log that we have “deployed” to our specified server:

We can also see that builds flow between the different environments and be able to see which build is deployed to which environment:

As always, download the code from GitHub.

Pages