Integrating Additional Tools in a SharePoint Continuous Integration Build (Chris O'Brien)

We have finally come to the last post in our series on Continuous Integration for SharePoint. Today we’ll first wrap up on the subject of running tests from a build, and then primarily we’ll be discussing code quality. In particular, we’ll look at how we can use the build process as the vehicle for performing several forms of analysis on our code, using tools such as SPDisposeCheck and Visual Studio Code Analysis.

Running unit/integration tests in a build

In the last post, Running Tests as Part of a Build, I showed how Visual Studio Coded UI Tests can be integrated into a build to verify its quality. Since we didn’t get to cover it there, it’s worth mentioning some considerations for running unit tests within a SharePoint build; of course, unit tests may provide equal or greater value than Coded UI Tests, and ultimately a mix of several test types is recommended. There is a caveat here – because SharePoint 2010 runs on .NET 3.5 and Team Foundation Server 2010 Team Build runs on .NET 4, it is not possible to use unit tests which call the SharePoint API within an automated build. Such tests are typically referred to as integration tests.

The table below gives a summary:

* Integration tests may work on a build server which has both the TFS Build Agent and SharePoint 2010 installed, but this configuration is not recommended.

It’s worth remembering that this caveat applies to automated builds only. Unit and integration tests for SharePoint can be run manually from within Visual Studio 2010 of course, so long as you have Visual Studio 2010 Service Pack 1 and some Visual Studio configuration settings are changed. See SharePoint 2010 and unit tests for more information.

With that covered, let’s move on to SPDisposeCheck and Code Analysis.

Integrating SPDisposeCheck into the build

If you’re not yet familiar with SPDisposeCheck, it’s a tool built by Microsoft which scans custom SharePoint code for issues related to object disposal – such issues can cause severe performance problems, and use of the tool is considered a best practice. Integration of such tools into the build tends to work well, since it means that individual developers do not need to remember to run the tools manually. First, however, a note on running tests in a build. Since SPDisposeCheck is a command-line executable, it can be integrated into a build easily. As you may remember from Deploying WSPs as part of an automated build, we can use the InvokeProcess activity in a TFS build workflow to call out to any external process. In this case, all we really need to do is drag an InvokeProcess activity onto the workflow design surface, and then configure it with the install location of SPDisposeCheck.exe and the location where the assemblies which are output from the build can be found.

Integrating SPDisposeCheck – the process

Here are the detailed steps:

1. Install SPDisposeCheck on the build server (i.e. the server running the TFS Build Agent – this could be multiple if you have scaled out this role). We’ll need to take a note of the install location of the executable file – by default, the installer will use C:\Program Files (x86)\Microsoft\SharePoint Dispose Check\SPDisposeCheck.exe.

2. In TFS source control, open the XAML file which represents your build process from the BuildProcessTemplates folder. Double-click the file to open it in the Visual Studio workflow designer.

3. Identify a suitable place in the workflow to run SPDisposeCheck. This needs to be somewhere after the step which performs compilation, as we need the assemblies to have been created in the build output directory. For simplicity, consider implementing these modifications to the build template in the same location as any other custom steps you have, e.g. just before steps to deploy WSP packages.

4. Drag on a Sequence activity. Rename the display name to something like “Run SPDisposeCheck”.

5. Inside the Sequence, drag on an InvokeProcess activity. Rename the display name to something like “SPDisposeCheck”.

6. [OPTIONAL] Define an integer variable to collect the number of failures returned by SPDisposeCheck. This should be configured as below:

7. Configure the InvokeProcess activity. The FileName property should be the location of SPDisposeCheck.exe on the build agent, and the Arguments property should be the outputDirectory build variable, wrapped in a quotation marks. Additionally, if we defined the integer variable in step 6, the Result property should populate this:

8. Add logging activities to the InvokeProcess activity, so that any messages output from SPDisposeCheck appear in the build report. Drag a WriteBuildMessage activity to capture the stdOutput variable, and a WriteBuildError activity to capture errOutput.

9. Edit the properties of the WriteBuildMessage activity - change the Importance property to BuildMessageImportance.High. Without this, messages from SPDisposeCheck will not be written to the build report by default.

10. [OPTIONAL] Write a warning message to the build summary if SPDisposeCheck finds issues.

a. Add an ‘If’ activity immediately after the InvokeProcess activity which calls SPDisposeCheck. Modify the Condition property to be an expression which tests if the integer variable (created in step 6) is greater than zero. This may look like “SPDisposeCheckResult > 0”.

b. Within the ‘If’ activity, drop a WriteBuildWarning activity. Edit the Message property to log the number of issues reported by SPDisposeCheck. This may look like String.Format(“SPDisposeCheck reported {0} issues – these should be investigated.”, SPDisposeCheckResult)

11. Save and check in the build process template file.

Integrating SPDisposeCheck – the result

When a build runs, the build report will now contain details from the analysis SPDisposeCheck performed across the assemblies which were built. Since SPDisposeCheck writes to the console, our WriteBuildMessage activities will take this output and add it to the build log:

If you performed the optional steps to write a warning message to the build summary, this can be seen in the summary view for the build (before going into the detailed log):

Using Code Analysis in a build

Like other tools which developers can run manually but may forget to, Code Analysis is also perfectly suited to Continuous Integration. Code Analysis is a Visual Studio 2010 (Premium or above) feature which checks code for adherence to good practices, including those documented in the .NET Framework Guidelines. Microsoft supplies several Code Analysis rule sets to check code for different issues, and it is also possible to create custom rule sets and suppress individual rules to have full control over the analysis process. My experience of the capability is that whilst some tuning is often beneficial, even adept developers can discover valid issues with their code which they had not previously noticed.

Enabling Code Analysis – the process

Here are the detailed steps:

1. Edit the build definition, and in the ‘Process’ tab, find the ‘Perform Code Analysis’ option. Set this to either ‘Always’ or ‘AsConfigured’ – using the latter means the settings on each individual Visual Studio project being built will be respected.

2. Queue a new build using this build definition. When the build finishes, the build report will most likely contain many CA0060 errors mainly relating to SharePoint assemblies. The description of each CA0060 error will be similar to “The indirectly-referenced assembly [assembly details] could not be found. This assembly is not required for analysis, however, analysis results could be incomplete.” as shown in the image below:

3. Add missing assemblies to build server. Whilst Code Analysis should in fact run successfully despite the CA0060 errors (and any issues should now appear in the build report), it is highly recommended that the additional ‘noise’ caused by the CA0060 errors is dealt with. To do this, each of the assemblies listed with a CA0060 error should be copied to the server(s) running the TFS Build Agent service. The assemblies can be copied from the GAC of a SharePoint 2010 server – these should be added to the folder on the build server which already contains other SharePoint assemblies. If you used the Microsoft-supplied SharePointProjectToTfsBuildServer.ps1 PowerShell script as documented in Creating your first TFS Build Process for SharePoint projects, this location will be ‘C:\Program Files (x86)\Reference Assemblies\Microsoft\SharePoint14’.

4. Restart the TFS Build service. Ensure no builds are currently running, and then open the Team Foundation Server Administration Console on the TFS Server. In the ‘Build Configuration’ view, use the ‘Restart’ link for the Build Service to do this:

5. Test the changes by queuing a new build. The build report should now only valid Code Analysis issues, without the noise of the CA0060 errors:

Summary

Once automated builds are implemented, they provide a great facility for automatically running processes ‘in the background’ which developers would otherwise have to run manually. In real-world conditions, this provides a significant advantage since individual developers do not have to periodically run such tools between coding. Additionally, the automated build never forgets to do these tasks.

In this article we showed how to include SPDisposeCheck and Visual Studio Code Analysis in a build process. Practically any tool which can be called from the command-line can be run from an automated build, though it’s worth noting that some Visual Studio capabilities such as Code Profiling are not supported in TFS 2010/VS 2010.

In broader terms, this series has hopefully shown that continuous integration can provide huge gains in the SharePoint development process. Bugs can be found more quickly, and productivity can be enhanced by automating regular tasks such as those related to code quality.