We’ve received several reports that our NuGet packages broke the NuGet package restore feature. In this post, I’ll explain what the issue is, how you can work around it, and finally how we plan on fixing this issue in the long term.

The problem and solution

Microsoft.Bcl.Build and Microsoft.Bcl.Compression require custom target files, which do not work well with NuGet’s package restore feature. The easiest way to fix the package restore issues is by checking in any .targets files that are stored under the packages directory.

What’s package restore?

When you add a NuGet package to your project, NuGet essentially does two things:

1. It downloads the package and puts it in your solution under a directory called “packages”.
2. It installs the package to your project. This will add references to additional framework assemblies, references to assemblies provided by the NuGet package, add additional content to your project and last, but not least, import any custom target files. Of course, not all NuGet packages use all those features.

In order to build your project on a build server you have to check in all sources as well as all 3rd party libraries. Many developers cringe when binaries need to be checked in to version control as they typically aren’t stored very efficiently and cause bloat over time. This is especially problematic for distributed version control systems (DVCS) like git or Mercurial where developers have to download the repository with the entire history (typically referred to as “cloning”).

For that reason NuGet has a feature called package restore. You need to enable that feature explicitly by right-clicking your solution and invoking the Enable NuGet Package Restore menu item:

After package restore is enabled you can delete the “packages” directory from your solution. Rebuilding the project will automatically re-create this directory and retrieve all missing packages; whether they are downloaded from the internet or a local file share. This allows excluding the packages directory from version control as the build machine can retrieve the packages and thus doesn’t need a checked-in version of the NuGet packages.

What’s the issue?

Two of our packages provide a custom targets file:

1. Microsoft.Bcl.Build
2. Microsoft.Bcl.Compression

Targets files are MSBuild files that provide additional functionality that extend the build process. We use it for several features for which NuGet doesn’t provide declarative features today, for example, binding redirects and choosing the correct binary for the selected architecture. We also use it for additional diagnostics, as explained in this blog post (section “The CPU architecture matters”).

Installing Microsoft.Bcl.Build will change your project by adding an entry for the target file to your project. The target file is located in the packages folder

If package restore is enabled, it’s likely the packages folder is excluded so the .targets file isn’t available on the build machine. You may now think “hold on – isn’t package restore supposed to take care of this?” Unfortunately, not in this case. In order to restore packages you need to build the project, since package restore is plugged into the build process. Before building a project MSBuild loads the project, which in turn requires loading all necessary .targets files. Since the .targets files can’t be located at load time the build fails well before package restore has a chance of executing. You will see an error message like this:

YouProject.csproj(44,3): error MSB4019: The imported project "D:\YouProject.csproj\packages\Microsoft.Bcl.Build.1.0.7\tools\Microsoft.Bcl.Build.targets" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk.

You may be aware that NuGet 2.5 added first class support for installing custom target files. The end result is very similar, except that the import looks like this:

The import is conditioned on the existence of the file which allows loading the project file even if the .targets file doesn’t exist. Unfortunately this doesn’t solve the problem either.

On the build machine, the build will load the project without our .targets file, run package restore and then build the project. Since MSBuild evaluates all entries before building, the .targets files don’t participate in the build because they weren’t present when the project was loaded. Depending on what a custom .targets file does this can have virtually any impact on the build, for example the build may simply still fail or – worse – succeed with incorrect outputs. For that reason we decided to not use conditional imports because the build at least fails in a deterministic and predictable way.

On the developer machine you have a very similar problem. When a project has missing .targets files it will not successfully open in Visual Studio. You might think that conditional imports would be a good solution there but you’ve fundamentally the same problem as on the build machine: the first build will restore the packages but the build outputs might be bogus without yielding an error. This behavior would remain until you re-open the solution. This issue would be quite hard to diagnose.

Workarounds

In order to make package restore work for packages that have custom target files you have three options:

1. Stop using package restore and check-in all package files
2. Explicitly run package restore before building the project
3. Check-in the .targets files

The first option isn’t a really an option and we certainly wouldn’t recommend it either.

The second option requires changing the way your build is hooked up to your build environment. You need to run nuget.exe to restore packages before you actually run MSBuild on your solution:

        nuget.exe install .\PortableClassLibrary3\packages.config

This ensures that by the time MSBuild loads the project all .targets are already present.

The third option is the easiest solution as it doesn’t require any changes to your environment. You only need to check-in the .targets files – you don’t need to check in any other files from the packages directory. Since target files are simply text files, they aren’t causing the repository to bloat over time. This also solves the issue on the developer’s machine whereby the second option would still require each team member to run package restore before the affected projects can be opened in Visual Studio.

We are working with the NuGet team to make sure we don’t have to use .target files moving forward. The idea is to add some features that allow packages to be declarative as opposed to adding code that runs during the build.

Summary

We understand that this isn’t good long term solution and are working with NuGet to greatly reduce the need for custom .targets files.

Please let us know if you have any questions or additional concerns.