IntelliTest - hands on

In practical terms, white box unit test development includes an iterative workflow informed by code coverage - write a unit test, see what parts of the code are not covered by the test, write more tests to cover those parts, repeat until all of the code is covered – a workflow not different from what we would use while working with IntelliTest, as we will show in this demonstration with an application (https://github.com/dylan-smith/pokerleaguemanager) that ALM MVP Dylan Smith kindly permitted to use for the purpose.

Step: install the application and build PokerLeagueManager.sln.

The application tracks stats for a weekly poker league. There is a table that has the stats on each player (Games Played, Total Winnings, Total Profit, etc.). The method we want to test is in: PokerLeagueManager.Queries.Core \ EventHandlers \ GetPlayerStatisticsHandler.cs \ Handle(GameDeletedEvent), as shown below:

When a Game is deleted this method is responsible for updating the stats of the affected players.  It does this in steps:

1. Retrieve the list of Players that were in the deleted Game
2. For each player get their current stats
3. Update the stats object to reflect the necessary changes
4. Save the data

As with most real-world code, this code interacts with other objects and layers. Our goal with this demonstration is to enable IntelliTest reach 100% code coverage on the Handle method.

Understanding the warnings

Step: “Run IntelliTest” on the Handle method.

We see only a couple of tests generated and low coverage (6/42 blocks), but also 5 warnings.

Step: Click on the warnings button.

The first warning “Runtime Warning” says that IntelliTest has discovered, and will use, “PokerLeagueManager.Queries.Core.QueryDataStore” [PQCQ] as IQueryDataStore. Browsing through the code we see that IQueryDataStore is the type returned by the getter from the QueryDataStore property on the base class BaseHandler. In order to unit test this method, a concrete instantiation of this type is required – PQCQ is the instance IntelliTest has gone and discovered.

But is that the type you want to use?

Further, it has also discovered publicly accessible APIs though which to instantiate PQCQ (in this case that happens to be the public constructor). The APIs need to be publicly accessible because IntelliTest needs to actually call them to instantiate the type. So, the first of the Object Creation Warnings alerts us about the APIs that it discovered. If we prefer, those calls can be persisted as a Factory method.

When IntelliTest actually called them to instantiate PQCQ, it could not create an instance it could reason about. That is the second warning.

As it turns out the constructor had ended up calling into some, as yet, uninstrumented code.

On inspecting the stack trace we can see a call from the constructor into DbContext code, where execution transitioned into uninstrumented code. IntelliTest works by instrumenting code and monitoring execution. However, it does not instrument the entire universe of code for two reasons (1) it cannot know a priori what comprises that universe of code (2) that would make the system very slow. So, that is why we see that “uninstrumented method” warning.

And by then, the number of branches in the code path that IntelliTest is exploring is so large, that it trips an internal bound that has been setup for fast interactive performance. Hence, it raises a warning and stops the exploration.

Providing mock implementations

To proceed further, we need to answer that first question: is that the type you want to use? To unit test the method, we need to provide a mock implementation of IQueryDataStore. Browsing through the solution, we see a FakeQueryDataStore. Let’s tell IntelliTest to use that (instead of the PQCQ that it discovered).

To start assisting IntelliTest like this we need the Parameterized Unit Test (PUT aka IntelliTest).

Step: Click on the Warnings button to toggle back to the tests. Select, and Save, all the tests, and then delete the .g.cs file (these did not cover all of the code, remember?).

What we have in the GetPlayerStatisticsHandleTest.cs file is the PUT.

Step: Add a reference to PokerLeagueManager.Common.Tests in the generated test project, and add the lines in red as shown to the PUT.

Step: Do a “Run” from the exploration Results window OR “Run IntelliTest” on the PUT (or on the Handle method).

This time that bounds exceeded warnings is gone. We see 2 tests, but only 4 warnings this time.

Focusing on ‘just my code’

Step: Click on the warnings button.

IntelliTest says it has discovered how to instantiate PokerLeagueManager.Common.Tests.FakeQueryDataStore [PCTF], and alerts us about the APIs it can use to instantiate it (if we prefer, those calls can be persisted as a Factory method). That is the first warning.

But when it went ahead and called those APIs to instantiate PCTF, it ended up calling into uninstrumented code again. On inspecting the stack trace associated with these warnings, we can see the calls where execution transitions into uninstrumented code - it is at the constructor and at the GetData methods.

Since we are not testing the mock, lets us suppress these warnings.

Step: Select the Object Creation warning and click on Suppress to suppress the warning.

Note that all of the warnings related to uninstrumented method calls are coming from the same type [PCTF].

Step: Select any one of them click on Suppress to suppress all such warning coming from that type.

We should see this lines added in PexAssemblyInfo.cs

Step: Do a “Run” from the exploration Results window OR “Run IntelliTest” on the PUT (or on Handle method.).

We should see those same 2 tests generated. This time there are no more warnings.

But we should remember to setup PCTF to be able to return data from calls to the GetData method! If not we will not be able to exercise the code-under-test further (note that we have covered only 15/42 blocks).

Crafting the PUT

In the PUT, ‘target’ is the object that contains data that will be returned by calls to GetData - in particular, T is either a LookupGamePlayersDto[] or a GetPlayerStatisticsDto[]. We need fill up PCTF with concrete instances of these. Let’s ask IntelliTest to generate concrete instances of these.

Since IntelliTest can synthesize data values, we will add this to the PUT’s signature. We want 2 instances of LookupGamePlayersDto, and 1 instance of GetPlayerStatisticsDto. Further, we will associate the statistics for the first player.

Step: Add a reference to PokerLeagueManager.Common.DTO in the generated test project, and add the lines in red to as shown to the PUT (NOTE we are changing the signature of the
PUT).

Next, we will prime the target with these:

and, then we will exercise the code under test:

at this point, we will simply query for the statistics and assert the observed value of its fields.

Add a using System.Linq and then assert:

Step: Since we changed the signature of the PUT, delete the g.cs file once again.

Full code coverage

Step: Do a “Run” from the exploration Results window OR “Run IntelliTest” on the IntelliTest (or on Handle method.).

Now we should see all of the code covered (52/52 blocks), and 3 passing tests, and 4 failing tests. And 7 warnings.

Step: Click on the warnings button.

Note that none of the warnings is related to the code under tests, and may therefore be suppressed.

Step: Select each of them click on suppress.

We should see this lines added in PexAssemblyInfo.cs:

Step: Do a “Run” from the exploration Results window OR “Run IntelliTest” on the IntelliTest (or on Handle method.).

Now we should see all of the code covered (52/52 blocks), and 3 passing tests, and 4 failing tests. And 0 warnings.

2 tests fails because they uncover a NulReferenceException; when ‘e’ is null (in that case e.AggregateId will raise the exception).

One of the test uncovers a DivideByZeroException. This will happen if stats.GamesPlayed has value of 1. In that case, the statement stats.GamesPlayed-- will make it 0, and subsequently stats.Profit / stats.GamesPlayed will raise the exception.

One of the tests uncovers an OverflowException. For the following data values: stats.Profit = -2147483648 and stats.GamesPlayed = -1 the statement stats.ProfitPerGame = stats.Profit == 0 ? 0 : stats.Profit /
stats.GamesPlayed; will raise the exception. We can see this from the details pane (the exact values seen may differ from those shown here).

IntelliTest has generated tests that uncover errors in the code. If we plug in more assertions about the expected behaviour then it will generate tests for validating that as well!

Q.E.D.