Using Selenium with Cloud Load Testing

This blog is an introduction to how users can execute Selenium tests using Cloud-based Load Testing from Visual Studio Online. 

Introduction

Let's get familiar with Cloud-based Load Testing (CLT) from Visual Studio Online (VSO) and Selenium.

• Cloud-base Load Testing:
• A good introduction to CLT with VSO and key feature can be found here - Introducing Cloud-based Load Testing
• To get started quickly follow the instructions at this link - Load Testing in cloud
• Selenium
• Selenium automates browsers Quote: Selenium Official Website
• Please visit the official web site to get an overall idea of Selenium
• For this tutorial we will be specifically using a Selenium Nuget Package and Phantomjs Nuget Package in Visual Studio

Let's get started.

Authoring Selenium unit tests in Visual Studio

1. Let's start by creating a Unit Test Project in Visual Studio. File -> New Project -> Templates -> Visual C# -> Test -> Unit Test Project
2. Once the library is created, install Selenium Nuget Package. Right Click on the project -> Manage Nuget Packages… -> "Search for Selenium" -> Install

3. After installing the Nuget package, the references would include WebDriver reference.
4. Download required Web drivers. For the sake of this example let's use PhantomJs. You can search for Phantomjs in the Nuget Package Manager and install it just as you did for Selenium. Change the property of phantomjs.exe "Copy to Output Directory" as "Copy if Newer"

5. You can create a CS file to get started with writing Unit Tests. Let's add a small unit test using Selenium

using Microsoft.VisualStudio.TestTools.UnitTesting;
 using OpenQA.Selenium;
 using OpenQA.Selenium.PhantomJS;
 using System;
 using System.Text;

 namespace SeleniumSample
 {
 [TestClass]
 public class SeleniumTests
 {
 [TestMethod]
 public void TheBingSearchTest()
 {
 TestContext.BeginTimer("BingSearchTest_Navigate");
 _driver.Navigate().GoToUrl("http://www.bing.com/");
 TestContext.EndTimer("BingSearchTest_Navigate");

 TestContext.BeginTimer("BingSearchTest_SearchBHarry");
 _driver.FindElement(By.Id("sb_form_q")).SendKeys("Brian harry blog");
 _driver.FindElement(By.Id("sb_form_go")).Click();
 TestContext.EndTimer("BingSearchTest_SearchBHarry");

 var elementText = _driver.FindElement(By.XPath("//ol[@id='b_results']/li/h2/a"));
 Assert.IsTrue(elementText.Text.Equals("Brian Harry's blog - Site Home - MSDN Blogs"), "Verified title of the blog page");
 }

 public TestContext TestContext { get; set; }

 #region Additional test attributes

 [TestInitialize]
 public void SetupTestSuite()
 {
 Console.WriteLine("Test init called: {0}");
 _driver = new PhantomJSDriver();
 }

 [TestCleanup]
 public void CleanupTestSuite()
 {
 _driver.Quit();
 }
 #endregion
 private IWebDriver _driver;
 }
 }

6. Right click on the solution and add a "Test Settings" File. This is important* in order to run the above code using Test Explorer (Test -> Windows -> Test Explorer)

7. Configure TestSettings to include phantomjs.exe as a deployment item

8. Build the solution.
9. From Test -> Test Settings -> Select Test Settings File, choose the testsettings file that is created in the previous step.
10. Verify this runs on your local machine using Test Explorer (Test> Windows> Test Explorer) to check if things are working fine.

Adding Load Test for the unit tests

1. Add a loadtest to the project (Right click -> Add ->  Load Test …). In the load test configuration wizard, in the "Text Mix", click add and choose unit test of your interest, for example: TheBingSearchTest
2. Verify the load runs on your local machine.

Running Selenium Tests on CLT On VSO

1. Congratulations you are just one click to run your tests on Microsoft Azure cloud!
2. From Test -> Test Settings -> Select Test Settings File, choose the Local.testsettings file that is created when the load test project is added.
3. Open this file change the option to "Run Tests using Visual Studio Online"

              For more information please visit this blog to get a good understanding of how you can run your tests on Cloud - http://www.visualstudio.com/get-started/load-test-your-app-vs

4. Once the run is completed and you have downloaded the report. Click on View report as mentioned in the above link.
5. You would be able get a lot of insights into how the test execution happened. You can get a view of the transactions that happened in your tests (BingSearchTest_Navigate is a transaction in your test). It would appear like

Please give this a try and do let us know your feedback :). Feel free to reach us at vsoloadtest[at]microsoft[dot]com 

Issues

1. If you try running the unit test without adding the Load Test Project you would run into System.NotSupported Exception at TestContext.BeginTimer(). It’s a known issue

• A connect bug tracking it - https://connect.microsoft.com/VisualStudio/feedback/details/774712/unit-test-testcontext-begintimer-throws-notsupportedexception
• An alternative that is blogged - http://karlz.net/blog/index.php/2012/11/17/where-is-testcontext-begintimer/

               By adding a testsettings file the above issue could be avoided :)