Testing private/intranet applications using Cloud-based load testing

Cloud-based Load Testing Service can be used for performance and scale testing of an application by generating load from Azure. This type of load generation can only hit/generate load on an internet/publically accessible application. But we have seen many times customer needs to load test their application which is not publically accessible. Reasons could be many, some of them are listed below:

1. Testing an internal application only –In many large scale organizations there are applications/web sites that have the cater the need of whole organization. It becomes crucial to test it with peak load to eradicate any performance/stress related bugs.
2. Testing the application internally before releasing it over internet –Before actually going into public for a big gung-ho launch, organizations love to make sure that there is no performance glitch or the worst site will not crash in high user load.

To provide support for above scenarios we are working on a feature using which users can load test their internal/ABF (application behind firewall) applications. Before talking about the solution, let us walk you through the following flow chart to figure out the best tailored solution as per your requirements. You can then read about a particular solution (1-5) and try it out.

1. Use default path, CLT will auto provision agents.  This is the default scenario for load testing using CLT where the application has a publicly available end point. The load testing service will automatically provision load agents (in Azure) to simulate the user load. Refer this page for more details on CLT usage.
2. Use ARM template to deploy your own load agents.  #1 solution provisions the agents in CLT service boundaries and user won’t be able to have control over it. If you wish to have control (access) over the load generation agents for any customization needs, you can deploy them yourself using ARM template in your Azure subscription. These machines will get registered with CLT can generate load. More details on this will follow in the same blog.
3. Use ARM template to deploy load agents in a VNet.    If the application under test (AuT) is inside a Azure VNet or if there is an ExpressRoute between application’s private network and Azure, you can use a pre-defined ARM template deploy IaaS VMs in Azure in a specific VNet to act as load agents. The machines will be provisioned in your Azure subscription and registered against your VSTS account. The VNet where you create these machines should have a line of sight to the application, so that the load generators are able to reach the app. More details on this can be seen in the further segments of this blog.
4. Use ARM template to deploy load agents with static IPs.  If you don’t have ExpressRoute connectivity and want to test apps hosted on-premises, you can use an ARM template to deploy IaaS VMs in Azure to act as load agents. Choose to create these VMs with static IPs that you can allow traffic from in the firewall, so that the necessary load can be generated. The machines will be provisioned in your Azure subscription and registered against your VSTS account. More details on this will follow in the same blog.
5. Use cloud load agents on your infrastructure.  We have also come up with a simple powershell script that can help you to configure physical or virtual machine(s) of your choice as load agents. These machine(s) will be registered against your VSTS account for load generation. You can read more on this by following a separate blog, ‘Use cloud load agents on your infrastructure’. 
6. Use Test Controller/Test Agents for on-premises testing on your infrastructure.  If you want to do test apps on-premises, but have constraints such as not being able to store results on the cloud for some reason (say, regulatory compliance) you can use the Test Controller / Test Agents based solution for load testing. This requires you to use your own infrastructure for load generation and results will be stored on SQL Server. https://msdn.microsoft.com/en-us/library/ms243155.aspx

Now we’ll deep dive into solution #2 and #3 in this blog.

This section talks about how you can provision load agent(s) using Azure IaaS VMs in details. User must have an Azure subscription where the IaaS VMs and related resources will be provisioned. This is primarily useful in following two use cases:

1. You want to test a private application which is not accessible through CLT/internet.
2. Bring your own Subscription (BYOS)– You have your own Azure subscription and want to leverage it for load testing. You can also use the Azure free credits, if you have.

Azure gives an edge here as users can spread their load testing across different geo-locations which we have seen many customers are interesting in these days.

Use ARM template to deploy load agents in a VNet

Following is the simple topology where load agents are present under user’s VNet and henceforth they will have a line of sight to the application. We have published one ARM template in github to help user to provision machines easily and quickly.

For this to work user must have an existing VNet as depicted in above diagram. VNet identification requires its resource group name as well. If you wish to use an existing subnet, you should have this info as well.

To deploy such rig we have published one ARM template. You can click on the following link to automatically load the ARM template in Azure management portal.

Provision load agents in an existing rig

Once you click on it, it will load the template in Azure portal and you’ll see following view. User can fill in the parameters and choose the subscription/resource group/location as per the requirement.

If you wish to dig deep into the template and modify as per your needs, you can check it out from following github repo:

https://github.com/Azure/azure-quickstart-templates/tree/master/201-vsts-cloudloadtest-rig-existing-vnet

Use ARM template to deploy load agents with static IPs

We have published another ARM template where user doesn’t need to have an existing VNet. This can be used for following two purposes:

1. If you don’t have ExpressRoute in Azure but want to do load testing using his own subscription, you can use this ARM template which just deploys a rig with its own VNet. If you need to test an private application, you can deploy the rig with static IPs (provided as an option), punch the firewall for these IP(s) to make a route for load agents.
2. If you want to have control over load generation agents (as CLT auto-provisioned agents can’t be accessed by the user). You can choose to have static/dynamic IPs for these VMs.

To leverage this, just click on following link to load the ARM template in Azure management portal.

Provision load agents in a new Vnet

Github repo link for this ARM template is as follows:

https://github.com/Azure/azure-quickstart-templates/tree/master/101-vsts-cloudloadtest-rig

Once you deploy the VMs, it may take 10-15 mins to have machines configured with CLT and ready for load test. The load test runs done on these agents will not be charged VUMs by the CLT service but user will incur the cost of Azure resources consumed under his subscription.

In ARM template we have set the machine size to ‘Standard_D4_V2’. This size machine(s) comes up with 8 CPU cores and 28 GB of memory. User can change this value at his end by editing the template. Refer this to know more about the azure machine sizes and capabilities.

How to queue a run using load agents

We are working on making it as first class experience in our product. Till then user can queue a run on these machines by having a specific context parameter in Visual Studio Load Test file.

Context parameter name – UseStaticLoadAgents

Context parameter value – true

In Visual Studio, it will look like something as follows:

User can set the number of machines to be used for a load test run through Agent core count property present in Run Settings. In user’s own load agents scenario every core is treated as a single machine. As shown in below image, 5 machines will be used for the run.

The runs done on user’s own load agents machines are not charged. User should be able to confirm this by looking into the status messages of the run.

Before you do that, we would recommend you to go through the following FAQs as well.

How to know about the registered machines with CLT

To check the status of machines configured under a VSTS account, users can download a basic script GetStaticMachines.ps1 from following link.

Download powershell script to list down the registered machines

After downloading it, please make sure you unblock the file.

Following are the mandatory parameters:

1. TeamServicesAccountName : It is the name of the VSTS account with which you want to get configured machines. Use just need to pass the account name here e.g., please put ‘xyz’ if your VSTS account url is https://xyz.visualstudio.com
2. PATToken : It is required for authentication. One has to first get the PAT token for the VSTS account. Follow this post to get it. The scope should be selected as ‘Load Test (read and write)’.

Example:

.\GetStaticMachines.ps1 -TeamServicesAccountName x -PATToken zzdw6bnzk2q73qsqxukmfonzycscdgmsl2quhqo24so7hrplctcq

Sample Output:

Account Uri – https://xyz.vsclt.visualstudio.com

machineName status lastHeartBeat

———– —— ————-

DDDSINGHAL016 Free 2016-04-26T12:41:00.6548647Z

DDDSINGHAL017 Free 2016-04-26T12:40:59.5005437Z

With this user can get idea about the status of his machines configured for load test run.

Frequently asked question –

1. How do the load agents communicate with CLT?The load agents will communicate with CLT using HTTPs protocol. Since these machines/VMs are inside user’s private network (Azure/on-prem), they can reach the Application under Test (AuT) easily. The results are published back to the CLT service so that the analysis can happen in similar manner as done for other type of load test runs in CLT.
2. How I am being charged for this?At present this feature is in preview mode and you will not incur load testing VUM charges for the runs where you deploy load agents on your premises or in your Azure subscription. However, you will be charged the applicable Azure VM costs.
3. Can I use these machines for some other purposeThese machines can be used other your other tasks as well but it is recommended to not have anything running while a load test run is in progress.
4. Can I shut down the machines where I have configured load test agent?Yes, the machines can be shut down when not in use. The load agent service will automatically start to receive commands from CLT, once the machine is up. If you are using Azure ARM template to deploy these agents, you can start/stop the VMs based on your need. You can also do this using a powershell script. Refer following link to more on this.https://gallery.technet.microsoft.com/scriptcenter/Stop-All-VMs-in-Specified-40c8531eYou are also recommended to delete the Azure resource group once you are done with load testing. You should be able to re-create it anytime later if you need to.https://azure.microsoft.com/en-in/documentation/articles/resource-group-portal/
5. I have proxy settings on my machines, will this work?We support only default proxy scenario i.e., when the proxy settings are controlled through IE and it uses the current user’s credentials to connect to proxy server. In other cases, please reach out to us.

With all this, you should be able to go ahead and try out the load testing. Do reach out to us on vsoloadtest@microsoft.com, if you have any query.

Happy Load Testing!