Architecting a Disaster Recovery plan for Multi-tier Web Applications using Microsoft Azure Site Recovery

Application software in an organization is the engine of business productivity and the most visible portion of its IT environment. Whether it be intranet applications or customer facing web applications, each of these serve an important purpose and in the event of a disaster it can be utmost critical for an organization to bring up the web applications with minimal downtime. Hence, IT administrators are always on the lookout for a fool-proof disaster recovery strategy.

Many DR strategies require a huge degree of precision and effort not only while architecting the plan but also for day-to-day monitoring of the web applications. A typical DR strategy, for example, might require the user to set up an IIS server on the secondary site and hence add to the complexity of infrastructure setup and maintenance. The user will be required to backup various configuration files and in the event of a disaster, he will need to ensure that the steps to recover are followed precisely to recover the entire application. The myriad of steps involved leave an IT admin struggling to be able to recover the applications at the last minute. Also, execution of the plan requires the user to be quite knowledgeable about the application configuration, adding to the risk and increased investment in educating the IT personnel.

Microsoft Azure Site Recovery Plans allow the creation of a single click failover solution for applications thus achieving the below objectives:

• Fully automated disaster recovery using a single-click failover
• Better RTO (Recovery Time Objective)
• Ease of performing DR drills to validate the Recovery Plan

Creating a Recovery Plan for a multi-tier IIS web application using Microsoft Azure Site Recovery

A Recovery Plan allows sequencing the failover of various tiers in a multi-tier application, hence, maintaining application consistency. Follow the below steps while creating a Recovery Plan for a multi-tier web application. The below steps refer to creation of a recovery plan when using another on-premises data center as a target and using Hyper-V Replica as the replication technology.

1. A typical multi-tier IIS web application will consist of a database tier with SQL VMs, the web tier constituted by a web server and an application tier. For using Hyper-V Replica as the replication technology, the different tiers should be virtualized. Add the virtual machines with the various tiers to a cloud and protect them to a target cloud on the secondary site.
2. Map the VM network on the primary site with the VM network on the secondary site to ensure that the virtual machines for the application come up in the specified network and hence are accessible by the other tiers as well as the customers.Please note that integration services should be installed on your virtual machine in order for it to be allocated an IP address in the mapped network.
3. Create a recovery plan with the virtual machine in the database tier added under Group 1 to ensure that they are shutdown last and brought up first.
4. Add the virtual machines in the application tier under Group 2 such that they are brought up after the database tier has been brought up.
5. Add the virtual machines in the web tier (with the IIS server) in Group 3 such that they are brought up after the application tier has been brought up.
6. If you are using a static IPs for the VMs, you will need to add a script to the recovery plan after Group 3 in order to update dependencies for the IIS web application. If you are using dynamic IPs, no more steps are required. You can use the script template accompanying this post in order to update these dependencies as described in the example below.
7. When using static IPs, you will also need to update the DNS with entries for the IP addresses of the virtual machines and the web application as specified here.

Dependencies to be updated for a multi-tier IIS web application post failover when using static IPs are:

1.Site bindings for the application:

Every site consists of binding information that includes the type of binding, the IP address at which the IIS server listens to the requests for the site, the port number and the host names for the site.At the time of a failover, these bindings might need to be updated if there is a change in the IP address associated with them.

Note: If you have marked ‘all unassigned’ for the site binding as in the example below, you will not need to update this binding post failover. Also, if the IP address associated with a site is not changed post failover, the site binding need not be updated (Retention of the IP address depends on the network architecture and subnets assigned to the primary and recovery sites and hence may or may not be feasible for your organization.)

If however, you have associated the IP address with a site, you will need to update all site bindings with the new IP address.

2.Connection string in an application’s web.config: The connection string specifies the database that the web site communicates with.

Note: If the connection string carries the name of the database VM, no further steps will be needed post failover and the application will be able to automatically communicate to the DB. Also, if the IP address for the database VM is retained, it will not be needed to update the connection string.

If however, the connection string refers to the database VM using an IP address, it will need to be updated post failover. E.g. the below connection string points to the DB with IP 127.0.1.2

3.The SSL cert binding for an https connection

Websites can have an associated SSL certificate that helps in ensuring a secure communication between the webserver and the user’s browser. If the website has an https connection and an associated https site binding to the IP address of the IIS server with an SSL cert binding, a new site binding will need to be added for the cert with the IP of the IIS VM post failover.

The SSL cert can be issued against-

a) The fully qualified domain name of the website

b) The name of the server

c) A wildcard certificate for the domain name

d) An IP address – If the SSL cert is issued against the IP of the IIS server, another SSL cert will need to be issued against the IP address of the IIS server on the secondary site and an additional SSL binding for this certificate will need to be created. Hence, it is advisable to not use an SSL cert issued against IP. This is a less widely used option and will soon be deprecated as per new CA/browser forum changes.

4.DNS Update

You will need to update the DNS server with a mapping of the IP address of the recovery site IIS server IP against the host header of the website. It is advisable to have a lower TTL value for the DNS server so that the DNS update can happen and the application is up within the expected RTO.

5.Update the dependency between the web and the application tier

If you have an application specific dependency based on the IP address of the VMs, you will need to update this dependency post failover.

A Sample Recovery Plan for an IIS web application with IIS front end and SQL as the backend with script to update dependencies

For the below example, I am using a 2 tier CMS web application, Umbraco, with IIS 7.5 in the front-end and SQL server 2012 in the backend.

exampleumbracosite is the name of the website that I need to protect with site name as “UmbracoSite”.

“iisvm” is the VM with the IIS server installed on it.

“iisvm_dbtier” is the name of the VM with the SQL DB in the backend.

Steps to protect the web application:

1. ‘iisvm’ and  ‘iisvm_dbtier’ are added to a cloud that is configured for protection to a target cloud on the recovery site and protection is enabled on them.

2. The VM network on the primary site is mapped to the VM network on the secondary site.

3. Create a recovery plan with iisvm_dbtier in Group 1 and iisvm in Group 2.

4. Use the script template below to create a script and add it to the recovery plan in order to be executed on ‘iisvm’ on the secondary site post failover. In case of a 2 VMM server setup, the script needs to be added to the VMM library on the secondary site. Ensure that the script is signed or the execution policy for the VMM server allows scripts to be remotely run on the VMM server.

# Replace all parameters in the Parameters Block template below with the actual values needed for the IIS web application. 
# Replace "username" with the username required to login to the IISVM. The user should have the permissions to access and configure IIS application data.
# Replace "password" with the password required for the above user to login to the IISVM.
# Replace "UmbracoSite" with the site name for the IIS website as specified on the IIS server.
# Replace "iisvm" with the name of the IIS server VM.
# Replace example values of binding information for Https bindings for your site with the Site binding information needed for your website. Use the format IP address : Port number : Host header for the site.
# Replace example values of binding information for Http bindings for your site with the Site binding information needed for your website. Use the format IP address : Port number : Host header for the site.
# Replace "UmbracoDbDSN" with the name of the connection string specified on the IIS server.
# Replace example values for the connection string with the connection string used to connect to the database on the IIS server in the correct format.
# Replace example values for the SslBindingList with the values of SslBinding and the corresponding certificate thumbprint for the https site bindings in the format Ip address!Port Number. Add all SslBindings and thumbprints needed for the site in separate rows in the example format.

#Parameters Block

$IISVMUsername = "username"                                                                          
$IISVMPassword = "password" 
$SiteName = "UmbracoSite"                                                                                           
$IISVMName = "iisvm"												    
$BindingInformationListHttps = "10.171.57.167:443:exampleumbracosite","10.171.57.168:443:exampleumbracosite2"                     
$BindingInformationListHttp = "10.171.57.167:80:exampleumbracosite","10.171.57.167:80:exampleumbracosite2"
$connectionName = "UmbracoDbDSN"
$connectionStringNew = "server=10.171.57.126;database=Umbraco_db;user id=umbracouser;password=222_bbbb"  

$SslBindingList = @(
		     ("10.172.57.167!443", "221A3C1275693098031A1002D49C29AC9C463359"), 
		     ("10.172.57.168!443" ,"311A3C1275693098031A1002D49C29AC9C463359")
		   )       	
#Parameters Block End

$pass = ConvertTo-SecureString -AsPlainText $IISVMPassword -Force
$cred = New-Object System.Management.Automation.PSCredential -ArgumentList $IISVMUsername,$pass	

$pathVariable = $env:windir+"\system32\inetsrv\"		                             															                                                                                                                             

$ScriptBlockContent = {param($BindingInformationListHttps,$connectionStringNew,$SiteName,$BindingInformationListHttp,$SslBindingList,$connectionName,
$pathVariable)

NET STOP W3SVC;
cd $pathvariable

#Add new site bindings corresponding to the IP address on the recovery site
foreach($BindingInformation in $BindingInformationListHttps) {
.\appcmd set site /site.name:$SiteName /+"bindings.[protocol='https',bindingInformation='$BindingInformation']"
}

foreach($BindingInformation in $BindingInformationListHttp) {
.\appcmd set site /site.name:$SiteName /+"bindings.[protocol='http',bindingInformation='$BindingInformation']"
}

#Add an SSL cert binding for the new IP
import-module webadministration
foreach($SslBinding in $SslBindingList) {
$Thumbprint = $SslBinding[1]
$SslBindingValue = $SslBinding[0]
Get-ChildItem cert:\LocalMachine\My | where { $_.Thumbprint -match $Thumbprint } | New-Item IIS:\SslBindings\$SslBindingValue
}

# Edit the connection string values

.\appcmd set config $SiteName /section:connectionstrings /"[name='$connectionName'].connectionstring:$connectionStringNew";
NET START W3SVC;}

Invoke-command -ComputerName $IISVMName -scriptBlock $ScriptBlockContent -Argumentlist $BindingInformationListHttps,$ConnectionStringNew,$SiteName,$BindingInformationListHttp,$SslBindingList,$connectionName,$pathVariable -Credential $cred

The script template needs to be edited to update parameters as per the application configuration:

1. IISVMUsername: This is the username needed to execute the PS script on the IIS server VM. This user should have administrative privileges on the iisvm.
2. IISVMPassword: This is the password needed for the IISVMUsername to be able to access the IIS VM.
3. SiteName: This is the name of the site as specified in the IIS server.
4. IISVMName: This is the name of the VM with the IIS server.
5. BindingInformationListHttps: This is an array of the site binding information for https type of connections that need to be updated post failover. Every binding is of the format IP address: Port number: Host header. E.g. for the exampleumbracosite, the bindings are “10.171.57.167:443:exampleumbracosite“,”10.171.57.168:443:exampleumbracosite2“.The script will add binding information specific to these IP addresses post failover. You can get the IP addresses of the replica VMs by logging into the secondary VMM.
6. BindingInformationListHttp: This is an array of site binding information for http type of connections that need to be updated post failover. Every binding is of the format IP address: Port number: Host header. E.g. for exampleumbracosite, the bindings are “10.171.57.167:80:exampleumbracosite“,”10.171.57.167:80:exampleumbracosite2“
7. connectionName: This is the name of the database connection as seen on the IIS server.
8. connectionStringNew: This is the value of the connection string required to connect to the SQL database from the web applications. This connection string is visible under connection strings in the website content view on the IIS server. Add the connection string in the correct format as “server=10.171.57.126;database=Umbraco_db;user id=umbracouser;password=222_bbbb”
9. SslBindingList: This is the value of the SSL bindings for all https bindings. The required format is IP address!port number , Thumbprint. Enter all the SslBindings corresponding to the IP addresses that exist for the website.
10. Use the script template to create a script to be added to the SCVMM library on the primary site. Update the script parameters with IP addresses to be used on the primary site.
11. If you are using a single VMM server, add both the scripts to SCVMM library and add contextual parameter ‘FailoverDirection’ to determine which code needs to be run. Details on how to use contextual variables with scripts can be found here.

Steps to perform a test failover of the Recovery Plan

After the recovery plan has been created, perform a test failover of the recovery plan VMs by specifying a test VM network at the time of failover.

1. While doing a test failover use a VM network with an IP Pool that has the same address range as the IP Pool used for the production network for the recovery site VM network. The test network should be isolated form the production recovery site VM network. This is to ensure that the production network resources are available for a planned failover during the duration of the test failover.

2. Update the DNS for the test network for the website. Check for the website connectivity in the isolated network.

3. Complete the test failover to delete the test VMs.

Planned/Unplanned failover of the Recovery Plan

In the event of a disaster you can perform a planned or unplanned failover of the Recovery Plan and the website should be up and running after the recovery plan failover is complete.

If you have any application configurations different than the ones mentioned in this blog, please leave a comment here and we will try to address your query. We will post a following blog with instructions on how to create a recovery plan when failing over from an on-premises data center to Microsoft Azure using Azure Site Recovery.