Azure Automation: Runbook Input, Output, and Nested Runbooks

With Azure Automation, a new capability in Microsoft Azure, Dev/Ops and IT professionals are able to create and run runbooks to automate repetitive and complex tasks on their Azure resources.  Azure Automation uses the PowerShell Workflow engine to run runbooks, which means that runbooks are created as PowerShell Workflows (see the Runbook Concepts article for an introduction).

As you create Azure Automation runbooks, you will want to be aware of and follow best practices in order to get the most out of the system.  Creating runbooks involves knowledge of PowerShell, PowerShell Workflow, and Azure Automation.  At the end of the day, if you are familiar with PowerShell, then creating Azure Automation runbooks is straightforward.

In this post, I will cover some of the basic concepts that will help you create high-quality runbooks.  Other posts in this series will cover other useful aspects of runbook creation and management.  The concepts covered in this post are the following:

• Defining input parameters
• Defining output type
• Calling child runbooks from within a runbook

Defining Input Parameters

Most runbooks require some input data to be provided before execution.  Thus, it is common during runbook authoring to define the input parameters that will be needed.  The main attributes that define an input parameter are these:

• Name
• Type (.Net type)
• Mandatory (or not)
• Default value (if any)

Azure Automation supports these attributes of input parameters for runbooks.  PowerShell supports more attributes of input parameters such as validation, aliases, and parameter sets; however, Azure Automation currently supports only the list above.

The code snippet below shows parameters defined in a runbook.  (For more illustration you can look at the Use-RunbookParameterSample runbook available from the Automation gallery on ScriptCenter.)

workflow Add-User
{
    param (
        [Parameter(Mandatory=$true)]
        [PSCredential] 
        $Credential,

        [Parameter(Mandatory=$true)]
        [object] 
        $FullName,

        [Parameter(Mandatory=$true)]
        [string] 
        $Alias,

        [Parameter(Mandatory=$false)]
        [string]
        $Company="Contoso",

        [Parameter(Mandatory=$false)]
        [boolean]
        $HasAdminRights = $false            
    )
    # Do work to add the user to the system...
}

If you want to invoke a runbook inline within a runbook then keep these things in mind about input parameters:

• If the input parameter is either a simple type or a complex object (like [PSCredential]), you can directly pass simple type value or the complex object as the input value as needed.
• An [object] type can be passed as a PowerShell hashtable, with name=value pairs for the object properties.  In the example below, the parameter “FullName” is passed as a hashtable.

    $name = @{"FirstName"="Joe";"MiddleName"="Bob";"LastName"="Smith"}
    Add-User -FullName $name -Alias "jsmith" -HasAdminRights $true -Credential $cred

In some cases, the [switch] parameter type can be useful in Azure Automation runbooks; however, in general it is more predictible to use the [boolean] type.  The reasons for using one or the other are explained in this blog post.  It is a best practice to use [boolean] rather than [switch], because it is less complicated to use in PowerShell Workflow.

In PowerShell Workflow when you call an activity or another workflow (runbook) all parameters must be referenced by name and not position.  To be safe, it is a good practice in Workflow to always use named parameters when calling another runbook, an activity, or a cmdlet.

Best Practice:  As a best practice, you should be very explicit when declaring input parameters to your runbooks.  Take the Add-User example above as a guide and always include the parameter Type, Name, and whether it is Mandatory or not.  Note that if you set a default value for a parameter, then it will be regarded by PowerShell an optional parameter regardless of how you set the Mandatory attribute.  If you leave out the Mandatory attribute then by default the parameter is optional; however, it is always best to explicitly declare this.  When you name a parameter use letters, numbers, and underscore characters (don’t use the hyphen character because parameter names with hyphens need special handling that you want to avoid).

If you want to be able to start your runbooks from the Azure Automation portal using the Start Runbook wizard UI then keep these things in mind about input parameters:

• The wizard UI allows you to input values for runbook parameters that can be represented by number, string, datetime, switch, boolean, Azure Automation credential asset name, JSON array, or JSON object.
• If a runbook has a parameter with a default value, this default value will not appear in the UI; however, if you leave the parameter blank in the UI, then the default value will be used when the runbook runs (we are looking at improving this experience in a later release).
• If a runbook parameter takes an [array] or [object] type, then these must be passed in JSON format in the start runbook dialog.  For example:

• A parameter of type [object] that expects a property bag can be passed in the UI with a JSON string formatted like this: {“StringParam”:”Joe”,”IntParam”:42,”BoolParam”:true}.
• A parameter of type [array] can be input with a JSON string formatted like this: [“Joe”,42,true].

• If a runbook parameter takes a PSCredential type then you need to pass the string name of a Azure Automation credential asset.  Behind the scenes, the Azure Automation credential asset with that name will be retrieved and passed to the runbook.

If you want to start a runbook asynchronously from the PowerShell console or within a runbook, use the Start-AzureAutomationRunbook cmdlet (more about this cmdlet later in this post).  Keep these things in mind about input parameters (and see the code example below):

• Input parameters to the runbook that is started by Start-AzureAutomationRunbook activity are passed in a hashtable as key/value pairs, where key is parameter name and value is value of the parameter.
• The runbook that is started by Start-AzureAutomationRunbook should have input parameters that can be represented by number, string, datetime, switch, boolean, Azure Automation credential asset name, JSON array, or JSON object.
• If an input parameter is a PSCredential type you need to pass the string name of an Azure Automation credential asset.  Behind the scenes, the credential asset with that name will be retrieved and passed to the runbook.
• If an input parameter is a switch type then the parameter needs to be declared in the hashtable with a Boolean value: $true or $false.  For example, if HasAdminRights was a [switch] parameter it would be declared like the example below.  It is also be declared this same way if it is a Boolean parameter.
• If the input parameter is a property bag, then define it as [object] type in the called runbook and pass a PowerShell hashtable object as the input value.  For example, see the $name parameter below.
• If the input parameter is a complex object (e.g., [System.Diagnostics.Process]), then define it as [object] type in the called runbook and pass the complex object as the input value.

    $name = @{"FirstName"="Joe";"MiddleName"="Bob";"LastName"="Smith"}
    $params = @{"Credential"="MyCred";"Alias"="jsmith";"FullName"=$name;"HasAdminRights"=$true}
    $job = Start-AzureAutomationRunbook `
                -Name "Add-User" `
                -Parameters $params `
                -AutomationAccountName $account

Something to keep in mind during the assignment of complex types to input parameters is whether the runbook will ever be called from another runbook and if so whether it will be invoked inline or started using Start-AzureAutomationRunbook.  If you think that it will only be invoked inline, then you can assign input parameters that are complex types, because when you invoke a runbook inline you can pass the complex type directly.  However, if the runbook will be started manually via the UI or started as a child runbook with Start-AzureAutomationRunbook, then you will need to set the type for any complex type to [object], because you can pass only a JSON array or JSON hashtable through the web service.  As a best practice, if you are not sure how a runbook will be started, then define complex input parameters as type [object].

Defining Output Type

PowerShell has long supported the definition of output type in functions and cmdlets with the use of the OutputType attribute.  This attribute has no effect during runtime, and instead has been provided as a way for tools to learn, at design time, the object types that cmdlets output without having to run them.

OutputType can be used in PowerShell Workflow, and you should include OutputType in runbooks that have output.  Place the OutputType attribute near the top of the runbook, just before any parameter declarations.  Here is an example of the use of OutputType in a runbook.

workflow Get-UserNames
{
    [OutputType([string])]

    $names = @()
    # Do work here to get the names...

    Write-Output $names
}

As Azure Automation evolves, and the toolset for creating runbooks is expanded and enhanced, it will be important that cmdlets and runbooks include the OutputType definition and then adhere to that contract.

Best Practice:  Always include OutputType in your cmdlets and runbooks that have output.

Calling Other Runbooks from a Runbook

One of the best practices when creating code of any kind is modularization: creating discrete, reusable code units.  For Azure Automation this means putting self-contained tasks within cmdlets and runbooks, and then calling those cmdlets and runbooks in runbooks that need the functionality.  Thus, it could be common practice for a parent runbook to call one or more child runbooks as part of the process being executed.

There are two ways to call child runbooks in Azure Automation:

• Invoke inline
• Start with Start-AzureAutomationRunbook cmdlet

As a side note, we use the general terms “nested” or “child” for any child runbook that is called from a parent runbook.  PowerShell uses the term “invoke” for operations that are initiated and run synchronously and the term “start” for operations that are initiated and run asynchronously; we will follow this precedent and use the terms “invoke” and “start” for synchronous and asynchronous child runbooks, respectively.

Invoke a Runbook Inline

Runbooks that are invoked inline run in the same job as the parent runbook.  This means that the parent workflow will wait for a child runbook to finish (synchronous) before continuing with the next part of the process.  This also means that all exceptions thrown by the child runbook and all stream output produced by the child runbook are going to be associated with the parent job; therefore all tracking of child runbook execution and output will be through the job associated with the parent runbook.

When you start a runbook that has child runbooks invoked inline, the child runbooks (and all descendants) get compiled into the parent runbook before execution starts.  During this compilation phase the parent runbook is parsed for the names of child runbooks; this parsing happens recursively through all descendant runbooks.  When the complete list of runbooks is obtained the scripts for these runbooks are retrieved from storage and assembled into a single file that is passed to the PowerShell workflow engine.  For this reason, at the time a runbook job is submitted the parent and descendant runbooks must already be published otherwise an exception will occur during compilation.  Currently, the order of publishing also matters: you must publish the child runbooks first and then publish the parent.  Similarly, when testing the parent runbook in the Azure Automation authoring page you must first publish the child runbooks and then you can test the parent.  Another consequence of this is that you cannot use a variable to pass the name of an child runbook invoked inline: you must always explicitly name the child runbook within the parent runbook.

Here is an example of a runbook that invokes two child runbooks inline: one child runbook returns output and the other child runbook does not return output.

workflow Process-VMs
{
    param (
        [Parameter(Mandatory=$true)]
        [string]
        $ScaleUnit
    )

    # Invoke a child runbook that has a return object
    $vms = Get-VMs -scaleunit $ScaleUnit

    # Invoke a child runbook that has no return object
    Do-StuffToVMs -vm $vms
}

Start a Runbook with the Start-AzureAutomationRunbook Cmdlet

You can start a runbook in a separate job by using the Start-AzureAutomationRunbook cmdlet (which is one of the Azure Automation cmdlets in the Azure PowerShell module, that comes pre-imported into Azure Automation).  When you start a child runbook using Start-AzureAutomationRunbook a new job is created for the runbook.

When a child runbook is started in this way the parent runbook does not wait for the child runbook to finish before continuing (asynchronous).  This approach is useful in cases when a parent runbook wants to spin off processes and then forget about them.  However, it does come at the cost of creating more jobs in the system, which can make troubleshooting somewhat more involved; and it is also more involved if you need to get back output from the child runbook.

Below is the code of a runbook, Start-ChildRunbook, that uses Start-AzureAutomationRunbook to start a child runbook and provides options to wait for the job to finish and get output. This runbook also uses other Azure automation cmdlets like Get-AzureAutomationJob and Get-AzureAutomationJobOutput.  This utility runbook can be very useful in your runbooks when you want to start child runbooks.  It encapsulates all of the work of starting the child runbook and getting back any output. You can download this helper runbook from Script Center and import it into your Azure Automation account for general use.

workflow Start-ChildRunbook
{
    [OutputType([object])]

    param (
        [Parameter(Mandatory=$true)]
        [string] 
        $ChildRunbookName,

        [Parameter(Mandatory=$false)]
        [hashtable] 
        $ChildRunbookInputParams,

        [Parameter(Mandatory=$true)]
        [string] 
        $AzureConnectionName,

        [Parameter(Mandatory=$true)]
        [string] 
        $AutomationAccountName,

        [Parameter(Mandatory=$false)]
        [boolean] 
        $WaitForJobCompletion = $false,

        [Parameter(Mandatory=$false)]
        [boolean] 
        $ReturnJobOutput = $false,

        [Parameter(Mandatory=$false)]
        [int] 
        $JobPollingIntervalInSeconds = 10,

        [Parameter(Mandatory=$false)]
        [int] 
        $JobPollingTimeoutInSeconds = 600
    )

   # Determine if parameter values are incompatible
   if(!$WaitForJobCompletion -and $ReturnJobOutput) {
       $msg = "The parameters WaitForJobCompletion and ReturnJobOutput must both "
       $msg += "be true if you want job output returned."
       throw ($msg)
   }

    # Connect to Azure so that this runbook can call the Azure cmdlets
    Connect-Azure -AzureConnectionName $AzureConnectionName

    InlineScript {
        # Convert the params in the Workflow scope into params in InlineScript scope
        $AutomationAccountName = $using:AutomationAccountName
        $AzureConnectionName = $using:AzureConnectionName
        $ChildRunbookInputParams = $using:ChildRunbookInputParams
        $ChildRunbookName = $using:ChildRunbookName
        $JobPollingIntervalInSeconds = $using:JobPollingIntervalInSeconds
        $JobPollingTimeoutInSeconds = $using:JobPollingTimeoutInSeconds
        $ReturnJobOutput = $using:ReturnJobOutput
        $WaitForJobCompletion = $using:WaitForJobCompletion

        # Select the Azure subscription we will be working against
        Select-AzureSubscription -SubscriptionName $AzureConnectionName

        if ($ChildRunbookInputParams -eq $null) { $ChildRunbookInputParams = @{} }

        # Start the child runbook and get the job returned
        $job = Start-AzureAutomationRunbook `
                    -Name $ChildRunbookName `
                    -Parameters $ChildRunbookInputParams `
                    -AutomationAccountName $AutomationAccountName `
                    -ErrorAction "Stop"

        # Determine if there is a job and if the job output is wanted or not
        if ($job -eq $null) {
            # No job was created, so throw an exception
            throw ("No job was created for runbook: $ChildRunbookName.")
        }
        else {
            # There is a job

            # Log the started runbook’s job id for tracking
            Write-Verbose "Started runbook: $ChildRunbookName. Job Id: $job.Id"

            if ($WaitForJobCompletion -eq $false) {
                # Don't wait for the job to finish, just return the job id
                Write-Output $job.Id
            }
            elseif ($WaitForJobCompletion -eq $true) {
                # Monitor the job until finish or timeout limit has been reached
                $maxDate = (Get-Date).AddSeconds($JobPollingTimeoutInSeconds)

                $doLoop = $true

                while($doLoop) {
                    Start-Sleep -s $JobPollingIntervalInSeconds

                    $job = Get-AzureAutomationJob `
                        -Id $job.Id `
                        -AutomationAccountName $AutomationAccountName

                    $status = $job.Status

                    $noTimeout = ($maxDate -ge (Get-Date))

                    if ($noTimeout -eq $false) {
                        $msg = "The job for runbook $ChildRunbookName did not "
                        $msg += "complete within the timeout limit of "
                        $msg += "$JobPollingTimeoutInSeconds seconds, so polling "
                        $msg += "for job completion was halted. The job will "
                        $msg += "continue running, but no job output will be returned."
                        throw ($msg)
                    }

                    $doLoop = (($status -ne "Completed") `
                              -and ($status -ne "Failed") `
                              -and ($status -ne "Suspended") `
                              -and ($status -ne "Stopped") `
                              -and $noTimeout)
                }

                if ($job.Status -eq "Completed") {
                    if ($ReturnJobOutput) {
                        # Get the output from job
                        $jobout = Get-AzureAutomationJobOutput `
                            -Id $job.Id `
                            -Stream Output `
                            -AutomationAccountName $AutomationAccountName

                        # Return the output string
                        Write-Output $jobout.Text
                    }
                    else {
                        # Return the job id
                        Write-Output $job.Id
                    }
                }
                else {
                    # The job did not complete successfully, so throw an exception
                    $msg = "The child runbook job did not complete successfully."
                    $msg += "  Job Status: $job.Status.  Runbook: $ChildRunbookName."
                    $msg += "  Job Id: $job.Id.  Job Exception: $job.Exception"
                    throw ($msg)
                }
            }
        }
    }
}

When you start a child runbook using Start-AzureAutomationRunbook the return value is a Job object.  If you want return data from the child runbook then you need to monitor the the job to learn when the job completes, and then extract any output.  In the runbook example above, you can see how to use Get-AzureAutomationJob and Get-AzureAutomationJobOutput cmdlets to monitor the job and retrieve output from the child runbook job.  Note that a timeout factor has been included to guarantee exit from the loop in the case of a job not completing within expected time.  Also note that if you start several child jobs the system does not guarantee the order in which the jobs will be started.

Comparison of Child Runbooks that are Invoked Inline versus Started with Start-AzureAutomationRunbook

Invoked Inline

• Pro
  • Parent and child runbooks run in the same job, so there are fewer jobs in the system, which allows easier job tracking.
  • The parent runbook waits for the child runbook to finish before continuing, and the parent runbook can directly get any return data from the child.
  • Exceptions thrown by the child runbook and stream output produced by the child are associated with the parent job, which can make troubleshooting easier, because there is only one job to investigate.
  • Runbook input parameters can be of any type, primitive to complex.
  • You are charged for the runtime of just one job, because the child runbook is running in the same job as the parent.
• Con
  • The parent runbook must wait for the child runbooks to complete, which can increase the overall time for the parent runbook to complete.
  • You cannot use a variable or parameter in the parent runbook to pass in the name of an inline child runbook.
  • The child runbook must be published before the parent runbook is published.
  • The parent and child runbooks must be in the same Automation account.

Started with Start-AzureAutomationRunbook

• Pro
  • The parent and child runbooks run in different jobs, which allows the parent to spin off multiple jobs that can run in parallel.
  • The parent runbook does not wait for the child runbooks, and so can continue processing while the child runbooks run.
  • You can use a parameter or variable in the parent runbook to pass in the name of the runbook to invoke.
  • You can start runbooks stored in different Automation accounts in the same subscription, and even in different Azure subscriptions as long as you have an Automation connection asset to that Azure subscription.
• Con
  • Parent and child runbooks run in different jobs, so exceptions and stream output are stored separately and must be tracked down separately, which may make troubleshooting more difficult.
  • More jobs are created in the system which can cause jobs to wait longer in the queue before starting.
  • Getting return data from a child runbook job is not straightforward.
  • When Start-AzureAutomationRunbook starts a runbook and waits for it to complete to get output, you get charged for the run time of both runbooks.  This is not the case though if you use Start
  • AzureAutomationRunbook to start a runbook and then immediately return the job id.
  • Runbook input parameters are restricted to primitive types, array, and object, because they must survive the JSON serialization of objects that occurs with the call through the web service.

Summary

Azure Automation is a useful tool that allows you to create runbooks that take advantage of the numerous beneficial features of the PowerShell Workflow engine.  With an understanding of some of the inner workings of Azure Automation and PowerShell Workflow, and of some best practices, you can create high-quality, reliable, and maintainable runbooks.

In this post, we have discussed several best practices for defining input parameters, for defining runbook output, and for calling child runbooks.  Now that you have read through the information, take the time to experiment with a few simple runbooks in Azure Automation using the examples so that you can see for yourself the power of runbooks and automation.  Thanks for reading, and have fun putting this new knowledge into practice to make your management of Azure resources simpler, more predictable, and more reliable.

Not an Azure Automation user yet? Sign up for the preview and then check out the Getting Started guide.