Azure Automation service is one of the main components that form Operations Management Suite (OMS). It can be deployed as standalone or used as part of Automation & Control offering in OMS. Azure Automation is a service that will help you in automating manual, long-running, error-prone and frequently repeated tasks that you need to perform in any cloud or on-premises environment. Currently, Azure Automation has two major capabilities: Process Automation and Configuration Management. This chapter focuses on the Process Automation capability.
When we talk about Process Automation within Azure Automation, we should mention that the service has on-premises equivalent products, which are System Center Orchestrator (SCOrch) and System Center Service Management Automation (SMA). Some of the advantages of Azure Automation over those on-premises products are:
These advantages over the on-premises products make Azure Automation much more flexible and easier service with a richer feature set.
Note: When working with Azure Automation, you must use the Azure portal at https://portal.azure.com/ Azure Automation is no longer available in the classic Azure Management Portal at https://manage.windowsazure.com
Note: Both of the principal capabilities of Azure Automation, Process Automation and Configuration Management, have free tiers. The free tier of Process Automation offers 500 minutes of job run time every month. Configuration Management is free for up to 5 nodes. The only requirement is that you have an Azure subscription. For detailed pricing information on Azure Automation, visit the Microsoft website https://azure.microsoft.com/en-us/pricing/details/automation/ when used as part of OMS license in Automation & Control offering, visit the Microsoft website http://www.microsoft.com/en-us/server-cloud/operations-management-suite/pricing.aspx
The first step to start using Azure Automation is to create an automation account. An automation account is an isolation boundary for your Azure Automation resources such as runbooks, Hybrid Runbook Workers (Hybrid Workers), DSC Nodes and DSC Configurations. Multiple Automation accounts can be created under one Azure subscription. Some of the most common reasons for multiple automation accounts include logical environments separation, physical location separation, team separation or billing separation. When an automation account is created, an Azure region must be specified where that account will reside. This designates where all Azure Automation resources for that account will be located. An automation account created in a specific region can manage Azure resources in other regions as well. In fact, Azure Automation can manage resources in other Azure subscriptions, on premise resources, other cloud resources and/or resources located in other clouds.
Note: An OMS Log Analytics Workspace can be linked to only one Azure Automation account as described in Chapter 1.
There are three ways for creating an Azure Automation account: The Azure portal, PowerShell (Azure Resource Manager module) or with an Azure Resource Manager (ARM) JSON template.
Note: Azure Automation can manage Azure resources created in the classic portal as well.
Follow these steps to create an Azure Automation Account in the Azure portal.
FIGURE 1. CREATION OF NEW AUTOMATION ACCOUNT
Name: OMSBook
Subscription: Choose suitable subscription
Resource Group: Create a new resource group, for example, InsideOMS (as shown in Figure 2).
Location: Choose your closest location or based on design requirements Create Azure Run As Account: Select Yes.
FIGURE 2. INPUT PARAMETERS FOR NEW AUTOMATION ACCOUNT
Note: "Create Azure Run As Account" option will be greyed out if you are using an account that does not have Azure AD admin rights.
FIGURE 3. NEW AUTOMATION ACCOUNT CREATED
The new Automation Account is now created and available for use, as shown in Figure 4.
FIGURE 4. AUTOMATION ACCOUNT BLADE
Notice that under Account Settings, Run As Accounts, Azure Run As Account and Azure Classic Run As Account are populated as shown in Figure 5. There are two Azure AD Service Principal created when Create Azure Run As Account option is selected when creating Automation Account. The service principal for Azure Run As Account is given contributor permissions on all Azure resources in the current subscription and the service principal for Azure Classic Run As account is given co-administrator permissions for Azure Classic resources. These service principals are imported as Connections and Certificates in Azure Automation assets and can be used in runbooks for managing Azure resources. These Run As Accounts will not be created when ARM or PowerShell is used for creating Azure Automation account.
FIGURE 5. AZURE AUTOMATION RUN AS ACCOUNTS
Instead of creating an Automation Account in the Azure portal, you can also deploy it using AzureRM PowerShell. You can download the full script from the GitHub repository provided in the link at the end of this section.
AzureRM PS module v4.0.0 or later is required for executing the PowerShell code in the next sections.
Step 1: Connect and Authenticate
You will begin by connecting and authenticating to your Azure subscription. This sample will also prompt you for which Azure subscription you wish to you use, easing the process for users working with multiple subscriptions.
# Provide credentials
$Creds = Get-Credential `
-Message 'Provide Azure Subscription Credentials...'
# Login to Azure
Login-AzureRmAccount`
-Credential $Creds`
-ErrorAction Stop | Out-Null
# Pick Subscription/TenantID
$Azure =
(Get-AzureRmSubscription`
-ErrorAction Stop |
Out-GridView`
-Title 'Select a Subscription/Tenant ID...'`
-PassThru)
# Select Subscription
Select-AzureRmSubscription `
-SubscriptionId $Azure.Id`
-TenantId $Azure.TenantId `
-ErrorAction Stop| Out-Null
Step 2: Create Resource Group
If the resource group does not already exist, you will need to create a resource group.
# Create Resource Group
New-AzureRmResourceGroup`
-Name 'InsideOMS2' `
-Location "West Europe"
Step 3: Create Automation Account
The last step is the actual creation of Automation account.
# Create Automation Account
New-AzureRmAutomationAccount`
-ResourceGroupName 'InsideOMS2'`
-Name 'OMSBook23' `
-Location "West Europe" `
-Plan Free
To check the results after running the script, go to the Azure portal and select Hamburger menu -> Resource groups -> InsideOMS.
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is CreateAutomationAccount.ps1.
The third option to create an Automation Account is to use an ARM json template.
The following example creates an Automation Account, with the same settings used earlier in the portal example.
To create an Automation Account using the provided json template, perform the following steps. You will first need to download the json template from the GitHub repository described below.
Download the Code
You can download the json template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is CreateAutomationAccount.json.
Step 1: Select Template Deployment
You will begin by selecting the 'Template Deployment' option, as detailed in the steps below.
FIGURE 6. TEMPLATE DEPLOYMENT IN AZURE MARKETPLACE
Step 2: Paste json template into 'Edit template' window
Next, you will paste the json template into the space provided.
1. Select Edit and paste the sample json template you downloaded into the window provided, replacing any sample json that may be present.
Click Save to save your changes. Step 3: Create Resource Group
The resource group is the container for deployment. Select an existing or create a new resource group.
FIGURE 7. CUSTOM TEMPLATE DEPLOYMENT
Before proceeding with using the Process Automation feature of Azure Automation, we will first review the solution components available to you when creating runbooks.
Process Automaton has three main components: Runbooks, Assets, and Jobs. Some of these components have sub-components.
A runbook is a collection of one or more linked activities that together automate a process or operation. A runbook can be a process as simple as starting a service, or a complex one, such as orchestrating installation of operating system updates in a distributed application consisting of multiple multi-server tiers. A runbook can execute tasks on any resource, whether it is located in a public cloud, on-premises, or in a hybrid cloud. Runbook automation can be used to automate processes on a variety of operating environments, including Windows, Linux, and containers, as well devices like network switches, firewalls, and load balancers.
As we mentioned earlier, Azure Automation uses PowerShell as a platform and for runbooks more specifically, uses Windows PowerShell or Windows PowerShell Workflow. PowerShell is a very popular platform that has many modules enabling you to execute actions against many application and services. These same modules can be uploaded into Azure Automation and used in runbooks. PowerShell can also execute actions against applications or services without modules by using their application programming interface (API). This functionality also surfaces in Azure Automation runbooks through the PowerShell platform.
In Azure Automation, there are four types of runbooks you can create – Graphical, Graphical PowerShell Workflow, PowerShell Workflow and PowerShell. You can view the runbooks in your automation account on the Runbooks blade, as shown in Figure 8.
FIGURE 8. RUNBOOKS BLADE
Graphical PowerShell Workflow runbooks are based on Windows PowerShell Workflow. They are authored in a graphical editor available in the Azure portal. The experience is similar to the Runbook Designer Console in System Center Orchestrator. The primary purpose of this type of runbook is to provide authoring capabilities for those who are not PowerShell savvy or who are used to authoring runbooks in System Center Orchestrator. This type of runbook cannot be authored outside of the Azure v2 portal. However, it is possible to export graphical runbooks and import them into the same or other automation accounts. When a graphical runbook is created, on the backend, the PowerShell Workflow code is generated, but that cannot be viewed or edited. The biggest advantage of a graphical runbook is that it visualizes the process created and that visualization can be understood by less technical people. As graphical runbooks use PowerShell Workflow code under the hood, they also inherit some of its advantages, like the ability to create checkpoints and resume the runbook from the last checkpoint upon failure, or to perform parallel processing by executing several tasks at the same time.
The graphical runbook is very similar to Graphical PowerShell workflow runbook. The only difference is that Graphical runbooks are based on PowerShell where Graphical PowerShell Workflow runbooks are based on PowerShell Workflow thus any of the advantages that workflows have are missing in this one.
As the name implies, PowerShell Workflow runbooks are based on Windows PowerShell Workflow. As such, they are written in simple text. They can be authored in the Azure portal via the text editor or offline in any text editor, such as Windows PowerShell Integrated Scripting Environment (ISE) with Azure Automation Authoring Toolkit, Notepad++ or even Visual Studio. When Workflow runbooks are authored offline, they can be saved in files with the .ps1 extension and imported into an Azure Automation account. Some of the advantages of a PowerShell Workflow runbook include parallel processing of activates, ability to create checkpoints and resume from last checkpoint upon failure and ability to execute complex logic. They also present some challenges you should be aware of, like time for Workflow Runbook compilation, familiarity with PowerShell Workflow code and deserialization of objects. When an object is returned by PowerShell Workflow cmdlet, that object is serialized, which means that you cannot have object method, and properties with complex types are deserialized.
The last type of runbook is the PowerShell runbook, which is written in standard PowerShell just like any other PowerShell script. They can be authored in the text editor in the Azure portal or offline in a tool such as PowerShell ISE. When using PowerShell runbooks, you can execute complex logic within a runbook, and call other runbooks. Generally speaking, PowerShell runbooks start faster than PowerShell workflow runbooks, since they do not need to be compiled before execution. However, comparing with the PowerShell Workflow runbooks, you lose capabilities like parallel processing, creating checkpoints and resuming from the last checkpoint upon failure.
Every type of Azure Automation runbook has its own advantages and disadvantages, which are summarized in table 1 for easier comparison.
Capability | Graphical | Graphical PowerShell Workflow | PowerShell Workflow | PowerShell |
Process Visualization | + | + | - | - |
Checkpoints and Resuming on error | - | + | + | - |
Parallel Processing | - | + | + | - |
Source Control Integration | - | - | + | + |
Hybrid Worker Support | + | + | + | + |
Webhooks | + | + | + | + |
Child runbook execution (Nesting) | Graphical, PowerShell | Graphical PowerShell Workflow, PowerShell Workflow | Graphical PowerShell Workflow, PowerShell Workflow | PowerShell |
Child runbook execution (via StartAzureAutomationRu nbook) | Any | Any | Any | Any |
Runbook Authoring | Azure portal | Azure portal | Azure portal or offline editor | Azure portal or offline editor |
Startup Time | Short | Longer, Depends on runbook complexity | Longer, Depends on runbook complexity | Short |
Complex Logic Implementation | Minimal | Minimal | Full | Full |
TABLE 8. AZURE AUTOMATION RUNBOOK TYPES COMPARISON
When a runbook is started, a job is created. The job represents a single instance of execution of a runbook. Once the job is created, it is picked up and run by an Azure Automation Worker. Azure Automation has pools of workers that are shared between multiple Automation accounts. Although jobs from multiple Automation accounts can run on the same Worker, they are isolated from each other. With the Hybrid Worker, which we will cover later in this chapter, there is the possibility to run jobs on-premises on your own servers. Jobs can be found in the Jobs blade, as shown in Figure 9, but also in a blade inside every runbook.
FIGURE 9. JOBS BLADE
Every Automation Account has an Assets store. The purpose of assets is to store information or resources that can be used by all runbooks in an Automation Account. The Assets store is also used by DSC configuration in Configuration Management capability. There are six types of assets – Schedules, Modules, Certificates, Connections, Variables, and Credentials. These assets can be managed by PowerShell, such as retrieval of assets, creation of assets, import, export, etc. Assets are located in the Assets blade and are categorized by their type, as shown in Figure 10.
FIGURE 10. ASSETS BLADE
Some assets can be exported and imported into another automation account with Azure Automation PowerShell commands.
In Azure Automation, certificates can be stored as Assets. Certificates are used by many services as an authentication mechanism. For example, Azure cmdlets allow you to authenticate with a certificate instead of credentials. For example, the run as accounts deployed with Automation account, when created via Azure portal, use a certificate asset.
Connections are a mix of variables and credentials. A good example would be a connection to an SMTP server, which will have fields such as username, password, SMTP Server and SMTP Port number. The advantage of connections compared to credentials and variables is that all the necessary information for connecting to a service or application is stored in a single object. When a connection is created, a type must be chosen. Connection type is much like a template that defines the properties of a connection and what values are allowed for them. Connection types are added through importing Azure Automation modules. When importing PowerShell modules into Azure Automation, be aware that not all modules include connection type definitions.
A Credential is a set of username and password values. The password is stored securely as a secure string. When the credential is retrieved inside a runbook, it is retrieved as a PSCredential object. Credentials are used for authenticating against servers, applications or services.
Modules in Azure Automation are almost the same as PowerShell Modules. A PowerShell module is a group of PowerShell cmdlets that are executed in PowerShell console, scripts, workflows or runbooks. An example for such cmdlet is Get-Service, which will return services running on a computer. That particular cmdlet is located in the
Microsoft.PowerShell.Management module. Azure Automation modules can be imported through the PowerShell Gallery at www.powershellgallery.com or manually. DSC resources are also imported as modules and are used by the Configuration Management capability.
Schedules enable a runbook to be executed automatically at a pre-defined time in the future, Schedules allow runbooks to be executed once or multiple times through recurring interval. Once a schedule is created, it can be attached to one or more runbooks. Schedules also are created for a specific time zone to reflect changes happening with daylight saving time in different countries. When creating a reoccurring schedule, you can also define the expiration date for the schedule.
Variables are containers for storing information. For example, you may have a variable named "OMSWorkspaceID" where you store the ID of your OMS Workspace. You can extract the value of that variable in a runbook when you want to install Microsoft Monitoring Agent (MMA, or simply 'the agent') on a server and connect it to your OMS Workspace. Variables can be of different types such as String, Boolean, DateTime, Integer and Not Specified. Variables also have the property to be encrypted or not. When a variable is not encrypted, the value can be seen by anyone who has access to the Automation account. When a variable is encrypted, it is stored securely in Azure Automation and can be retrieved only inside a runbook. For example, as the primary key for an OMS Workspace is sensitive data, it should be stored as encrypted variable if you want to use it inside Azure Automation account.
A common task for an operations or deployment team could be installing the Microsoft Monitoring Agent and connecting it to OMS workspace. For that purpose, we can create a runbook that automates the task and can be executed by those users who have access to the Automation Account without having access to the Azure virtual machine (VM) or to the VM itself. MMA will be installed through a VM extension for Azure VMs. We will use the Azure Resource Manager PowerShell cmdlets, so the target VM must be an Azure v2 VM. As we have four types of runbooks, we will see how that automated task looks in each type. Before creating the actual runbooks, we will create several assets that will be used by the runbooks.
Follow these steps to create an asset variable of type string named "AzureSubscriptionID".
FIGURE 11. CREATE "AZURESUBSCRIPTIONID" VARIABLE
Name: AzureSubscriptionID
Type: String
Value: <your Azure Subscription ID>
Encrypted: No
Follow these steps to create asset variable of type string named "OMSWorkspaceID".
FIGURE 12. CREATE "OMSWORKSPACEID" VARIABLE
Name: OMSWorkspaceID
Type: String
Value: <your OMS Workspace ID>
Encrypted: No
Note: You can find your OMS Workspace ID on OMS portal at https://www.mms.microsoft.com under Overview, Settings, Connected Sources, Windows Servers as shown in Figure 13.
FIGURE 13. OMS WORKSPACE ID, PRIMARY KEY, AND SECONDARY KEY
Follow these steps to create asset variable of type string that is encrypted, named "OMSWorkspacePrimaryKey".
FIGURE 14. CREATE "OMSWORKSPACEPRIMARYKEY" VARIABLE
Name: OMSWorkspacePrimaryKey
Type: String
Value: <your OMS Workspace Primary Key> Encrypted: Yes
Note: You can find your OMS Workspace Primary Key on OMS portal at mms.microsoft.com under Overview, Settings, Connected Sources, Windows Servers as shown in Figure 13. The Primary Key should be stored as an encrypted variable for security reasons.
You can also create variables with ARM PowerShell cmdlets. You will have to configure the values for the variables when you want to create the variables via the following PowerShell code:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is CreateAutomationVariables.ps1.
The third alternative is to create those via Azure Resource Manager Template:
Download the Code
You can download the template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is CreateAutomationVariables.json.
Follow these steps to create a Credential asset named "AzureCredentials".
FIGURE 15. CREATE "AZURECREDENTIALS" CREDENTIAL
Name: AzureCredentials
User Name: <your service account user name>
Password: <your service account password>
Confirm Password: <your service account password>
Note: For best experience better create an organizational account (one created in the Azure Active Directory associated with your Azure subscription) and not a Microsoft (Live) account. This account must have full access to your Azure subscription or to the resources that you want to manage in your Azure subscription with Azure Automation.
This credential asset can be created with ARM PowerShell cmdlets as well. You will have to enter the service account credentials when prompted while executing the following PowerShell code:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is CreateAutomationCredential.ps1.
Alternatively, credentials can be created with ARM template as well:
Download the Code
You can download the full template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is CreateAutomationCredential.json.
We mentioned earlier that graphical PowerShell Workflow runbooks can only be authored in the Azure portal. As shown in Figure 16, you can see what the graphical PowerShell Workflow runbook looks like when viewed in the Azure portal. You can also have a look at the activities contained in this runbook.
FIGURE 16. INSTALL-OMSEXTENSIONV2VMGRAPHICAL GRAPHICAL RUNBOOK
To create our first graphical PowerShell Workflow runbook, you will need to import it, using the steps shown here.
Download the Code
You can download the graphical runbook from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\runbooks directory.
The file name is Install-OMSExtensionV2VMGraphical.graphrunbook.
Runbook file: Click on the folder icon and select the Install-OMSExtensionV2VMGraphical.graphrunbook file located on your local computer.
Runbook Type: Graphical PowerShell Workflow Name: Install-OMSExtensionV2VMGraphical
FIGURE 17. IMPORT RUNBOOK BLADE – GRAPHICAL POWERSHELL WORKFLOW
VMName: <the name of your Azure VM>
VMResourceGroup: <the resource group name where your Azure VM is located>
VMLocation: <the Azure region where your VM is located>
Note: The Azure VM should be version 2 (ARM VM) created in the Azure portal. VM Extension can be installed only if the VM is in running state. The account that was entered as a credential asset in Figure 15 should have contributor or owner Azure rights on that VM.
FIGURE 18. START RUNBOOK BLADE
FIGURE 19. JOB OUTPUT BLADE
FIGURE 20. VM EXTENSIONS BLADE
Importing runbooks and starting jobs can be done with AzureRM PowerShell as well:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is ImportRunbookGraphPSWAndStartJob.ps1.
The third alternative is to use ARM Template to deploy a runbook and start a job:
Download the Code
You can download the full template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is ImportRunbookGraphPSWAndStartJob.json.
If you have previously run the graphical runbook and you want to use the same VM for the next exercises, you will have to remove the OMS Extension. That can be done via Azure portal.
We mentioned earlier that graphical PowerShell runbooks can only be authored in the Azure portal. In Figure 21, you can see what the graphical PowerShell runbook looks like when viewed in the Azure portal. As you can see, we have the same set of activities compared to Graphical PowerShell Workflow runbook in the previous section. Basically, the difference is how these runbooks are executed at the backend. You can also have a look at the activities contained in this runbook.
FIGURE 21. INSTALL-OMSEXTENSIONV2VMGRAPHICALPS GRAPHICAL RUNBOOK
To create our first graphical PowerShell runbook, you will need to import it, using the steps shown here.
Download the Code
You can download the graphical runbook from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\runbooks directory.
The file name is Install-OMSExtensionV2VMGraphicalPS.graphrunbook.
Runbook file: Click on the folder icon and select the Install-
OMSExtensionV2VMGraphicalPS.graphrunbook file located on your local computer.
Runbook Type: Graphical
Name: Install-OMSExtensionV2VMGraphicalPS
FIGURE 22. IMPORT RUNBOOK BLADE – GRAPHICAL POWERSHELL
VMName: <the name of your Azure VM>
VMResourceGroup: <the resource group name where your Azure VM is located>
VMLocation: <the Azure region where your VM is located>
FIGURE 23. START RUNBOOK BLADE
FIGURE 24. JOB OUTPUT BLADE
Importing runbooks and starting jobs can be done with AzureRM PowerShell as well:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is ImportRunbookGraphPSAndStartJob.ps1.
The third alternative is to use ARM Template to deploy a runbook and start a job:
Download the Code You can download the full template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is ImportRunbookGraphPSAndStartJob.json.
If you have previously run the graphical runbook and you want to use the same VM for the next exercises, you will have to remove the OMS Extension. That can be done via Azure portal.
While Graphical runbooks do not allow you to view the code, Workflow runbooks are completely text based.
Download the Code
You can download the PowerShell Workflow Runbook from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\runbooks directory.
The file name is Install-OMSExtensionV2VMWorkflow.ps1.
Follow these steps to create a PowerShell Workflow runbook in the Azure portal.
Runbook file: Click on the folder icon and select the Install-
OMSExtensionV2VMWorkflow.ps1 file located on your local computer.
Runbook Type: PowerShell Workflow
Name: Install-OMSExtensionV2VMWorkflow
FIGURE 25. IMPORT RUNBOOK BLADE – POWERSHELL WORKFLOW
VMName: <the name of your Azure VM>
VMResourceGroup: <the resource group name where your Azure VM is located>
VMLocation: <the Azure region where your VM is located>
FIGURE 26. START RUNBOOK BLADE
FIGURE 27. JOB OUTPUT BLADE
Importing runbooks and starting jobs can be done with AzureRM PowerShell as well:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is ImportRunbookPSWAndStartJob.ps1.
The third alternative is to use ARM Template to deploy a runbook and start a job:
Download the Code
You can download the full template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is ImportRunbookPSWAndStartJob.json.
If you have previously run the PowerShell Workflow runbook and you want to use the same VM for the next exercises, you will have to remove the OMS Extension. That can be done via Azure portal.
PowerShell runbooks are also text based like PowerShell Workflows. You will notice a lot of similarities between them.
Download the Code
You can download the PowerShell Runbook from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\runbooks directory.
The file name is Install-OMSExtensionV2VMScript.ps1.
Follow these steps to create a PowerShell Runbook in the Azure portal.
Runbook file: Click on the folder icon and select the Install-
OMSExtensionV2VMScript.ps1 file located on your local computer.
Runbook Type: PowerShell
Name: Install-OMSExtensionV2VMScript
FIGURE 28. IMPORT RUNBOOK BLADE – POWERSHELL
VMName: <the name of your Azure VM>
VMResourceGroup: <the resource group name where your Azure VM is located>
VMLocation: <the Azure region where your VM is located>
FIGURE 29. START RUNBOOK BLADE
FIGURE 30. JOB OUTPUT BLADE
Importing runbooks and starting jobs can be done with AzureRM PowerShell as well:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is ImportRunbookPSAndStartJob.ps1.
The third alternative is to use ARM Template to deploy a runbook and start a job:
Download the Code
You can download the full template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is ImportRunbookPSAndStartJob.json.
By default, Azure Automation runbooks are executed on worker nodes located in Azure that are shared between different automation accounts, although jobs are completely isolated from one another in a secure manner. Since jobs run on Azure worker nodes, it is challenging to access resources that are located on-premises. Fortunately, Azure Automation has a solution introduced with OMS called the Hybrid worker. Simply put, you can connect servers located in your own networks to your Azure Automation Account as Hybrid Workers. When you execute runbooks, you can specify that you want the job to be executed on Hybrid Worker servers instead of in Azure. The runbooks are stored and managed in Azure Automation, but delivered to one or more Hybrid Worker servers and executed there. The server designated as a Hybrid worker does not require an inbound connectivity to Internet, only outbound one. This is because the Hybrid workers initiate all communication to Azure Automation in the cloud. When a runbook is started from Azure portal and assigned to a Hybrid Worker Group, One of the Hybrid Worker server within the group retrieves instruction for the runbook. The server pulls down the runbook and any assets from Azure Automation before it runs the job.
Note: Currently modules and certificates are not distributed to Hybrid Workers automatically. To use them on Hybrid Workers you should import/install them manually on the servers that are acting as Hybrid Workers.
When a Hybrid Worker is being set up, it is added to a Hybrid Runbook Worker Group. In a Hybrid Runbook Worker Group, you can add one or multiple servers. When you start a runbook, you designate on which Hybrid Runbook Worker Group should be running. By having a Hybrid Runbook Worker Group with more than one node you achieve highavailability because runbooks will start only on servers that are running and healthy.
Important considerations regarding Hybrid Worker and Update Management
Here are a few important considerations regarding the Hybrid Worker and the Update Management solution.
After Hybrid Worker is setup it will communicate to specific URL depending on the region of your Automation Account. Table 2 lists the URLs for the different Azure Automation Account regions:
Region | URL |
South Central US | scus-jobruntimedata-prod-su1.azure-automation.net |
East US 2 | eus2-jobruntimedata-prod-su1.azure-automation.net |
West Europe | we-jobruntimedata-prod-su1.azure-automation.net |
North Europe | ne-jobruntimedata-prod-su1.azure-automation.net |
Canada Central | cc-jobruntimedata-prod-su1.azure-automation.net |
South East Asia | sea-jobruntimedata-prod-su1.azure-automation.net |
Central India | cid-jobruntimedata-prod-su1.azure-automation.net |
Japan East | jpe-jobruntimedata-prod-su1.azure-automation.net |
Australia South East | ase-jobruntimedata-prod-su1.azure-automation.net |
South Central US | scus-jobruntimedata-prod-su1.azure-automation.net |
TABLE 9. AZURE REGIONS AND HYBRID WORKER REPORTING ADRESSES
Outbound Internet access on TCP Port 443 should be allowed to those URLs for proper Hybrid Worker functioning. Hybrid Worker servers support proxy configuration applied to the system. You can use netsh winhttp commands to configure system proxy on the Hybrid Worker Server.
Note: Hybrid Workers currently are configured on Windows based operating systems but you can reach and manage any Linux OS by using PowerShell modules for SSH. In the future, we might be able to configure Hybrid Workers on Linux OS as well.
Hybrid Workers require Microsoft Monitoring Agent with a connection to OMS workspace for setup. Automation Hybrid Worker solution also needs to be added to your Log Analytics Workspace. OMS is only used to install and maintain the management agent and to monitor the functionality of the worker. Delivery of runbooks and instructions to run them is handled by Azure Automation. The server which will be designated for Hybrid Worker must be Windows Server 2012 or higher and must have Windows PowerShell 4.0 or higher. If you have on-premises components like System Center Orchestrator or Service Management Automation, the Hybrid Worker can coexist with them on the same servers. It is recommended to use Automation Account located in an Azure region that is close to the physical location of the machines to which you will setup Hybrid Workers.
For the next steps, we assume you have setup the following prerequisites:
Follow these steps to setup Hybrid Worker on an on-premises server.
FIGURE 31. AZURE AUTOMATION ACCOUNT KEYS
Files\Microsoft Monitoring
Agent\Agent\AzureAutomation\<version>\HybridRegistration" as shown in Figure 32.
Name: <this is the name of the Hybrid Runbook Worker Group>
URL: <this is the URL value from step 3>
Key: <this is Primary Access Key from step 3>
Note: If you provide Name for Hybrid Runbook Worker Group that does not exist in your Automation Account, that group will be created. If a Hybrid Runbook Worker Group with such name exists, the Hybrid Worker would be added to the other Hybrid Workers in that group.
FIGURE 32. REGISTER HYBRID WORKER
FIGURE 33. HYBRID WORKERS BLADE
The runbook jobs on a Hybrid Worker server by default run with Local System account, but you can configure to be run with local or domain accounts. To take advantage of this option first create a credential asset with local or domain account credentials. If the account is domain one the user name format can be either domain\username or username@domain if it is local account enter only the username. Once the credential asset is created navigate to Azure portal, <your Automation account>, Shared Resources, Hybrid Worker Groups, <Hybrid Worker group name>, Hybrid Worker Group Settings as shown in Figure 31. From Hybrid Worker group settings, switch from default to custom for run as and choose the credential asset created earlier. Once you have settled on a configuration click Save.
FIGURE 34. HYBRID WORKER GROUP RUN AS ACCOUNT
We have looked at the scenarios of installing Microsoft Monitoring Agent and connecting it to OMS workspace on Azure VM through VM extension. For executing a runbook on a Hybrid Worker, we will go through a scenario where the runbook connects to another onpremises server that is part of the same Active Directory domain where the Hybrid Worker resides, downloads the MMA installation from Internet, installs MMA and connects it to OMS workspace.
Download the Code
You can download the PowerShell Workflow Runbook from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\runbooks directory.
The file name is Install-MMA.ps1.
Follow these steps to execute runbook on Hybrid Worker.
FIGURE 35. NEW CREDETNAIL BLADE
Name: DomainCreds
User Name: <your service account user name>
Password: <your service account password>
Confirm Password: <your service account password>
Note: The credentials entered here needs to be domain credentials for the domain to which the Hybrid Worker belongs and also of the server on which we will install Microsoft Monitoring Agent. The credentials also need to have administrator rights on the server where MMA agent will be installed.
Runbook file: Click on the folder icon and select the Install-MMA.ps1 file located on your local computer.
Runbook Type: PowerShell Workflow
Name: Install-MMA
FIGURE 36. IMPORT RUNBOOK BLADE
ServerName: <the NetBIOS or FQDN of your on-premises server where MMA will be installed>
Run on: Hybrid Worker
Choose Hybrid Worker Group: <your Hybrid Worker Group>
FIGURE 37. START RUNBOOK ON HYBRID WORKER
FIGURE 38. JOB OUTPUT BLADE
"C:\Program Files\Microsoft Monitoring
Agent\Agent\AgentControlPanel.exe" if your Hybrid Worker computer is using Server Core), then go to the Azure Log Analytics (OMS) tab and you will see a green checkmark for a successful connection to the Log Analytics workspace, as shown in Figure 39.
FIGURE 39. MICROSOFT MONITORING AGENT PROPERTIES
Importing runbooks and starting jobs on Hybrid Worker Group can be done with AzureRM PowerShell as well:
Download the Code
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is ImportRunbookHWAndStartJob.ps1.
The third alternative is to use ARM Template to deploy a runbook and start a job on Hybrid Worker Group:
Download the Code
You can download the full template from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\templates directory.
The file name is ImportRunbookHWAndStartJob.json.
When you author runbooks, you usually go through a number of changes before reaching to a runbook version that gives you the desired results. However, even when you reach that version, you often introduce small improvements or new features to your runbook. You may introduce many changes throughout the lifecycle of a runbook. When a change to a runbook is made, it needs to be documented. Documentation provides the reason why the change was introduced and also describes the state of the runbook before and after the change. It also provides information on when the change was made and by whom. To ease this process, you can use a source control system. The term for change in a source control system is called a revision. Source control systems track those revisions automatically. Revisions can be compared, restored, and with some types of files, merged. Runbooks should always be treated as a code and because of that, you should always use the source control system when you author them.
Currently, Azure Automation allows integration with a popular source control system called GitHub. Integration with another system called Visual Studio Online is planned as well but is not yet available.
IMPORTANT: Source Control Integration in Azure Automation currently works only for PowerShell Workflow and PowerShell Runbooks.
Follow these steps to create source control integration with GitHub and test how it works.
FIGURE 40. GITHUB HOMEPAGE
Repository Name: OMSRunbooks
Type: Public
Initialize this repository with a Readme: Check
Add .gitignore: None
FIGURE 41. CREATE REPOSITORY PAGE
FIGURE 42. REPOSITORY PAGE
FIGURE 43. SELECT REPOSITORY LOCATION WINDOW
FIGURE 44. REPOSITORY LOCAL LOCATION
FIGURE 45. GITHUB COMMIT
FIGURE 46. GITHUB REPOSITORY SYNC
FIGURE 47. REPOSITORY WEBPAGE
Source Control.
FIGURE 48. CHOOSE SOURCE BLADE
FIGURE 49. AUTHORIZATION BLADE
FIGURE 50. RUNBOOK FOLDER PATH BLADE
FIGURE 51. SET UP SOURCE CONTROL BLADE
FIGURE 52. REPOSITORY SYNCHRONIZATION BLADE
FIGURE 53. SYNC JOB OUTPUT BLADE
workflow Test-Runbook {
#Test Source Control
}
Commit new file: Test Source Control
FIGURE 54. GITHUB CREATE FILE PAGE
FIGURE 55. SYNC JOB OUTPUT BLADE
The same runbook that was created in step 22 should be visible, as shown in
Figure 56. Runbooks that are synced for the first time shown with a status of "New". Runbooks that previously existed in the Automation account, but modified on GitHub when synced are shown with a status of "In Edit".
FIGURE 56. VIEWING SYNCED RUNBOOK IN AZURE PORAL 27. Modify Test-Runbook as shown in Figure 57.
FIGURE 57. EDITING RUNBOO
FIGURE 58. CHECK IN DIALOG
FIGURE 59. INPUT PARAMETERS BLADE
FIGURE 60. AZURE AUTOMATION GITHUB COMMIT
A webhook gives you the ability to start a runbook with a simple web request. This allows other services or applications to programmatically start runbooks without using the Azure Resource Manager REST API. For example, such services are Visual Studio Team Services, GitHub, Azure alerts and OMS. When a webhook is created, a few properties need to be specified:
If a runbook has mandatory input parameters, we must give values to those parameters upon webhook creation. If there are input parameters that are not mandatory, we can assign values but we are not obligated to do it. Multiple webhooks linked to a single runbook can each use different parameter values. When a service starts, a runbook using a webhook, it cannot override the parameter values defined in the webhook. In order to send data into the runbook, you must create input parameter called $WebhookData of type [object]. This object will contain the data that is included in the web request to the webhook. The $WebhookData object has three properties:
When the value for the $WebhookData input object is specified during webhook creation, that value will be overridden when the webhook is called from another service.
As the URL of the webhook is sensitive data and can be compromised, it is best practice to include additional logic in your runbook on the data passed to $WebhookData. Such logic could be to check if specific data is passed to the runbook and that data can only come from the service for which the webhook was intended to.
In the next sections, we will import a runbook similar to the runbook from the previous section, but modified to work with webhooks. After the runbook is imported, we will create a webhook for it. The last step will be to execute the runbook through a webhook by simply using PowerShell.
The first step in this example is to import the runbook and create a webhook for it.
Download the Code
You can download the PowerShell Workflow Runbook from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\runbooks directory.
The file name is Install-OMSExtensionV2VMWorkflowWebhook.ps1.
If you have previously run one of the runbooks to install the OMS extension on the Azure VM, you will have to remove the OMS Extension. This can be done via Azure portal.
Follow these steps to import the "Install-OMSExtensionV2VMWorkflowWebhook" runbook and create webhook for it in the Azure portal.
Runbook file: Click on the folder icon and select the Install-
OMSExtensionV2VMWorkflowWebhook.ps1 file located on your local computer.
Runbook Type: PowerShell Workflow
Name: Install-OMSExtensionV2VMWorkflowWebhook
FIGURE 61. IMPORT RUNBOOK BLADE – POWERSHELL WORKFLOW
Name: PowerShellWebhook
Enabled: Yes
Expires: <leave default>
URL: <copy the URL to secure location>
FIGURE 62. CREATE A NEW WEBHOOK BLADE
FIGURE 63. WBHOOK PARAMETERS BLADE
In this section, we will execute runbook via Webhook using PowerShell.
Download the Code
You can download the PowerShell script from GitHub at https://github.com/insidemscloud/OMSBookV2 in the \Chapter 3\scripts directory. The file name is ExecuteWebhook.ps1.
Follow these steps to execute runbook through a webhook:
Note: Some variables like $WebhookURI may need to be modified to work in your environment.
FIGURE 64. WEBHOOK EXECUTION FROM POWERSHELL
FIGURE 65. JOB INPUT BLADE
Automation is a key component of modern IT management. Automating repeating tasks through Process Automation reduces IT Operations costs. Mastering this capability delivers business value to your organization by reducing IT Operations costs and competitive advantage over other organizations.
In this chapter, we have discussed Azure Automation and one of its principal capabilities - process automation. We have learned through examples how we can use process automation to achieve real-world tasks. We hope these examples will help you when need to automate tasks in your environment and also encourage you to explore more in Azure Automation.
In the next chapter, we will have a look at the other principal capability of Azure Automation –configuration management.