Operations Management Suite (OMS): Azure & Office 365 Solutions


In recent months, Microsoft has released a number of solutions enabling visibility into Microsoft Azure-based products and services, including:

  • Office 365 Analytics (Preview)
  • Activity Log Analytics
  • Azure SQL Analytics (Preview)
  • Azure Web Apps Analytics (Preview)
  • Application Insights Connector (Preview)
  • Azure Application Gateway Analytics
  • Azure Network Security Group Analytics
  • HDInsight HBase Monitoring (Preview)
  • Key Vault Analytics
  • Logic Apps B2B (Preview)
  • Service Fabric Analytics

What almost all of these solutions have in common is that they are based on ingestion of Azure Diagnostic Log data in OMS. In this chapter, we will touch briefly on each of these solutions, reviewing their features and functions. We will also look at how to enable diagnostic logging in greater depth, including how to enable logging on a more granular basis via PowerShell. Finally, we will touch on troubleshooting problems enabling diagnostic logging.

Azure Diagnostic Log Service & Metrics

Azure Diagnostic Logs are logs emitted by a resource that provides verbose data about the operation of that resource. The content of these logs varies by resource type (for example, Windows Event System Logs are one category of diagnostic log for VMs, whereas blob, table, and queue logs are categories of diagnostic logs for storage accounts) and differ from the Activity Log (formerly known as Audit Log or Operational Log), which provides insight into the operations that were performed on resources in your subscription. However, not all resources support the new type of Diagnostic Logs.

Note: A complete list of supported services and schema for Diagnostic Logs is available in "Collect and consume diagnostic data from your Azure resources" at https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs

What you can do with Diagnostic Logs?

Azure offers a number of processing options Diagnostic Logs (shown in Figure 1), including:

  • Saving them to a Storage Account for auditing or manual inspection. You can specify the retention time (in days) using the Diagnostic Settings.
  • Streaming them to Event Hubs for ingestion by a third-party service or custom analytics solution such as Power BI.
  • Analyzing them with OMS Log Analytics.


The storage account or event hub namespace does not have to be in the same subscription as the resource emitting logs, as long as the user who configures the setting has appropriate role-based access to both subscriptions.

Diagnostic Settings

Diagnostic Logs for non-compute resources are configured using Diagnostic Settings.

Diagnostic Settings for a resource control the following:

  • Where Diagnostic Logs are sent (Storage Account, Event Hubs, and/or OMS Log Analytics).
  • Which log categories are sent.
  • How long each log category should be retained in a Storage Account, with a retention of zero days meaning that logs are kept forever.

How to enable collection of Diagnostic Logs

You can enable diagnostic logs in one of five ways, though not all log enablement options are possible across all five methods (shown in Figure 2):

  • In the Azure Portal
  • Via the REST API
  • Via the CLI
  • Via PowerShell
  • Via Resource Manager Templates


While it does not make sense to duplicate the step-by-step instructions for all five methods, it is worth mentioning that some OMS solutions involve resource providers that cannot be configured through the Azure portal UI. Specifically, the SQL Server Analytics and Web App Service solutions tap into resource providers that fall into this category. For these resource types, and indeed all OMS-supported types, Microsoft provides a PowerShell script to make enabling diagnostic logging an easier task.

Enable Diagnostic Logging via PowerShell

Now, we will look at the steps for enabling diagnostic logging leveraging the script provided by Microsoft.

Note: These steps assume that you have the latest version of Azure Resource Manager PowerShell installed (4.2 at the time this was written). You can install via the Web Platform Installer at https://www.microsoft.com/web/downloads/platform.aspxor via PowerShell Gallery at https://www.powershellgallery.com/packages/AzureRM.

  1. You will begin by saving a copy of Microsoft's script provided for enabling Azure Diagnostic Logs by running the following one-liner at an elevated PowerShell prompt or PowerShell ISE window. You can choose the path.

save-script -Name Enable-AzureRMDiagnostics -Path "C:\temp"

  1. Next, change your current directory to the path you provided to the -Path parameter in the previous step.

Push-Location C:\temp

  1. Run the following one-liner to run the script, replacing the values for -SubscriptionID and -WSID with the values for your Azure subscription ID and OMS workspace ID, respectively.

.\Enable-AzureRMDiagnostics.ps1 ` -SubscriptionId "0b62f50c1-c15a-40e2-xxxx-xxxxxxxxxxxx" ` -WSID "47837ac4-89f5-476b-xxxx-xxxxxxxxxxxx"

The script will prompt you to choose the resource type for which you want to enable diagnostic logging, as shown in Figure 3. Just choose a number and script will enable diagnostic logging for the resource type you selected.


Note: The script enables logging for all resources of the selected type. It is possible to enable logging for resources of the specified type for a specific resource group by adding the -ResourceGroup parameter and specifying the resource group name.

Enable Diagnostic Logging via ARM

You can also enable Azure Diagnostic Logs via Azure Resource Manager template. A sample template is shown below.

Download the Code

You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2, in the \Chapter 9\scripts directory. The file name is DiagArm.json.

Enable Diagnostic Logging via Azure Portal

Finally, you can enable Diagnostic Logs via the Azure portal. A sample template is shown here. The Activity Log is available in Azure for free without being an OMS customer, but if you are also an OMS customer, the Activity Log will show up as part of OMS Log Analytics.

  1. In the Azure portal on the More services, menu, select Diagnostic logs.
  2. Select the resource for which you would like to send logs to OMS.
  3. Under Status, select On.
  4. Select the checkbox next to Send to Log Analytics.
  5. On the menu provided, select the OMS workspace where you want to send the logs.

Within a few hours, log data for the solution you are working with should be available in the OMS portal.

For more information on collecting Azure service logs in OMS Log Analytics, see "Collect Azure service logs and metrics for use in Log Analytics" on the Microsoft website at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-storage.

Office 365 Analytics (Preview)

With the Office 365 solution, you can perform the following types of management activities:

  • Monitor user activities on your Office 365 accounts to analyze usage patterns as well as identify behavioral trends. For example, you can extract specific usage scenarios, such as files that are shared outside your organization or the most popular SharePoint sites.
  • Monitor admin activities to track configuration changes or high privilege operations.
  • Detect and investigate unwanted user behavior, which can be customized for your organizational needs.
  • Demonstrate audit and compliance. For example, you can monitor file access operations on confidential files, which can help you with the audit and compliance process.
  • Perform operational troubleshooting by using OMS Search on top of Office 365 activity data of your organization.


Data that is collected for Office 365 is all based on the current functionality of the Office 365 Management Activity API. Today, this API includes management activities for Exchange, SharePoint, and Azure Active Directory. This information is presented in the dashboard behind the Office 365 tile, shown in Figure 4.

STEP BY STEP: The detailed configuration instruction for OMS Office 365 solution can be at Microsoft Azure Documentation site: https://docs.microsoft.com/en-us/azure/operations-management-suite/oms-solution-office-365


The Office 365 solution offers four views, as shown in Figure 5:

  • The OPERATIONS section provides information about the active users from your all monitored Office 365 subscriptions. You will also be able to see the number of activities that happen over time.
  • The EXCHANGE section shows the breakdown of Exchange Server activities such as Add-Mailbox Permission, or Set-Mailbox.
  • The SHAREPOINT section shows the top activities that users perform on SharePoint documents. When you drill down from this tile, the search page shows the details of these activities, such as the target document and the location of this activity. For example, for a File Accessed event, you will be able to see the document that's being accessed, its associated account name, and IP address.
  • The AZURE ACTIVE DIRECTORY section includes top user activities, such as Reset User Password and Login Attempts. When you drill down, you will be able to see the details of these activities like the Result Status. This is most helpful if you want to monitor suspicious activities in your Azure Active Directory.

Alerting and customization with Office 365 solution

Office 365 solution in OMS will enable you to search Office 365 user activities in the OMS portal. Although you can utilize search capabilities provided by OMS, you have detailed information for your Office 365 activities with more than 50 fields for different Office workloads. You can find some use cases in the Example Queries section.

With custom search queries that are fine-tuned for your organization's needs, you can create alerts on these queries that will show up in the Alert Management solution on your Overview page. This will help you to monitor Office 365 alerts along with your other alerts in OMS using the Add Alert Rule interface, shown in Figure 6.


FAQs and Tips

Below are a few frequently asked questions and tips to ensure you are able to successfully implement and get the most value from the Office 365 Analytics solution.

  • You must be a global admin of the Office 365 account to be able to connect the account to OMS.
  • You can connect only your organizational Office 365 accounts to OMS. If the Office 365 admin account that you're planning to use is a Microsoft account
  • (email addresses that end in @Hotmail.com, @msn.com, @outlook.com, @live.com), you will not be able to complete the onboarding for this solution. As a workaround, get an Office 365 admin account that is also an organizational account, for example, Don.Funk@contoso.onmicrosoft.com).
  • Multiple office subscriptions can be connected to one OMS workspace. However, an Office 365 subscription can only be connected to one OMS workspace.

Activity Log Analytics

The Azure Activity Log Analytics solution is intended to provide insights into activities across multiple Azure subscriptions in a single view. The primary difference between this solution and all the others mentioned in this chapter is that this solution is based on Activity Logs, rather than Diagnostic Logs. The primary value of this solution (in the author's opinion) is that it provides an easy way to search for specific types of activities, or specific users across multiple Azure subscriptions to identify trends that would otherwise be difficult and laborious to gather one log at a time. Read and write activities across a wide variety of resource providers are covered by this solution.

What's the difference between Azure Activity and Diagnostic logs?

The Azure Activity Log is a log that provides insight into the operations that were performed on resources in your subscription. The Activity Log was previously known as "Audit Logs" or "Operational Logs," since it reports control-plane events for your subscriptions. Using the Activity Log, you can determine the 'what, who, and when' for any write operations (PUT, POST, DELETE) taken on the resources in your subscription.

You can also understand the status of the operation and other relevant properties.

Activity Logs provide data about the operations on a resource from the outside. Diagnostics Logs are emitted by a resource and provide information about the operation of that resource.

The Azure Activity Logs tile simply displays a count of Activity Log records in OMS, as shown in Figure 6.


When you click on the tile, the Activity Logs dashboard presents more detailed info, including log entry counts by date and event status, as well as counts by resource and resource provider, as shown in Figure 7.


You can then click on any of the views in the dashboard, to drill down to the query detail, and to events belonging to an individual resource or resource provider, as shown in Figure 8.


Using the log search, you can correlate data from this solution across other solutions, including user activities, authorization changes, service health and outages from logs across your subscriptions.

  • Activity logs by status
  • Resources with activity logs
  • Resource providers producing activity logs
  • Azure activity log entries

STEP-BY-STEP: Step-by-step instructions for configuring the Activity Log Analytics solution are available in "View Azure Activity logs" at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-activity

Azure SQL Analytics (Preview)

The Azure SQL Monitoring solution in OMS collects and visualizes important SQL Azure performance metrics. By using the metrics that you collect with the solution, you can create custom monitoring rules and alerts. You can also monitor and visualize Azure SQL Database and elastic pool metrics across multiple Azure subscriptions and elastic pools. The solution also helps you to identify issues at each layer of your application stack. The solution aggregates Azure diagnostic metrics together with Log Analytics views to present data from all your Azure SQL databases and elastic pools in a single Log Analytics workspace.

The Azure SQL Analytics solution does not use agents to connect to the Log Analytics service. As a result, the connected source that is supported by this solution is an Azure storage account. Azure metric data is sent to Log Analytics using the storage account. However, when you enable diagnostic logs to be forwarded to OMS, the data is not sent to Azure storage outside of OMS, avoiding duplication.

Because enabling diagnostic logging for Azure SQL is not possible in the Azure portal UI, Microsoft provides the sample script mentioned earlier in this chapter in the section titled "Enable Diagnostic Logging via PowerShell". To enable Azure SQL diagnostic logs to be forwarded to OMS Log Analytics, follow the steps detailed in that section and select the 'Microsoft.sql/servers/databases' resource provider.

While this solution supports data visualization from multiple Azure subscriptions, you have to use PowerShell to configure collection across multi-subscriptions. You can enable collection for an entire subscription or a single resource group.

Once you have enabled Diagnostic Log forwarding to Log Analytics and events are processed, the dashboard for this solution will be populated. The Azure SQL Analytics tile will reflect the number of Azure SQL databases and elastic pools for which logs have been forwarded, as shown in Figure 9.


The dashboard behind this tile presents more detailed data on the Azure subscriptions, Azure SQL servers, databases, and elastic pools, as shown in Figure 10.


As with all OMS solutions, the data surfaced through OMS Log Analytics can be leveraged to create e-mail alerts or trigger other automated response.

Azure Web Apps Analytics (Preview)

The Azure Web Apps Analytics solution provides insights into your Web Apps by collecting different metrics across all your Azure Web App resources. This solution provides visibility into the following metrics:

  • Top Web Apps with highest response time
  • Number of requests including successful and failed requests across your Web Apps
  • Top Web Apps with highest incoming/outgoing traffic
  • Top Service Plans with high CPU/memory utilization

As with the Azure SQL solution, enabling forwarding of diagnostic logs for Azure Web Apps to OMS Log Analytics is not possible in the Azure portal UI. Microsoft provides the sample script mentioned earlier in this chapter in the section titled "Enable Diagnostic Logging via PowerShell". To enable Azure Web Apps diagnostic logs to be forwarded to

OMS Log Analytics, follow the steps detailed in that section and select the 'Microsoft.web/sites' and 'Microsoft.web/ServerFarms' resource providers.

The Azure Web Analytics dashboard is shown in Figure 11.


Application Insights Connector (Preview)

Application Insights, which is an analytics service that's designed for developers, monitors live web applications. It enables you to detect and diagnose performance issues and understand what users do with your applications. Integrating your Application Insights apps to OMS increases your organization's visibility over your applications by having operation and application data in one place.

With the Application Insights Connector in OMS, you can:

  • View all your Application Insights apps in a single OMS instance, even if they are in different subscriptions.
  • Correlate infrastructure data with application data
  • Visualize application data with perspectives in OMS search
  • Pivot from the Log Analytics data to your Application Insights app in the Azure portal

To use the Application Insights Connector, you have to link your Application Insights apps to your OMS Log Analytics workspace. To do this in the Log Analytics portal:

  1. In the OMS portal, select the App Insights Connector solution and click the Add button.
  2. To configure data collection, go to the Settings page and click Data, then the Application Insights tab.
  3. Select your Azure subscription and Application Insights apps by checking the boxes, as shown in Figure 12.


  1. Check the boxes by the Application Insights instances you wish to import into OMS, then click Save.

STEP-BY-STEP: For step-by-step configuration guidance and screenshots, see "Improving Developer / IT Collaboration with

Application Insights Connector for OMS Log Analytics" on the Microsoft website at https://blogs.technet.microsoft.com/msoms/2016/09/26/application-insights-connector-in-oms/.

Once you have completed the configuration steps, the Application Insights tile will reflect the number of entries logged via the Application Insights Connector, as shown in Figure 13.


When you click on the tile, you will see relevant details of your application, such as the requests per hour of this Azure App Service application, as shown in Figure 14.


If you click on the chart or entries in the previous figure, you will get to the query used to populate the chart, as shown in Figure 15.


The Application Insights Connector solution enables deep monitoring for your custom applications, without the need to develop a monitoring plugin or solution for every application you build.

Azure Application Gateway Analytics

Microsoft Azure Application Gateway provides Application Delivery Controller (ADC) as a service, offering various layer 7 load balancing capabilities for your application. It allows customers to optimize web farm productivity by offloading CPU intensive SSL termination to the Application Gateway. It also provides several other Layer 7 routing capabilities.

Application Gateway also has a web application firewall (WAF).

Additional Reading: You can learn more about Azure Application Gateway at https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-introduction.

The Azure Application Gateway Analytics solution is intended to help you more easily troubleshoot application issues by providing visibility into application gateway logs. Key scenarios supported by the solution out-of-box include:

  • Client and server errors reported by your application gateway
  • Requests per hour per application gateway
  • Failed requests per hour per application gateway
  • Client and server errors by user agent
  • Count of healthy and unhealthy hosts per application gateway
  • Failed requests per application gateway

STEP-BY-STEP: For step-by-step installation and configuration guidance, see "Azure networking monitoring solutions in Log Analytics" on the Microsoft website at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-networking-analytics.

Azure Network Security Group Analytics

The Azure Network Security Group (NSG) Analytics solution helps you more easily troubleshoot issues across your Azure networks by providing visibility into NSG rules and logs. The solution enables visibility into the following scenarios out-of-box:

  • Top network security rules that blocked the most number of flows within a specified time frame
  • Top network security rules that allowed the most number of flows within a specified time frame
  • Top MAC addresses with the most number of blocked flows within a specified time frame
  • Top MAC addresses with the most number of allowed flows within a specified time frame

Diagnostic logging must be enabled for each NSG you want to collect data for. The section titled "What can you do with Diagnostic Logs?", earlier in this document, explains where diagnostic logs can be sent. The Network Security Group Analytics tile in the OMS portal is shown in Figure 16.


When you click on the NSG Analytics tile, shown in Figure 16, you will then see details on both allowed and blocked NSG flows on the Azure Network Security Group Analytics dashboard, as shown in Figures 17 and 18.


Notice that each includes the subnet and physical addresses of machines with blocked flows, enabling drill down to the subnet and individual host.


HDInsight HBase Monitoring (Preview)

The HDInsight HBase solution for OMS provides Log Analytics, monitoring and alerting capabilities for HDInsight HBase. With this solution, you can enable a number of monitoring capabilities, including:

  • Monitoring multiple clusters across multiple Azure subscriptions
  • Troubleshooting HBase issues faster by gaining access to common logs, as well as various HBase metrics in a single console
  • Setting alerts on thresholds that indicate performance or availability issues


In order to deploy and configure the solution, a couple of items are assumed to be true:

  • You have an active HDInsight HBase Linux cluster
  • You have deployed Spark, HBase, Interactive Hive, or Hive

As with other OMS solutions, you can add the HDInsight HBase Monitoring solution to your OMS subscription by browsing to it in the Solution Gallery and clicking the Add button. To configure the solution, you run a shell script (bash) and specific script, based on the components you have installed. There are separate scripts for Spark, HBase, Interactive Hive, and Hive.

STEP-BY-STEP: You can find step-by-step installation instructions, as well as a few sample Log Analytics queries, at https://github.com/hdinsight/HDInsightOMS

Key Vault Analytics

You can use the Azure Key Vault solution in Log Analytics to review Azure Key Vault Audit / Event logs. To use the solution, you need to enable logging of Azure Key Vault diagnostics and direct the diagnostics to a Log Analytics workspace. It is not necessary to write the logs to Azure Blob storage.

Azure Key Vault solution collects diagnostics logs directly from the Key Vault. It is not necessary to write the logs to Azure Blob storage and no agent is required for data collection. The Key Vault Analytics tile reflects recent levels of successful and failed requests to Key Vaults, as shown in Figure 19.


After you click on the tile, you can view summaries of your logs and then drill into details for several categories, including:

  • Volume of all key vault operations over time
  • Failed operation volumes over time
  • Average operational latency by operation
  • Quality of service for operations, with the number of operations that take more than 1000 milliseconds
  • List of operations that take more than 1000 milliseconds

All the items in the list above are shown in Figure 20 below.


You can then click on the individual items on the Key Vault Analytics dashboard to see the query itself, as well as query results.

STEP-BY-STEP: For step-by-step installation and configuration guidance, see "Key Vault Analytics solutions in Log Analytics" on the Microsoft website at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-key-vault

Logic Apps B2B

In B2B scenarios, logic apps exchange messages with other organizations using industry standard protocols, such as AS2, X12, and EDIFACT. You can also secure messages with both encryption and digital signatures.

By sending Diagnostic Logs for Logic Apps, you can access the following logs in your OMS workspace :

  • Failed AS2 Messages by Receive Partner
  • Failed AS2 Messages by Send Partner
  • Failed AS2 Messages by Workflow
  • Failed X12 Messages by Receive Partner
  • Failed X12 Messages by Send Partner
  • Failed X12 Messages by Workflow

There is an excellent deep dive into this solution in "Track B2B communication in the Microsoft Operations Management Suite (OMS)" on the Microsoft website at https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-track-b2b-messages-omsportal

As with the other solutions in this chapter, the Logic Apps B2B solution is based on Azure Diagnostic Logs. Add the solution per the usual steps in the Azure or OMS portal, then enable diagnostic logging for your Logic Apps, published to Log Analytics.

Service Fabric Analytics

The Service Fabric solution uses Azure Diagnostics data from your Service Fabric VMs, by collecting this data from your Azure WAD tables. Log Analytics then reads Service Fabric framework events, including Reliable Service Events, Actor Events, Operational Events, and Custom ETW events. With the solution dashboard, you are able to view important issues and relevant events in your Service Fabric environment.

To configure the Service Fabric Analytics solution, you need:

  • A Service Fabric Cluster deployed to an Azure subscription
  • The Microsoft Monitoring Agent (MMA) extension in each VM scale set
  • Connect the Service Fabric Cluster to the Log Analytics workspace

STEP-BY-STEP: For Detailed step-by-step instructions, including enabling Diagnostic Logs for your Service Fabric Analytics environment, are available at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-service-fabric-azure-resource-manager


When enabling Diagnostic Logs for your first round of the Azure monitoring solutions for OMS described in this chapter, you may encounter difficulties. Specifically, If you receive the following error message, the Microsoft.insights resource provider is not registered:

Failed to update diagnostics for 'resource'. {"code":"Forbidden","message":"Please register the subscription 'subscription id' with Microsoft.Insights."}

To resolve this issue, register the resource provider using the following steps in the Azure portal:

  1. In the navigation pane on the left, click Subscriptions.
  2. Select the subscription identified in the error message.
  3. Click Resource Providers.
  4. Find the Microsoft.insights provider.
  5. Click the Register link.


Once the Microsoft.insights resource provider is registered, retry configuring diagnostics.

If you are trying to enable Diagnostic Logs in PowerShell and you receive the following error message, you need to update your version of PowerShell:

Set-AzureRmDiagnosticSetting: A parameter cannot be found that matches parameter name 'WorkspaceId'.

To resolve this issue, update your version of PowerShell to the November 2016 (v2.3.0), or later, release using the instructions in the article, "Get started with Azure PowerShell cmdlets".


At the start of this chapter, we introduced you to the Office 365 Analytics Solution for viewing activity levels and significant events in Office 365. Then, we moved through an array of OMS solutions targeting a number of Azure features. These OMS solutions share a common element – they rely on Azure Diagnostic Logs for the feature they target.

We finished the chapter with troubleshooting tips for resolving the most common issues related to configuring OMS solutions that rely on Azure Diagnostic Logs. These solutions demonstrate the advantage of a cloud platform with a rich (and programmatically accessible) logging feature.