In recent months, Microsoft has released a number of solutions enabling visibility into Microsoft Azure-based products and services, including:
What almost all of these solutions have in common is that they are based on ingestion of Azure Diagnostic Log data in OMS. In this chapter, we will touch briefly on each of these solutions, reviewing their features and functions. We will also look at how to enable diagnostic logging in greater depth, including how to enable logging on a more granular basis via PowerShell. Finally, we will touch on troubleshooting problems enabling diagnostic logging.
Azure Diagnostic Logs are logs emitted by a resource that provides verbose data about the operation of that resource. The content of these logs varies by resource type (for example, Windows Event System Logs are one category of diagnostic log for VMs, whereas blob, table, and queue logs are categories of diagnostic logs for storage accounts) and differ from the Activity Log (formerly known as Audit Log or Operational Log), which provides insight into the operations that were performed on resources in your subscription. However, not all resources support the new type of Diagnostic Logs.
Note: A complete list of supported services and schema for Diagnostic Logs is available in "Collect and consume diagnostic data from your Azure resources" at https://docs.microsoft.com/en-us/azure/monitoring-and-diagnostics/monitoring-overview-of-diagnostic-logs
What you can do with Diagnostic Logs?
Azure offers a number of processing options Diagnostic Logs (shown in Figure 1), including:
FIGURE 1. AZURE DIAGNOSTIC LOG STORAGE & PROCESSING OPTIONS
The storage account or event hub namespace does not have to be in the same subscription as the resource emitting logs, as long as the user who configures the setting has appropriate role-based access to both subscriptions.
Diagnostic Logs for non-compute resources are configured using Diagnostic Settings.
Diagnostic Settings for a resource control the following:
You can enable diagnostic logs in one of five ways, though not all log enablement options are possible across all five methods (shown in Figure 2):
FIGURE 2. RESOURCE LOG CONFIGURATION OPTIONS
While it does not make sense to duplicate the step-by-step instructions for all five methods, it is worth mentioning that some OMS solutions involve resource providers that cannot be configured through the Azure portal UI. Specifically, the SQL Server Analytics and Web App Service solutions tap into resource providers that fall into this category. For these resource types, and indeed all OMS-supported types, Microsoft provides a PowerShell script to make enabling diagnostic logging an easier task.
Now, we will look at the steps for enabling diagnostic logging leveraging the script provided by Microsoft.
Note: These steps assume that you have the latest version of Azure Resource Manager PowerShell installed (4.2 at the time this was written). You can install via the Web Platform Installer at https://www.microsoft.com/web/downloads/platform.aspxor via PowerShell Gallery at https://www.powershellgallery.com/packages/AzureRM.
save-script -Name Enable-AzureRMDiagnostics -Path "C:\temp"
Push-Location C:\temp
.\Enable-AzureRMDiagnostics.ps1 ` -SubscriptionId "0b62f50c1-c15a-40e2-xxxx-xxxxxxxxxxxx" ` -WSID "47837ac4-89f5-476b-xxxx-xxxxxxxxxxxx"
The script will prompt you to choose the resource type for which you want to enable diagnostic logging, as shown in Figure 3. Just choose a number and script will enable diagnostic logging for the resource type you selected.
FIGURE 3. OUTPUT OF MICROSOFT ENABLE DIAGNOSTIC LOGGING SCRIPT
Note: The script enables logging for all resources of the selected type. It is possible to enable logging for resources of the specified type for a specific resource group by adding the -ResourceGroup parameter and specifying the resource group name.
You can also enable Azure Diagnostic Logs via Azure Resource Manager template. A sample template is shown below.
You can download the full script from GitHub at https://github.com/insidemscloud/OMSBookV2, in the \Chapter 9\scripts directory. The file name is DiagArm.json.
Finally, you can enable Diagnostic Logs via the Azure portal. A sample template is shown here. The Activity Log is available in Azure for free without being an OMS customer, but if you are also an OMS customer, the Activity Log will show up as part of OMS Log Analytics.
Within a few hours, log data for the solution you are working with should be available in the OMS portal.
For more information on collecting Azure service logs in OMS Log Analytics, see "Collect Azure service logs and metrics for use in Log Analytics" on the Microsoft website at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-storage.
With the Office 365 solution, you can perform the following types of management activities:
FIGURE 4. OFFICE 365 SOLUTION TILE
Data that is collected for Office 365 is all based on the current functionality of the Office 365 Management Activity API. Today, this API includes management activities for Exchange, SharePoint, and Azure Active Directory. This information is presented in the dashboard behind the Office 365 tile, shown in Figure 4.
STEP BY STEP: The detailed configuration instruction for OMS Office 365 solution can be at Microsoft Azure Documentation site: https://docs.microsoft.com/en-us/azure/operations-management-suite/oms-solution-office-365
FIGURE 5 OFFICE 365 SOLUTION VIEWS
The Office 365 solution offers four views, as shown in Figure 5:
Office 365 solution in OMS will enable you to search Office 365 user activities in the OMS portal. Although you can utilize search capabilities provided by OMS, you have detailed information for your Office 365 activities with more than 50 fields for different Office workloads. You can find some use cases in the Example Queries section.
With custom search queries that are fine-tuned for your organization's needs, you can create alerts on these queries that will show up in the Alert Management solution on your Overview page. This will help you to monitor Office 365 alerts along with your other alerts in OMS using the Add Alert Rule interface, shown in Figure 6.
FIGURE 5. OMS ALERT RULE CONFIGURATION
Below are a few frequently asked questions and tips to ensure you are able to successfully implement and get the most value from the Office 365 Analytics solution.
The Azure Activity Log Analytics solution is intended to provide insights into activities across multiple Azure subscriptions in a single view. The primary difference between this solution and all the others mentioned in this chapter is that this solution is based on Activity Logs, rather than Diagnostic Logs. The primary value of this solution (in the author's opinion) is that it provides an easy way to search for specific types of activities, or specific users across multiple Azure subscriptions to identify trends that would otherwise be difficult and laborious to gather one log at a time. Read and write activities across a wide variety of resource providers are covered by this solution.
What's the difference between Azure Activity and Diagnostic logs?
The Azure Activity Log is a log that provides insight into the operations that were performed on resources in your subscription. The Activity Log was previously known as "Audit Logs" or "Operational Logs," since it reports control-plane events for your subscriptions. Using the Activity Log, you can determine the 'what, who, and when' for any write operations (PUT, POST, DELETE) taken on the resources in your subscription.
You can also understand the status of the operation and other relevant properties.
Activity Logs provide data about the operations on a resource from the outside. Diagnostics Logs are emitted by a resource and provide information about the operation of that resource.
The Azure Activity Logs tile simply displays a count of Activity Log records in OMS, as shown in Figure 6.
FIGURE 6. AZURE ACTIVITY LOGS ANALYTICS TILE
When you click on the tile, the Activity Logs dashboard presents more detailed info, including log entry counts by date and event status, as well as counts by resource and resource provider, as shown in Figure 7.
FIGURE 7. AZURE ACTIVITY LOGS ANALYTICS DASHBOARD
You can then click on any of the views in the dashboard, to drill down to the query detail, and to events belonging to an individual resource or resource provider, as shown in Figure 8.
FIGURE 8. RESOURCE PROVIDERS LIST, ACTIVITY LOG ANALYTICS SOLUTION
Using the log search, you can correlate data from this solution across other solutions, including user activities, authorization changes, service health and outages from logs across your subscriptions.
STEP-BY-STEP: Step-by-step instructions for configuring the Activity Log Analytics solution are available in "View Azure Activity logs" at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-activity
The Azure SQL Monitoring solution in OMS collects and visualizes important SQL Azure performance metrics. By using the metrics that you collect with the solution, you can create custom monitoring rules and alerts. You can also monitor and visualize Azure SQL Database and elastic pool metrics across multiple Azure subscriptions and elastic pools. The solution also helps you to identify issues at each layer of your application stack. The solution aggregates Azure diagnostic metrics together with Log Analytics views to present data from all your Azure SQL databases and elastic pools in a single Log Analytics workspace.
The Azure SQL Analytics solution does not use agents to connect to the Log Analytics service. As a result, the connected source that is supported by this solution is an Azure storage account. Azure metric data is sent to Log Analytics using the storage account. However, when you enable diagnostic logs to be forwarded to OMS, the data is not sent to Azure storage outside of OMS, avoiding duplication.
Because enabling diagnostic logging for Azure SQL is not possible in the Azure portal UI, Microsoft provides the sample script mentioned earlier in this chapter in the section titled "Enable Diagnostic Logging via PowerShell". To enable Azure SQL diagnostic logs to be forwarded to OMS Log Analytics, follow the steps detailed in that section and select the 'Microsoft.sql/servers/databases' resource provider.
While this solution supports data visualization from multiple Azure subscriptions, you have to use PowerShell to configure collection across multi-subscriptions. You can enable collection for an entire subscription or a single resource group.
Once you have enabled Diagnostic Log forwarding to Log Analytics and events are processed, the dashboard for this solution will be populated. The Azure SQL Analytics tile will reflect the number of Azure SQL databases and elastic pools for which logs have been forwarded, as shown in Figure 9.
FIGURE 9. AZURE SQL ANALYTICS TILE
The dashboard behind this tile presents more detailed data on the Azure subscriptions, Azure SQL servers, databases, and elastic pools, as shown in Figure 10.
FIGURE 10. SQL SERVER ANALYTICS DASHBOARD
As with all OMS solutions, the data surfaced through OMS Log Analytics can be leveraged to create e-mail alerts or trigger other automated response.
The Azure Web Apps Analytics solution provides insights into your Web Apps by collecting different metrics across all your Azure Web App resources. This solution provides visibility into the following metrics:
As with the Azure SQL solution, enabling forwarding of diagnostic logs for Azure Web Apps to OMS Log Analytics is not possible in the Azure portal UI. Microsoft provides the sample script mentioned earlier in this chapter in the section titled "Enable Diagnostic Logging via PowerShell". To enable Azure Web Apps diagnostic logs to be forwarded to
OMS Log Analytics, follow the steps detailed in that section and select the 'Microsoft.web/sites' and 'Microsoft.web/ServerFarms' resource providers.
The Azure Web Analytics dashboard is shown in Figure 11.
FIGURE 11. AZURE WEB ANALYTICS DASHBOARD
Application Insights, which is an analytics service that's designed for developers, monitors live web applications. It enables you to detect and diagnose performance issues and understand what users do with your applications. Integrating your Application Insights apps to OMS increases your organization's visibility over your applications by having operation and application data in one place.
With the Application Insights Connector in OMS, you can:
To use the Application Insights Connector, you have to link your Application Insights apps to your OMS Log Analytics workspace. To do this in the Log Analytics portal:
FIGURE 12. LINKING APPLICATION INSIGHTS TO OMS WORKSPACE
STEP-BY-STEP: For step-by-step configuration guidance and screenshots, see "Improving Developer / IT Collaboration with
Application Insights Connector for OMS Log Analytics" on the Microsoft website at https://blogs.technet.microsoft.com/msoms/2016/09/26/application-insights-connector-in-oms/.
Once you have completed the configuration steps, the Application Insights tile will reflect the number of entries logged via the Application Insights Connector, as shown in Figure 13.
FIGURE 13. APPLICATION INSIGHTS CONNECTOR TILE
When you click on the tile, you will see relevant details of your application, such as the requests per hour of this Azure App Service application, as shown in Figure 14.
FIGURE 14. SERVER REQUESTS FROM APPLICATION INSIGHTS DASHBOARD
If you click on the chart or entries in the previous figure, you will get to the query used to populate the chart, as shown in Figure 15.
FIGURE 15. QUERY FOR SERVER REQUESTS PER HOUR
The Application Insights Connector solution enables deep monitoring for your custom applications, without the need to develop a monitoring plugin or solution for every application you build.
Microsoft Azure Application Gateway provides Application Delivery Controller (ADC) as a service, offering various layer 7 load balancing capabilities for your application. It allows customers to optimize web farm productivity by offloading CPU intensive SSL termination to the Application Gateway. It also provides several other Layer 7 routing capabilities.
Application Gateway also has a web application firewall (WAF).
Additional Reading: You can learn more about Azure Application Gateway at https://docs.microsoft.com/en-us/azure/application-gateway/application-gateway-introduction.
The Azure Application Gateway Analytics solution is intended to help you more easily troubleshoot application issues by providing visibility into application gateway logs. Key scenarios supported by the solution out-of-box include:
STEP-BY-STEP: For step-by-step installation and configuration guidance, see "Azure networking monitoring solutions in Log Analytics" on the Microsoft website at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-networking-analytics.
The Azure Network Security Group (NSG) Analytics solution helps you more easily troubleshoot issues across your Azure networks by providing visibility into NSG rules and logs. The solution enables visibility into the following scenarios out-of-box:
Diagnostic logging must be enabled for each NSG you want to collect data for. The section titled "What can you do with Diagnostic Logs?", earlier in this document, explains where diagnostic logs can be sent. The Network Security Group Analytics tile in the OMS portal is shown in Figure 16.
FIGURE 16. NETWORK SECURITY GROUP ANALYTICS TILE
When you click on the NSG Analytics tile, shown in Figure 16, you will then see details on both allowed and blocked NSG flows on the Azure Network Security Group Analytics dashboard, as shown in Figures 17 and 18.
FIGURE 17. NSG ANALYTICS DASHBOARD - BLOCKED FLOWS
Notice that each includes the subnet and physical addresses of machines with blocked flows, enabling drill down to the subnet and individual host.
FIGURE 18. NSG ANALYTICS DASHBOARD - ALLOWED FLOW
The HDInsight HBase solution for OMS provides Log Analytics, monitoring and alerting capabilities for HDInsight HBase. With this solution, you can enable a number of monitoring capabilities, including:
Prerequisites:
In order to deploy and configure the solution, a couple of items are assumed to be true:
As with other OMS solutions, you can add the HDInsight HBase Monitoring solution to your OMS subscription by browsing to it in the Solution Gallery and clicking the Add button. To configure the solution, you run a shell script (bash) and specific script, based on the components you have installed. There are separate scripts for Spark, HBase, Interactive Hive, and Hive.
STEP-BY-STEP: You can find step-by-step installation instructions, as well as a few sample Log Analytics queries, at https://github.com/hdinsight/HDInsightOMS
You can use the Azure Key Vault solution in Log Analytics to review Azure Key Vault Audit / Event logs. To use the solution, you need to enable logging of Azure Key Vault diagnostics and direct the diagnostics to a Log Analytics workspace. It is not necessary to write the logs to Azure Blob storage.
Azure Key Vault solution collects diagnostics logs directly from the Key Vault. It is not necessary to write the logs to Azure Blob storage and no agent is required for data collection. The Key Vault Analytics tile reflects recent levels of successful and failed requests to Key Vaults, as shown in Figure 19.
FIGURE 19. KEY VAULT ANALYTICS SOLUTION TILE
After you click on the tile, you can view summaries of your logs and then drill into details for several categories, including:
All the items in the list above are shown in Figure 20 below.
FIGURE 20. AZURE KEY VAULT ANALYTICS DASHBOARD
You can then click on the individual items on the Key Vault Analytics dashboard to see the query itself, as well as query results.
STEP-BY-STEP: For step-by-step installation and configuration guidance, see "Key Vault Analytics solutions in Log Analytics" on the Microsoft website at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-azure-key-vault
In B2B scenarios, logic apps exchange messages with other organizations using industry standard protocols, such as AS2, X12, and EDIFACT. You can also secure messages with both encryption and digital signatures.
By sending Diagnostic Logs for Logic Apps, you can access the following logs in your OMS workspace :
There is an excellent deep dive into this solution in "Track B2B communication in the Microsoft Operations Management Suite (OMS)" on the Microsoft website at https://docs.microsoft.com/en-us/azure/logic-apps/logic-apps-track-b2b-messages-omsportal
As with the other solutions in this chapter, the Logic Apps B2B solution is based on Azure Diagnostic Logs. Add the solution per the usual steps in the Azure or OMS portal, then enable diagnostic logging for your Logic Apps, published to Log Analytics.
The Service Fabric solution uses Azure Diagnostics data from your Service Fabric VMs, by collecting this data from your Azure WAD tables. Log Analytics then reads Service Fabric framework events, including Reliable Service Events, Actor Events, Operational Events, and Custom ETW events. With the solution dashboard, you are able to view important issues and relevant events in your Service Fabric environment.
To configure the Service Fabric Analytics solution, you need:
STEP-BY-STEP: For Detailed step-by-step instructions, including enabling Diagnostic Logs for your Service Fabric Analytics environment, are available at https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-service-fabric-azure-resource-manager
When enabling Diagnostic Logs for your first round of the Azure monitoring solutions for OMS described in this chapter, you may encounter difficulties. Specifically, If you receive the following error message, the Microsoft.insights resource provider is not registered:
Failed to update diagnostics for 'resource'. {"code":"Forbidden","message":"Please register the subscription 'subscription id' with Microsoft.Insights."}
To resolve this issue, register the resource provider using the following steps in the Azure portal:
FIGURE 21. REGISTERING THE MICROSOFT.INSIGHTS PROVIDER
Once the Microsoft.insights resource provider is registered, retry configuring diagnostics.
If you are trying to enable Diagnostic Logs in PowerShell and you receive the following error message, you need to update your version of PowerShell:
Set-AzureRmDiagnosticSetting: A parameter cannot be found that matches parameter name 'WorkspaceId'.
To resolve this issue, update your version of PowerShell to the November 2016 (v2.3.0), or later, release using the instructions in the article, "Get started with Azure PowerShell cmdlets".
At the start of this chapter, we introduced you to the Office 365 Analytics Solution for viewing activity levels and significant events in Office 365. Then, we moved through an array of OMS solutions targeting a number of Azure features. These OMS solutions share a common element – they rely on Azure Diagnostic Logs for the feature they target.
We finished the chapter with troubleshooting tips for resolving the most common issues related to configuring OMS solutions that rely on Azure Diagnostic Logs. These solutions demonstrate the advantage of a cloud platform with a rich (and programmatically accessible) logging feature.