Migrate GSM Tests from SCOM to Azure Application

Global Service Monitor (GSM) is a cloud service that provides a simplified way to monitor the availability of web-based applications from multiple locations around the world. GSM tests can be performed from 16 different locations around the world.

GSM supports the following two types of monitoring:

Web Application Availability Monitoring
Visual Web Tests monitoring (test is defined as webtest file)

Important: GSM is retiring in favor of Application Insights in November 2018.

Application Insights offer two types of tests, very similar to GSM:

A URL Ping test which will send out web requests to a site to see if it is responsive. This test can be created within the Azure portal.
A Multi-Step web test which is created using Visual Studio Ultimate or Enterprise.

Besides these, there are many other additional features which are offered by Application Insights that can help to diagnose application performance issues in depth https://docs.microsoft.com/en-us/azure/application-insights/app-insights-monitor-web-app-availability

Migrate your tests from Global Service Monitor to Application Insights

The System Center team has developed a Migration tool which will migrate all your Ping Tests and Web Tests from GSM to Azure Application Insights. This tool will also create corresponding alerts in Application Insights, like what you had configured in GSM for your tests.

Important: The following sections provide information on how to migrate the GSM tests to Application Insights, also few notes and points to consider while migrating.

Prerequisites

Azure Subscription: Subscription Name is displayed in Azure portal under Subscriptions.
ResourceGroupName refers to the resource group in Azure where all the tests will be migrated to. If you do not have a Resource group created in Azure, this script will create a Resource Group with the name provided in the parameters.
ResourceLocation refers to location of Resource group metadata in Azure.
Azure PowerShell

Other considerations

Understand the following details/do the actions applicable in case you are attempting to migrate the tests under specific scenarios as mentioned below:

Scenario	Notes
You are trying to migrate more than 800 tests	AI limits these. Learn more about the limits. The resource group can hold only 800 web tests and a component can only hold 100 tests. The migration tool creates a new resource group and migrates the remaining tests to this new resource group. This new resource group will be named as ""<RESOURCE_GROUP_NAME1>" and so on.
You do not wish to run the script from the SCOM server	Install SCOM Powershell on the computer, from where you want to run the script, and provide the SCOM server name in the PowerShell.
You wish to do a logical grouping of tests	The customer can run the tool and move everything on https://resources.azure.com . There is an option to moveResources under -resourceGroups à DTSLogCleaner.

Run the migration tool

Important: To avoid duplicate tests in Application Insights, ensure you run the script only once.

Save the script/tool from Download Center to a local folder on your machine, where you want to run the tool.
Run the following PowerShell script:
".\MigrateGSMToAI.ps1 -SubscriptionName "<AZURE_SUBSCRIPTION_NAME>" -AzureResourceGroupName "<RESOURCE_GROUP_NAME>" -ResourceLocation "<RESOURCE_LOCATION>"
Tests which fail to migrate, will be logged in log file along with TestInstance IDs.
To migrate these tests, please fix the error and rerun the tool along with FailedInstanceIDs as below
.\MigrateGSMToAI.ps1 -subscriptionName "<AZURE_SUBSCRIPTION_NAME>" -resourceGroupName "<RESOURCE_GROUP_NAME>" -resourceLocation "<RESOURCE_LOCATION>" -failedMultiStepTestsInstanceId "" -failedPingTestsInstanceId ""

Logging from the tool:

The script writes in a log file for the tests for which migration was attempted. The script logs suitable messages for both – the tests which were successfully migrated and those which failed to migrate.
The log file is created under the same folder where the script file was copied.
Tests that fail migration are logged in a log file along with TestInstance IDs.

Important: During migration if there is any issue with Application Insights and all the tests are not yet migrated then please follow these steps:

If only few tests are migrated, then try to delete these tests from https://resources.azure.com . Otherwise you could delete the Resource group if it is brand new and does not contain any other resources other than the newly created availability tests
If most of the tests are migrated to AI, then use the log file to retrieve instance IDs of failed tests. Fix errors and rerun the tool as described above.

GSM tests location in Azure portal

Once you have successfully migrated all your GSM tests to Application Insights, verify these tests in the Azure Application Insights.

In the Azure portal, open the ResourceGroup that was provided in the script.
Under this resource group, you will find the tests which are migrated from GSM.

Note: Once all the tests are migrated to Application Insights, the test summary displays the test results in some time.

Sample Ping test: Application Insights

Click the green/red dot to find the result summary,

Alerts configuration for SCOM tests

In Application insights, URL ping tests are configured to monitor HTTP response. Http Status code 200 indicates success that a normal web page has been returned. All the GSM availability tests report success/failure based on this criterion after they are migrated to Application Insights. The failures can be sent through an email or setup a webhook which would be called when an alert is raised.

Note: The migration tool with create Alerts for your GSM tests in Application Insights. See highlighted is enabled.

Application insights provides the description and status of alerts under

Home-> Resource groups -> - Activity Log in Azure Portal.

Features not supported in Application Insights

There are few functionalities available in GSM but not available in Application Insights.

You cannot monitor the following scenarios in AI:

GSM allows Performance monitoring for a website based on the following metrics, However Application Insight does not have a mapping to automatically collect these performance metrics for websites. You would only see the Response Time for tests; the other Performance metrics on the list will not be monitored.

GSM allows you to enable Alerts based on HTTP Status Code. For HTTP status code 200, you will see success for others, it will be a failure.
GSM allows you to create Alerts on content match. Application Insights only support "Content must contain" parameter.

MigrateGSMToAI.ps1

<# .SYNOPSIS . .DESCRIPTION The script will migrate all the GSM tests to Application Insights. .PARAMETER Path The path to the . .PARAMETER LiteralPath Specifies a path to one or more locations. Unlike Path, the value of LiteralPath is used exactly as it is typed. No characters are interpreted as wildcards. If the path includes escape characters, enclose it in single quotation marks. Single quotation marks tell Windows PowerShell not to interpret any characters as escape sequences. .EXAMPLE C:\PS> .\MigrateGSMToAI.ps1 -SubscriptionName MyAzureSubscription -AzureResourceGroupName MyResourceGroupName -ResourceLocation "East US" -SCOMMSComputerName "MySCOMComputer" #> Param( ############## The Azure subscription Name. [Parameter(Mandatory=$true)] [string]$SubscriptionName,

############## The name of the resource group in azure portal, if you do not have the resource group created in azure, the script will create it. [Parameter(Mandatory=$true)] [string]$AzureResourceGroupName,

############## The name of the Applications Insight's component, you can provide the parameter, by default it will be take SCOM's Management Group name and all tests will be created under this component.
[Parameter(Mandatory=$false)] [string]$AIcomponentName,

############## Resource location for the Azure Resource group. [Parameter(Mandatory=$true)] [string]$ResourceLocation,

############## Computer Name for SCOM Management Server machine. Provide the parameter if you are using a remote machine to connect to the Management Server or leave blank. [Parameter(Mandatory=$false)] [string]$SCOMMSComputerName,

############## Provide the Instance IDs of Ping tests we failed to migrate. [Parameter(Mandatory=$false)] [String[]]$FailedPingTestsInstanceIds,

############## Provide the Instance IDs of MultiStep tests we failed to migrate. [Parameter(Mandatory=$false)] [String[]]$FailedMultiStepTestsInstanceIds ) #end param

############## Log the progress of Migration. function LogMessage([string]$message, [string]$logFileName, [int]$logLevel, [string]$testNameForLog) { $levelText = "" If ( $logLevel -eq $ERROR ) { $levelText = "[ERROR]" $newLogFileName = $logFileName.Replace($testNameForLog, ($testNameForLog + "-Failed")) Move-Item -Path $logFileName -Destination $newLogFileName -Force $logFileName = $newLogFileName } ElseIf ( $logLevel -eq $FATALERROR ) { $levelText = "[FATALERROR]" Write-Host "The script has ended abruptly, Please see the logs @ :- $migrationLogFileName" -ForegroundColor Red } ElseIf ( $logLevel -eq $DUPERROR ) { $levelText = "[DUPLICATE TEST]" } Else { $levelText = "[INFO]" } $messageToBelogged = $($levelText + " : " + $message) $messageToBelogged | Out-File -Append -filepath $logFileName -Force }

############## Check prerequisites function Check-Prerequisites { If ( -not (Get-Module OperationsManager ) ) { Import-Module OperationsManager }

If ( -not (Get-Module OperationsManager ) ) 
{
    LogMessage "OperationsManager Powershell module not found." $migrationLogFileName $FATALERROR
    exit
}

If ( -not ( Get-Module AzureRm.Profile ) ) 
{
    Import-Module AzureRm.Profile
}

If ( -not (Get-Module AzureRm.Profile ) ) 
{
    LogMessage "AzureRm.Profile Powershell module not found." $migrationLogFileName $FATALERROR
    exit
}

}

############## Interactive login to connect to the Azure account. function Login-ToAzure { $azureConnection = $null Try { $azureConnection = Connect-AzureRmAccount } Catch { LogMessage "Please verify the login credentials. Unable to login to the azure portal." $migrationLogFileName $FATALERROR exit }

If ($azureConnection -eq $null)
{
    LogMessage "Please verify the login credentials. Unable to login to the azure portal." $migrationLogFileName $FATALERROR
    exit
}

}

############## Function creates the resource group if already not present on azure. function Create-ResourceGroup() { $resourceGroupFromAzure = $null $IsResourceGroupFoundToBeUsed = $false $IsResourceGroupToBeCreated = $false While ( -Not $IsResourceGroupFoundToBeUsed ) { Try { LogMessage "Verifying if the resource group with Name:- $script:resourceGroupName and location:- $ResourceLocation already exists." $migrationLogFileName $INFORMATION $resourceGroupFromAzure = Get-AzureRmResource -ResourceGroupName $script:resourceGroupName -ResourceType "microsoft.insights/components" -ApiVersion 2014-04-01 If ( $resourceGroupFromAzure -ne $null ) { If ( $( $resourceGroupFromAzure | Group-Object ResourceType).Count -ge 8 ) { $resourceGroupSuffix ++ $script:resourceGroupName = $( $AzureResourceGroupName + $resourceGroupSuffix) continue } } Else { $resourceGroupFromAzure = Get-AzureRmResource -ResourceId /subscriptions/$subscriptionID/resourceGroups/$script:resourceGroupName -ApiVersion 2014-04-01 If ( $resourceGroupFromAzure -ne $null ) { LogMessage "Resource group with Name:- $script:resourceGroupName and location:- $resourceLocation already exists." $migrationLogFileName $INFORMATION $IsResourceGroupFoundToBeUsed = $true } Else { LogMessage "The azure resource group:- $script:resourceGroupName does not exist. Will try to create the resource group." $migrationLogFileName $INFORMATION $IsResourceGroupToBeCreated = $true } } } Catch { LogMessage "The azure resource group:- $script:resourceGroupName does not exist. Will try to create the resource group." $migrationLogFileName $INFORMATION $IsResourceGroupToBeCreated = $true } $IsResourceGroupFoundToBeUsed = $true }

If ( $IsResourceGroupToBeCreated )
{
    LogMessage "Creating resource group with Name:- $script:resourceGroupName and location:- $ResourceLocation." $migrationLogFileName $INFORMATION
    Try	
    {
        New-AzureRmResourceGroup -Location $ResourceLocation -Name $script:resourceGroupName
    } 
    Catch	
    {
        LogMessage "Unable to create the resource group with Name:- $script:resourceGroupName and location:- $ResourceLocation. Please verify if you have the correct privileges to create the resource group." $migrationLogFileName $FATALERROR
        exit
    }
} 
Else 
{
    LogMessage "Resource group with Name:- $script:resourceGroupName and location:- $ResourceLocation already exists." $migrationLogFileName $INFORMATION
    return
}

LogMessage "Created Resource group with Name:- $script:resourceGroupName and location:- $ResourceLocation." $migrationLogFileName $INFORMATION

}

############## Function for creating component in Application Insights. function Create-Component( $componentName ) { $kindOfComponent = "web" $componentJSON = ""

$componentJSONObject = New-Object -TypeName PSObject

Add-Member -InputObject $componentJSONObject -MemberType NoteProperty -Name name -Value $componentName
Add-Member -InputObject $componentJSONObject -MemberType NoteProperty -Name location -Value $ResourceLocation
Add-Member -InputObject $componentJSONObject -MemberType NoteProperty -Name kind -Value $kindOfComponent

$componentPropertiesObject = New-Object -TypeName PSObject
Add-Member -InputObject $componentPropertiesObject -MemberType NoteProperty -Name Application\_Type -Value "web"
Add-Member -InputObject $componentPropertiesObject -MemberType NoteProperty -Name ApplicationId -Value $componentName
Add-Member -InputObject $componentPropertiesObject -MemberType NoteProperty -Name Flow\_Type -Value "Bluefield"
Add-Member -InputObject $componentPropertiesObject -MemberType NoteProperty -Name Request\_Source -Value "rest"

Add-Member -InputObject $componentJSONObject -MemberType NoteProperty -Name properties -Value $componentPropertiesObject

$componentJSON = $componentJSONObject | ConvertTo-Json

LogMessage $( "Component JSON:- " + $componentJSON ) $migrationLogFileName $INFORMATION

LogMessage "Creating Component with Name:- $componentName." $migrationLogFileName $INFORMATION
Try	
{
    New-AzureRmResource -ResourceName $componentName -Location $ResourceLocation -PropertyObject $componentJSON -ResourceGroupName $script:resourceGroupName -ResourceType microsoft.insights/components -ApiVersion 2015-05-01 -Force
} 
Catch	
{
    LogMessage "Unable to create Component with Name:- $componentName." $migrationLogFileName $FATALERROR
    exit
}

LogMessage $( "Created Component Name:- " + $componentName + " in Resource group :- " + $script:resourceGroupName ) $migrationLogFileName $INFORMATION

}

############## Function to validate the resource location provided by the user as the parameter ResourceLocation function ValidateResourceLocation() { $azureLocations = $(Get-AzureRmLocation).DisplayName If ( -Not $azureLocations.Contains($ResourceLocation) ) { LogMessage $("Please provide a valid resource location. Here are the valid locations "+$azureLocations) $migrationLogFileName $FATALERROR exit } }

############## Function for formating GeoLocations to REST API consumable format. function Format-GeoLocations ( $testLocations ) { $webTestGeoLocations = @{}, @{} Foreach ( $location in $testLocations ) { Switch ( $location ) {

        "emea-nl-ams-edge"  { $location = "emea-nl-ams-azr" }
        "us-ca-lax-edge"    { $location = "us-ca-sjc-azr" }
        "apac-sg-sin-edge"  { $location = "apac-sg-sin-azr" }
        "us-tx-sn1-edge"    { $location = "us-tx-sn1-azr" }
        "us-il-ch1-edge"    { $location = "us-il-ch1-azr" }
        "emea-gb-lts-edge"  { $location = "emea-gb-db3-azr" }
        "emea-gb-lts-edge"  { $location = "emea-gb-db3-azr" }
        "us-nj-ewr-edge"    { $location = "us-va-ash-azr" }
        "apac-tw-tpa-edge"  { $location = "apac-hk-hkn-azr" }

    }

    $webTestGeoLocations += @{Id = $location}
}

return $webTestGeoLocations | Select-Object -Skip 2

}

############## Common Function for Ping and Multistep test to create JSON. function Build-TestJSON ( $scomTest, $completeWebTestName, $componentName, $webTestGeoLocations, $webTestXML, $webTestKind, $frequency, $timeout, $testInstanceID) { $testID = $completeWebTestName + "-" + $componentName $testLogFileName = $( $testLogFilePath + $completeWebTestName + ".txt" ) "" | Out-File -filepath $testLogFileName -Force LogMessage $( "Building JSON for Test NAME:- " + $completeWebTestName ) $testLogFileName $INFORMATION LogMessage $( "SCOM Instance ID :- " + $testInstanceID ) $testLogFileName $INFORMATION Write-Host "Building JSON for Test NAME:- " $completeWebTestName -ForegroundColor Green ############# Convert SCOM GSM test frequency into Applications Insights UI supported values (5, 10, 15) minutes $newFrequency = [int][Math]::Ceiling($frequency/60) *60 If ( $newFrequency -gt 900 ) { $newFrequency = 900 } If ( $frequency -ne $newFrequency ) { Write-Host "The Test frequency is changed from :-"$frequency "to :-"$newFrequency -ForegroundColor Yellow LogMessage $( "The Test frequency is changed from :-" + $frequency + " to :- "+ $newFrequency) $testLogFileName $INFORMATION }

############# Convert SCOM GSM test timeout into Applications Insights UI supported values (30, 60, 90, 120) seconds
If ( $webTestKind -eq "ping" )
{
    $newTimeout = \[int\]\[Math\]::Ceiling($timeout/30) \*30
    If ( $newTimeout -gt 120 )
    {
        $newTimeout = 120
    }
    If ( $timeout -ne $newTimeout )
    {
        Write-Host "The Test Time out is changed from :-"$timeout "to :-"$newTimeout -ForegroundColor Yellow
        LogMessage $( "The Test Time out is changed from :-" + $timeout + " to :- "+ $newTimeout) $testLogFileName $INFORMATION
    }
}

$enabled = $true
$retryEnabled = $true

$propertiesObject = New-Object -TypeName PSObject
$configurationObject = New-Object -TypeName PSObject

Add-Member -InputObject $configurationObject -MemberType NoteProperty -Name WebTest -Value $webTestXML

Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name SyntheticMonitorId -Value $testID
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Name -Value $completeWebTestName
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Description -Value ("$webTestKind web test for " + $completeWebTestName)
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Enabled -Value $enabled
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Frequency -Value $newFrequency
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Timeout -Value $newTimeout
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Kind -Value $webTestKind
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name RetryEnabled -Value $retryEnabled
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Locations -Value @( $webTestGeoLocations )
Add-Member -InputObject $propertiesObject -MemberType NoteProperty -Name Configuration -Value $configurationObject

return $propertiesObject

}

############## Function for Building MultiStepTest JSON. function Build-MultiStepTest( $FailedMultiStepTestsInstanceIds ) { $multiTestJSON = "" $multiStepTestsForRequest = @() $numberOfTestsInComponent = 0 $componentSuffix = 1

$allGeoLocations = Get-SCOMClass -Name "Microsoft.SystemCenter.Omonline.OutsideIn.OutsideInPop" | Get-SCOMClassInstance

If ( $FailedMultiStepTestsInstanceIds.count -eq 0)
{
    $multiStepTests = Get-SCOMClass -DisplayName "Visual Studio Web Test Container" | Get-SCOMClassInstance
}
Else
{
    $multiStepTests = Get-SCOMClassInstance -Id $FailedMultiStepTestsInstanceIds
    $componentSuffix = $( $( Get-AzureRmResource -ResourceGroupName $script:resourceGroupName -ResourceType "microsoft.insights/components" -ApiVersion 2014-04-01 ) | Group-Object ResourceType).Count
    $componentSuffix ++
}

Foreach ( $multiStepTest in $multiStepTests ) 
{
    $multiStepTestID = $multiStepTest.Id
    If ( $numberOfTestsInComponent -eq 0 )
    {
        ##############  Invoke Create Component using New-AzureRmResource. More info -- https://docs.microsoft.com/en-us/powershell/module/azurerm.resources/new-azurermresource?view=azurermps-6.5.0
        $componentName = $("MultiStep-" + $AIcomponentName + $componentSuffix)
        $componentSuffix++
    }

    \[xml\]$testLocationsXML = $multiStepTest.'\[Microsoft.SystemCenter.Omonline.OutsideIn.VSWebTest.VSWebTestContainer\].Locations'.Value
    $testLocationsIDs = $testLocationsXML.Locations.Location.managementActionPointId

    $testLocations = $allGeoLocations | Select-Object -Property Id, Name | Where-Object {$\_.Id -in $testLocationsIDs}
    $webTestGeoLocations = Format-GeoLocations $testLocations.Name

    ## Case where test is configured to run on internal agent.
    If ( $webTestGeoLocations -eq $null )
    {
        continue;
    }
    $NumOfLocMultiTest = $webTestGeoLocations.Count
    If ( $NumOfLocMultiTest -ge 3 )
    {
        $NumOfLocMultiTest = 3;
    }
    
    \[xml\]$testConfigXML = $multiStepTest.'\[Microsoft.SystemCenter.Omonline.OutsideIn.VSWebTest.VSWebTestContainer\].VSDeclarativeTestConfigs'.Value
    $webTestXML = $testConfigXML.VSWebTestConfigs.VSWebTestElement.content
    $multiTestFrequency = $multiStepTest.'\[Microsoft.SystemCenter.Omonline.OutsideIn.VSWebTest.VSWebTestContainer\].IntervalInSeconds'.Value
    
    ## Case where test name is a duplicate
    If ( $allTests.Contains($multiStepTest.DisplayName) )
    {
        $testLogFileName = $( $testLogFilePath + "MultiStep-DuplicateTest.txt" )
        LogMessage $( "Cannot migrate test, as there is already a multistep test with name :- " + $multiStepTest.DisplayName + ". Please rename the test to migrate." ) $testLogFileName $DUPERROR
        LogMessage $( "Information to find the SCOM Instance ID :- " + $multiStepTestID + " with name :- " + $multiStepTest.DisplayName ) $testLogFileName $DUPERROR
        Write-Host "Cannot migrate test, as there is already a multistep test with name :- " $multiStepTest.DisplayName ". Please rename the test to migrate." -ForegroundColor Red
        continue
    }
    Else 
    {
        $allTests.Add($multiStepTest.DisplayName)
        $numberOfTestsInComponent++
        If ( $numberOfTestsInComponent -ge $componentLimit )
        {
            $numberOfTestsInComponent = 0
        }
    }
    
    $multiStepTestProperty = Build-TestJSON $multiStepTest $multiStepTest.DisplayName $componentName $webTestGeoLocations $webTestXML "multistep" $multiTestFrequency 120 $multiStepTestID

    $multiStepTestForRequest = New-Object -TypeName PSObject
    Add-Member -InputObject $multiStepTestForRequest -MemberType NoteProperty -Name NAME -Value $multiStepTest.DisplayName
    Add-Member -InputObject $multiStepTestForRequest -MemberType NoteProperty -Name ID -Value $multiStepTestID
    Add-Member -InputObject $multiStepTestForRequest -MemberType NoteProperty -Name PROPERTY -Value $multiStepTestProperty
    Add-Member -InputObject $multiStepTestForRequest -MemberType NoteProperty -Name COMPONENT -Value $componentName
    Add-Member -InputObject $multiStepTestForRequest -MemberType NoteProperty -Name NUMOFLOC -Value $NumOfLocMultiTest

    $multiStepTestsForRequest += $multiStepTestForRequest
}

return $multiStepTestsForRequest

}

############## Function for Http Headers for PingTest. function Build-HTTPHeaders( $httpHeaders ) { [xml]$pingHeaders = '

Migrate your tests from Global Service Monitor to Application Insights

Prerequisites

Other considerations

Run the migration tool

GSM tests location in Azure portal

Sample Ping test: Application Insights

Alerts configuration for SCOM tests

Features not supported in Application Insights

MigrateGSMToAI.ps1

Related articles

3 Strategies for Moving SAP Applications to Microsoft Azure

A walk-through of Microsoft Azure

Additional Azure Services