OpsMgr 2012 R2 – PSScript: Automate Config Failover Gateway for SCOM agents

It’s an easy way to monitor servers in an untrusted domain. With a gateway is SCOM able to monitor the servers in a untrusted domain. Mostly and normally, you planned to implement 2 gateways per untrusted domain for high availability of monitoring the untrusted domain. Unfortunately, if you discover a new server in that untrusted domain the second gateway is not configured as failover automatically. The agent has only a connection with the primary gateway, which you gave up in the installation or discovery.

To set the failover you have to use PowerShell. With some Cmdlets you could set the primary and the failover gateway per agent. I’m not a fan of manual actions. So, I made a PowerShell script for a monitor and one script for setting the failover on SCOM agents. The reason for automate this process is; if you add a new server into SCOM, you will forget this manual action to set the failover and that’s not good for the availability of the servers, if the primary gateway goes down for a restart or whatsoever. So it’s was time to build a script to automate this process..

I have basic PowerShell skills. If you have another, better and efficient idea or you have a comment. Please let me know, I appreciate that..

I have to skip some standard steps, otherwise the blog will be too big..

What I did:

  • Made a new monitor (rule is also possible) Unfortunately, you have to use Authoring Console or MPAuthor to make a PowerShell based monitor or rule, instead the SCOM console himself.
  • Made a PowerShell script to set the failover for the SCOM agent.
  • Made a notification based on the alert from the new monitor. Rule is also possible, but I will only explain the monitor in this blogpost.
  • The notification start the script.
  • The script update the alert, closed the alert later and reset the monitor.

I have worked with System Center Authoring Console to build the monitor. That’s why this blogpost is based on Authoring Console only.

Before you build a monitor you have to make a Probe action. This Probe contains the PowershellPropertyBagProbe, like this:

Click on the Edit button to edit the Probe. You have to choose which editor you want to edit. (you have to hit 2 times on Edit for editing the XML file)

Then, you have to add the PowerShell script into the XML file between <SCRIPTBODY> </SCRIPTBODY> We are not using Arguments (is for VBScript) and Parameters. Please use also the <![CDATA[ at the beginning of the PowerShell Script and ]]> at the end of the script. This one is needed if you are using strange characters which are illegal in XML elements, like & and <

Like this:

<Configuration p1:noNamespaceSchemaLocation=”C:\Users\albert.neef\AppData\Local\Temp\Script – Microsoft.Windows.PowerShellPropertyBagProbe.xsd” xmlns:p1=”http://www.w3.org/2001/XMLSchema-instance”&gt;

<ScriptName>GetFailoverConfig.ps1</ScriptName>

<Arguments />

<ScriptBody> <![CDATA[

#SCOM settings

$api = New-Object -ComObject “MOM.ScriptAPI”

$bag = $api.CreatePropertyBag()

[xml]$XML = Get-Content “C:\Program Files\Microsoft Monitoring Agent\Agent\Health Service State\Connector Configuration Cache\SystemCenterTest\OpsMgrConnector.Config.xml”

$Parents = $XML.Message.State.Parents.Added.Item

$api.LogScriptEvent(“GetFailoverConfig.ps1”, 451, 0, “Script is reading the XML file “)

if($error) {

$bag.AddValue(“Result”, “GOOD”)

$bag.AddValue(“Info”, “o OpsMgrConnector.Config.xml found. Maybe this is a gateway”)

$bag

$api.LogScriptEvent(“GetFailoverConfig.ps1”, 451, 0, “No OpsMgrConnector.Config.xml found. Maybe this is a gateway”)

exit

}

if($Parents.Count -gt 1) {

$bag.Addvalue(“Result”, “GOOD”)

$bag.Addvalue(“Info”, “Failovergateway has been set”)

}else{

$bag.AddValue(“Result”, “ERROR”)

$bag.AddValue(“Info”, “No failover gateway found in OpsMgrConnector.Config.xml”)

}

$bag

$api.LogScriptEvent(“GetFailoverConfig.ps1”, 451, 0, “Script is done with reading”)

]]> </ScriptBody>

<TimeoutSeconds>60</TimeoutSeconds>

</Configuration>

The script is reading the XML file from the SCOM agent. This file is located in the Health Service State. This XML has the information about the connection with the gateway. If the agent is connected with more than 1 gateways you should see more gateways in the XML.

An example of the OpsMgrConnector.Config.xml file:

 

XML_Config_SCOM

It’s a very simple and basic PowerShell script. What it does is; get the gateway information and count how much gateways are in the XML. If one then it’s an error. If there are more than 1 gateways, it gets a Good status. Save this and close the editor. You should see that the textboxes are refreshed into the information from the editor.

Click on Apply and Ok or only Ok . You have to add a DataSource into the Management Pack.

You have to add the new created Probe and SimpleScheduler. The scheduler is needed for running the script in XX seconds. I have set this scheduler on 60 seconds for testing the script, but this is temporarily . once a day is ok for monitoring. But first you have to promote IntervalSeconds for using overrides. With override you could change the intervals for running the script. Click on the ‘triangle’ icon/button and choose for Promote…

SyncTime must be empty. Click on Ok and go to Configuration Schema tab.

Here you have to change the Type for IntervalSeconds. Default is String but IntervalSeconds is Integer, so changed that into Integer. Go to the next tab, Overridable Paremeters.

This one is empty and you should add intervalSeconds as an override. So, click on Add and choose $Config/IntervalSeconds$ You have to name it. I always use the same name as from the Configuration Scheme. Change the Configuration Element into Integer. Click on Apply and Ok. Next step is to make a MonitorType . MonitorType is only for a monitor, thus not for a Rule.

You have to add 3 things and that are the created DataSource and 2 ExpressionFilter. With ExpressionFilter you can link the healthy and unhealthy with the results from the PropertyBag out the script. Because of this the monitor knows which result is bad or good.

Like this:

Parameter Name is: Property[@Name=’Result’] Result is the name of the PropertyBag in the PowerShell. Use this setting also for the ERROR result. The parameter is case sensitive.

This is DataSource. You have to promote also the IntervalSeconds. Repeat also the steps for Configuration Schema and Overridable Parameters. Click Apply and Ok. Next step create a monitor.

We will use the Windows Server class, so the target is Microsoft.Windows.Server.Computer. Parent Monitor is System.Health.ConfigurationHealth. Go to the Configuration Tab and browse to the created monitortype. You have the edit the IntervalSeconds for the monitor.

Also Health and Alerting must be configured. Choose which unhealthy(critical or warning) you want to choose and add some text into the Alert, with automatically close if you object is healthy.

Best practices is to disable the monitor by default and use overrides for enabling to monitor for a specific server. So change Enabled from true to false. Save the management pack and import this management pack in SCOM. Error handling is because of this a lot easier. I skip the override step, this is a default step in the SCOM console.

This is the PowerShell script for setting the failover on the SCOM agent.

Param($ComputerName, $alertid)

#Function GetGatewayServers is a function that’s searching for gateways in a specific untrusted domain.

Function GetGatewayServers {

Param($Node, $startTime)

$node = $node.split(“.”)

if($node.Count -eq 3) {

$domain = $node[1]

}elseif($node.Count -eq 4) {

$domain = $node[2]

}

$Gateway = Get-SCOMGatewayManagementServer -Name “*.$domain.local”

#Logging

$GatewayNames = $Gateway.DisplayName

“$StartTime : The gateways in $domain are $GatewayNames” | Out-File “Failover.log” -Append

$script:Gateway = $Gateway

}

#This function is checking the primary and the failover gateway. Only the failovergateway server will be used for this script.

Function SetFailoverForAgent {

Param($ComputerName, $IsFailOver, $startTime, $alertid)

$Agent = Get-SCOMAgent | where {$_.DisplayName -eq $ComputerName}

$Primary = Get-SCOMManagementServer -Name $IsPrimary

$Failover = Get-SCOMManagementServer -Name $IsFailOver

Set-SCOMParentManagementServer -Agent $Agent -FailoverServer $Failover

#Logging

$FailoverName = $Failover.DisplayName

“$StartTime : The failover has been set. The failover is: $FailoverName.” | Out-File “Failover.log” -Append

#This for logging into the Alert in SCOM.

Get-SCOMAlert -Id “$alertid” | Set-ScomAlert -Comment “Failover has been set. The Failover is $FailoverName”

}

#this function is getting the primary gateway and will be used later in the ‘body’ script to filter out the primary to get only the failover gateway.

Function GetIsPrimary {

Param($ComputerName, $startTime)

$Agent = $Agent = Get-SCOMAgent | where {$_.DisplayName -eq $ComputerName}

$isPrimary = Get-SCOMParentManagementServer -Agent $Agent

#Logging

$PrimaryName = $isPrimary.DisplayName

“$StartTime : Function GetIsPrimary has found the primary gatewayserver: $PrimaryName” | Out-File “Failover.log” -Append

$script:isPrimary = $isPrimary

}

#Reset monitor

Function ResetMonitor {

Param($AlertId)

$Alert = Get-SCOMAlert -Id $AlertId

$Monitor = Get-SCOMMonitor -Id $Alert.MonitoringRuleId

Get-SCOMClassInstance -id $Alert.MonitoringObjectId | foreach { $_.ResetMonitoringState($Monitor) }

#Logging

“$StartTime : Reset monitor: $Monitor” | Out-File “Failover.log” -Append

}

#Date and time for the logfile.

$startTime = [DateTime]::Now

Import-Module OperationsManager

#if $ComputerName is empty, stop the script and log into the logfile.

if(!$ComputerName) {

“$StartTime : ERROR No ComputerName: $ComputerName” | Out-File “Failover.log” -Append

exit

}

#Logging

“$StartTime : Starting for Agent $ComputerName” | Out-File “Failover.log” -Append

#Logging into the Alert in SCOM

Get-SCOMAlert -Id “$alertid” | Set-ScomAlert -Comment “Starting script SetFailoverOnAgent.ps1”

#Get the gataways from the domain where the agent is located.

GetGatewayServers -Node $ComputerName -StartTime $startTime

#Get the Primary Gateway Server

GetIsPrimary -ComputerName $ComputerName -StartTime $startTime

#Get the failover gatewayname.

foreach($GWnode in $Gateway) {

if($IsPrimary.DisplayName -ne $GWNode.DisplayName) {

$isFailover = $GWNode.DisplayName

}

}

#Set failover for the agent

SetFailoverForAgent -ComputerName $ComputerName -IsFailover $IsFailOver -startTime $StartTime -alertid $alertid

#Logging

“$StartTime : Done..” | Out-File “Failover.log” -Append

#This for logging into the Alert in SCOM.

Get-SCOMAlert -Id “$alertid” | Set-ScomAlert -Comment “Script SetFailoverOnAgent.ps1 has finished”

Get-SCOMAlert -Id “$alertid” | Set-ScomAlert -ResolutionState 255 -Comment “Closed by SetFailoverOnAgent.ps1 Script”

#Call function RestMonitor for resetting the monitor.

“$StartTime : Reset the monitor” | Out-File “Failover.log” -Append

ResetMonitor -AlertId $alertid

Save this script on the Management Servers (all SCOM management servers)

Next step is; we have to make a new Notification. Go to the SCOM console and make a new channel. This channel is a command channel. Give the channel a name and click next.

You have to add the path to Powershell.exe and the path where the script is located. As parameters you have to add $Data/Context/DataItem/ManagedEntityDisplayName$ and $Data/Context/DataItem/AlertId$. Startup folder is the same as the path to the script. Then you have to add subscribers and the subscriptions. The subscription must be pointed to the Alert of the created monitor.

For check you could look into the eventvwr of OperationsManager on the server which has the override enabled. The monitor logged ID 451 into the Eventvwr. If you see this event ID, then the monitor is working properly and will reporting to SCOM if the monitor does not find the second/failover gateway.

You will get an alert and that alert will start the subscription that’s linked to the Powershell Script. The script will make a log file in the start-up folder. This whole process will also be logged into the history of the alert itself. You have to run this Powershell Cmdlet to check if the agent has configured failover gateway. Via this CmdLet you know that the script has ran successful.

Get-SCOMParentManagementServer –Agent (Get-SCOMAgent where {$_.DisplayName –eq “YOURHOSTNAMESERVER”} )

If you have any questions, let me know..

Thanks for reading!

Advertisements

#7 Troubleshoot: OpsMgr VBScript error event ID 21405

At the moment I’m working with OpsMgr 2007 R2 at a customer here in the Netherlands. It was time to troubleshoot the VBScript and PowerShell scripts error or exited codes 🙂  We are monitoring over 600 servers per environment(we got 4) (Windows and Linux) and so often I see some script errors from the Base OS management pack or SQL management pack.There is a view for these events. This view is located in the folder Operations Manager\PowerShell Scripts or Script and Executable Responses.

I had one server that couldn’t run VBscripts for monitoring free space and network bandwidth. I saw this event in the eventlog and in the view.

The process started at 8:55:01 failed to create System.PropertyBagData, no errors detected in the output. The process exited with 1

Command executed: “C:\Windows\system32\cscript.exe” /nologo “Microsoft.Windows.Server.FreeSpace.vbs” HOSTNAME false 500 10 300 5 2000 10 1000 5
Working Directory: D:\Program Files\System Center Operations Manager 2007\Health Service State\Monitoring Host Temporary Files 33\136046\

One or more workflows were affected by this.

Workflow name: Microsoft.Windows.Server.2008.LogicalDisk.FreeSpace
Instance name: C:
Instance ID: {946C9D1C-AEAA-3B75-B442-D16701AC4B11}
Management group: MG1

This server has 3 logical disks, C:, D: and E: and for both I got this warning. So for these logical disks the monitoring doesn’t not work properly.

What I did to troubleshoot this warning was; First I copied the vbs file from the Health Service State in the root directory of the SCOM agent. Mostly in C:\Program Files\System Center Operations Manager 2007\Health Service State\ The best way is searching on the name of the vbs file in that directory After that I pasted the file into a temp directory and opened the command prompt.  I tried to run the vbs file and got this message.

Microsoft (R) Windows Script Host Version 5.8
Copyright (C) Microsoft Corporation. All rights reserved.

Input Error: There is no script engine for file extension “.vbs”.

Ok, this is odd, because there are more VBScripts that are running for monitoring, but I got especially from freespace en networkbandwidth. So, now I have to troubleshoot this warning because they are, of course, related 😉 I tried to associate the .vbs extension via assoc.vbs, but that didn’t help. I got still the same message after running the vbs file.  So, I had to search on Google and I found this blog; http://jamesmcdonald.id.au/it-tips/input-error-there-is-no-script-engine-for-file-extension-vbs

The problem was; the server was missing the registry key ScriptEngine. After adding the key and edit the default string, I ran the VBscript again. As result I didn’t get the same error message but another message that I’m not using the arguments correctly. This is good and means that the vbs is associated with the script engine again.. 🙂

The server must have these keys and strings in the registry to associate the .vbs extension.

Windows Registry Editor Version 5.00

[HKEY_CLASSES_ROOT\.vbs]
@="vbsfile"

[HKEY_CLASSES_ROOT\.vbs\PersistentHandler]
@="{5e941d80-bf96-11cd-b579-08002b30bfeb}"

[HKEY_CLASSES_ROOT\.vbs\ScriptEngine]
@="VBScript"

That’s all folks.. Good luck!

 

Opsmgr 2007 / Powershell: Change command line parameters in the Notification Channel

I have found a PowerShell commands to change the command line parameters in the notification channel. This one works only for SCOM 2007.  This is the only way to change or update the subscription.

NOTE: this one works only on channel type: Command.


#Get the channel by name
$channel = Get-NotificationAction | Where-Object {$_.DisplayName -imatch “%NAME OF THE CHANNEL%”}

#The change for the command line parameters.
$channel.CommandLine = “-command C:\TEST\test.ps1 ‘$Data/Context/DataItem/ManagedEntityPath$\$Data/Context/DataItem/ManagedEntityDisplayName$’ -a ‘$Data/Context/DataItem/AlertName$’”

#Update the channel in SCOM
$channel.update()


 

Good luck! 🙂

Update Rollup 2 for System Center 2012 R2 Operations Manager

Today Microsoft has released a new update rollup for System Center 2012 R2.  In this blog I describe only the update for Operations Manager 2012 R2.

Description from http://support.microsoft.com/kb/2929891/ :

Issues that are fixed in this update rollup

Operations Manager

Issue 1

This update rollup makes the stored procedure performance aggregate more robust against out-of-range values.

Issue 2

Adding multiple regular expressions (RegEx) to a group definition causes an SQL exception when the group is added or run.

Issue 3

Web applications fail when they are monitored by the System Center Operations Manager 2012 R2 APM agent.

Issue 4

Service Level Objectives (SLO) dashboards sometimes load in several seconds and sometimes take minutes to load. Additionally, the dashboard is empty after it loads in some cases.

Issue 5

Operations Manager Console crashes when you try to override the scope in the Authoring pane.

Issue 6

The System Center Operations Manager console is slow to load views if you are a member of a custom Operator role.

Issue 7

This update rollup includes a fix for the dashboard issue that was introduced in Update Rollup 1.

Issue 8

SQL Time Out Exceptions for State data (31552 events) occur when you create Data Warehouse workflows.

Issue 9

This update rollup includes a fix for the Event Data source.

Operations Manager – UNIX and Linux Monitoring (Management Pack Update)

Issue 1

All IBM WebSphere application servers that run on Linux or AIX computers are not automatically discovered by the Management Pack for Java Enterprise Edition (JEE) if multiple application servers are defined in a single WebSphere profile.

Download:

You could download and install it manually or, you could use Windows Update. Link: http://catalog.update.microsoft.com/v7/site/Search.aspx?q=2929891

OpsMgr: How to use Run As Profile in Powershell

I was working on a new management pack  for Tivoli Storage Manager. I want to discover the nodes from the TSM server via an administrative console. Unfortunately you have to login in  that console to get the info out TSM. I have made a Powershell script that is using the console to export a list of nodes with some information to a text file. After the export the script imports the text file as CSV and put the information into SCOM for making it as discovered objects.  

The best and nicest thing to do is using the credentials via a Run As Profile, not hard coded in the script. This is not handy and it’s wrong. What happens if the user not exist anymore or the password has expired. You don’t want to edit the management pack for only to change the username and password. The perfect why to do that is to use the Run As Profiles via SCOM console.

After some resourcing I have found a solution to use Run As Profile in the script. This means you have to make a Run As Profile for your management pack and use the Parameters(if you are using Powershell) in the Probe action.

Let’s see. First you have to make a Run As Profile in Authoring Console. This is located in Health Model\Secure References.  You need the ID later for your Probe action configscript. Give it a nice name, because you will see that later in the SCOM console.

Go to your Powershell Script in Probe Module and add these parameters at the end of the configuration. Between </SCRIPTBODY> and <TIMEOUT>

<Parameter>

      <Name>TSMuser</Name>

      <Value>$RunAs[Name=”TEST.CST.TSM.ADMIN.RUNAS”]/UserName$</Value>

    </Parameter>

    <Parameter>

      <Name>TSMpass</Name>

      <Value>$RunAs[Name=”TEST.CST.TSM.ADMIN.RUNAS”]/Password$</Value>

    </Parameter>

  </Parameters>

 Change it to your ID. With param you can use the parameters in the script.  

Import the management pack in SCOM and go to administration. You have to make a Run As Account. This can only in SCOM and not in Authoring Console. If you  are done with making an account, go to Run As Profile. Open the Profile and add the account you just created.

So, after this your are done and the script use the credentials from SCOM instead hard coded in the script 😉

Opsmgr 2007: find SCOM database location

I was trying to find the databaseserver in a not familiar SCOM infrastucture. I have tried the Powershell, but unfortunately there are no cmdlets about that. Next thing was the Registry and there I have found the databasename en server. It’s easy, go to (on the RMS/MS server) HKLM\Software\Microsoft\Microsoft Operations Manager\3.0\Setup\ There you will find Databasename en DatabaseServerName