New Agent Maintenance Mode

Here is a new version of the Agent Maintenance Mode MP. This version is only for OpsMgr R2 as it leverages the new powershell modules only available in R2.

To recap on this solution:
It allows you to start Maintenance Mode from an monitored server using just a vbscript. Using this requires absolutely no permissions in Operations Manager. All of the components required to have this function are part of a standard OS with an agent installed. It works through Gateways so can be used in firewalled or untrusted environments.

The MaintMode.vbs writes an event to the event log that indicates the length of time Maintenance Mode should remain on. This is picked up by the management pack and processed on the RMS to turn on Maintenance Mode for that server. The on time can be specified in days, hours or minutes.

Download here

The zip file contains a Readme with more information and usage instructions.

Highligh of changes in this version:
Management Pack Changes
* Added Powershell modules in MP (R2 feature) to replace scripts on RMS. Some logic reworking in scripts
* Support for Server 2008
* Fixed ON/Off logic to not put Health Service and Health Service Watcher in to Maint Mode as R2 puts HS and HSW in to MM automatically
• Added Maint Mode event collection rules
• Added Schedule Maint Mode text file option at C:\MaintenanceModeSchedule.txt - origional by Steve Rachui - http://blogs.msdn.com/steverac/archive/2008/12/09/maintenance-mode-by-config-file.aspx
* Disables discovery on RMS as Maintenance Mode should not be used on RMS

MaintMode.vbs
* Now logs the username in the event log of who turned it on: “Maintenance Mode: ON for 6 hour(s): 360 . Turned on by domain\user”
* Adds registry markers to indcate current status under HKLM/Software/OpsMgr/Maintenance Mode. Can be used by patching tools.
* Added STATUS option that reads current status from registry.

Port Monitoring from a text file

This is another solution on how to make something that requires permissions and experience in OspMgr available to everyone to use. Teach a man to fish and all that.

These two Management packs will allow any user you specify using simple NTFS permissions to add port monitoring for a server, network device etc.

Usage

Download and Import the 2 management packs:





On each management server a discovery in the AgentlessAvailabilty MP will create a text file C:\AgentlessMonitoring\resource.txt

You can add entries to this file in the format:
{name or IP address}, port - (comma seperated)

If no port is specified it will default to port 445.
Use the hash # at the start of a line to insert a comment.

The discovery runs every 1 hour. After the discovery runs any entries in the file will then display in the console under the standard Synthetic Transaction folder
Each entry will be checked every 10 minutes.












These MPs simply reuse the existing Port Monitoring template so there is a full range of specific alerts, health states and associated Knowledge articles





To make this so that anyone can use it simply share the C:\AgentlessMonitoring folder giving write permissions to those who need it

Technical Details


The sealed Custom.PortMonitoring.Library is essentially the Port Monitoring template with the module types having any specific information promoted to class properties. The discovery in the unselaed Custom.Infra.AgentlessAvailabilty MP then creates class instances with the necessary properties.

You can leverage the sealed MP from other management packs to add port monitoring. All it takes is writing a discovery.

How to upgrade or apply a hotfix to a Clustered RMS

We have had a some difficulties over the months trying to apply hotfixes to any clustered Root Management Server. The hotfix installers try and stop the 3 OpsMgr services, apply the hotfix and them restart the services. These steps result in some unexpected behavior on a Clustered RMS.

What I have seen is that after the hotfix installer stops the RMS services the Cluster Service will try and restart them again.

This results in either the services running on the node you are trying to patch or the RMS resource group failing over. If the RMS fails over then, when the hotfix tries to restart the services after it has completed, it either times out or you have the RMS services running on both Cluster nodes. It it restarts the service on the node you are trying to patch the install will fail as the files are in use.

Here are some steps that can avoid these issues:

1. In Cluster Administrator ensure the RMS resource group is on the node you are patching. Then Pause the Node you are not patching. Simply right Click on the node and select Pause. This stops the resource group failing over to the other node.

2. Set the 3 RMS services not to restart automatically. Selecting each service, right Click and select Properties. On the Advanced tab set it to "Do not restart"

3. Apply the hotfix to the current node

4. When the hotfix has completed reset the services to restart automatically and resume the other node.

Repeat this to apply the patch to the second node.

This process also worked for me in upgrading a clustered lab environment to the Release Candidate of R2

Operations Manager Data Warehouse High CPU Utilization

Some good steps if you are having trouble with CPU on your DW.

http://www.afinn.net/2008/10/operations-manager-data-warehouse-high-cpu-utilization/

Data Warehouse Queries for Noisy Rules

Here are a few SQL queries that you can run on the Operations Manager Data warehouse to determine what are the Noisy Event and Performance rules in your enviroment.

Noisy Event Rules

select count(*) as cnt, RuleDefaultName
from Event.vEvent as ev
Left Join Event.vEventRule as evi
On ev.eventOriginID = evi.eventOriginID
Left Join vRule as r
On r.RuleRowId = evi.RuleRowId
group by RuleDefaultName
order by cnt desc

Noisy Performance Rules

select count(*) as cnt, RuleDefaultName
from Perf.vPerfRaw as pr
Left Join vPerformanceRuleInstance as pri
On pr.PerformanceRuleInstanceRowId = pri.PerformanceRuleInstanceRowId
Left Join vRule as r
On r.RuleRowId = pri.RuleRowId
group by RuleDefaultName
order by cnt desc


As expected, in a few enviroments I tested, some of the busiest performance rules were the Processor and Memory collection rules.

However there were several unexpectedly busy Event rules that accounted for a significant portion of all the event data and some of these were subsequently disabled.

Thanks to Reut for pointing me in the right direction for the initial query.

Speed up Command Shell startup

There is a post over on the Powershell team blog about how to speed up Powershell startup times. I ran this on a few of my Operations Manager Servers and it noticeably made a difference to Powershell startup times. However the Operations Manager command shell was still quite a bit slower.

I then ran the same script from within the command shell and it found something to NGEN

NGENing : Microsoft.EnterpriseManagement.OperationsManager.ClientShell.dll

In my very scientific way of couting seconds in my head before and after running this it definitley starts faster.

Check it out - http://blogs.msdn.com/powershell/archive/2008/09/02/speeding-up-powershell-startup-updating-update-gac-ps1.aspx

Clearing Console Cache and User Settings

There are a few steps to be taken to really clean up a console if you have performed some personalization.

Open regedit and delete the key HKEY_CURRENT_USER\Software\Microsoft\Microsoft Operations Manager\3.0\Console

Start the Operations Console from the command line with the option /clearcache

Example:

C:\Program Files\System Center Operations Manager 2007\Microsoft.MOM.UI.Console.exe" /clearcache