RSS

Daily health check script for VMAX storage

06 Jul

One of the challenge for our cloud team is daily checks. Not coming from a storage background, we did our daily checks by logging on to the storage web gui daily. Interestingly, there is almost no daily check scripts to be found. However, I found a reference in the EMC forum a list of commands to run for a generic health checks.

symcfg list -sid $vmax_sid -env_data -v
symcfg list -sid $vmax_sid -dir ALL
symdev -sid $vmax_sid list -v
symevent -sid $vmax_sid list -error -fatal
symdisk list -sid $vmax_sid -failed

The next task now is to put these into a health check script and figure out the output messages to check for errors.  In general, you just need to check for devices that reports degraded, failed, error or one of standard errors. However for the symdisk command, the execution is different:

symdisk list -sid $vmax_sid -failed

The command returns an error in command line when executed. That is because we are asking it to give us a list of failed devices and it returns error because no failures are found. If you execute this command in powershell, you will need to trap the error code, but this way too much work. The workaround is the pipe the error to the stdout as such and it would show up as an error in powershell.

symdisk list -sid $vmax_sid -failed 2>&1

Below is the script which I had created. This is version 1 so its pretty crude and I wished I am better a string manipulated to make the script tight.


<#
.FILE get-vmaxstatus-sendreport.ps1

.SYNOPSIS
Perform check of overall health check for VMAX and sends email with results

.DESCRIPTION
Performs a series of check using symcli*.
This script must be run on the element manager with symcli installed .NOTES

Author: Kelvin Wong, version 1.0

.PARAMETER GblVar (optional)
The name of the global variable ps1 file to use

.INPUT globalvariables_.ps1
- by default the script looks at this file in the local folder

.EXAMPLE
.\Get-VMAXStatus-sendreport.ps1
.\Get-VMAXStatus-sendreport.ps1 -gblvar globalvariables_2.ps1
#>

param(
[Parameter(Mandatory=$false)]
[String]$GblVar = "GlobalVariables.ps1"
)

function check-messageforIssue {

<#
.DESCRIPTION
Search input strings for error strings and send them back to the caller

.PARAMETER devconnstr
String pattern to confirm that device connection is successful.
If the input string doesn't contain this pattern it means device
connection has failed and this is returned back to caller

.PARAMETER result The input string to check for errors

.PARAMETER posmatch {Optional)
This is added for special conditions where the input strings has only a
message, e.g. "no failed devices found", if there are no errors.
If this message is not matched it means the input string contains the
error messages or devices.

.RETURN i
An array of strings with matched error. If no errors found, the array is empty
#>

param(
[Parameter(Mandatory=$true)]
[String]$devconnstr,
[Parameter(Mandatory=$true)]
$result,
$posmatch = ""
)

$errstrings = "(degrade|fail|error|crit|warn|offline)"
$i = @()

#All successful execution should have a header message
if (($result | select-string -pattern $devconnstr).matches.count -ge 1) {

if (!($posmatch)) {
foreach ($l in $result) {
if ($l -match $errstrings) {
$i+= $i
}
}
}

elseif (($result | select-string -pattern $posmatch).matches.count -eq 0) {
$i+= $result
}
}

else {
$i += "Error connecting to Device!"
}

return $i
}

#===== MAIN PROGRAM STARTS HERE =======

$ScriptPath = [System.IO.Path]::GetDirectoryName($myInvocation.MyCommand.Definition)
$GblVar = $ScriptPath + "\" + $GblVar

. $GblVar

$rpterr = $false #error found in this report?
$body = @() #contains main message in the email report

$vmaxcmds = [ordered]@{
symcfg_env_date = 'symcfg list -sid $vmax_sid -env_data -v'
symcfg_dir = 'symcfg list -sid $vmax_sid -dir ALL'
symdev = 'symdev -sid $vmax_sid list -v'
symevent = 'symevent -sid $vmax_sid list -error -fatal'
symdisk = 'symdisk list -sid $vmax_sid -failed 2>&1'
}

foreach ($key in $vmaxcmds.keys) {

$result = invoke-expression $vmaxcmds[$key]

if ($key -eq "symdisk") {
$issues = check-messageforIssue -devconnstr $validconnstr -result $result -
posmatch "No devices were found"
}
else {
$issues = check-messageforIssue -devconnstr $validconnstr -result $result
}

if ($issues.count -gt 0) {
$body += "$key : Issues found!"
$body += $issues
$rpterr = $true
}
else {
$body += "$key : OK"
}
}

if ($rpterr) { $subject += "RED"}
else { $subject += "GREEN"}

#you need to convert the array to string so that it displays correctly in email.
$body = $body | out-string

send-mailmessage -to $to -from $from -subject $subject -body $body -smtpserver $smtp

The global variable file contains site dependent variables.

# .FILE get-vmaxstatus-sendreport.ps1

$vmax_sid = "nnnnnnnn"
$validconnstr = "Symmetrix ID.*" + $vmax_sid

$smtp = "smtp.acme.com"
$to = "me@acme.com"
$from = "cloud@@acme.com"
$subject = "Daily Checks: VMAX $vmax_sid status - "
 
2 Comments

Posted by on July 6, 2018 in powershell

 

Tags: , , , , , ,

2 responses to “Daily health check script for VMAX storage

  1. dipeshgupta619

    February 22, 2020 at 1:20 pm

    Hi i am getting error of body argument is empty.pls help

    PS E:\VMAX-health check> .\VMAXhealthcheck.ps1
    Send-MailMessage : Cannot validate argument on parameter ‘Body’. The argument is null or empty. Provide an argument that is not null or empty, and then try the
    command again.
    At E:\VMAX-health check\VMAXhealthcheck.ps1:129 char:62
    + send-mailmessage -to $to -from $from -subject $subject -body $body -smtpserver $ …
    + ~~~~~
    + CategoryInfo : InvalidData: (:) [Send-MailMessage], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.SendMailMessage

     
    • Kelvin Wong

      February 24, 2020 at 6:30 pm

      It hard to tell what is the issue, but the error is obvious, for some reasons $body variable is empty. You may need to run this in powershell ISE to debug the issue.

       

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

 
%d bloggers like this: