SSY-V Powershell timeout

Monitoring DataCore isn't that easy if you don't have SC Ops Mgr or any of the other supported monitoring tools in place. And even if you have it the provided plugins sometimes lack functionality you expect from a full features monitoring system. Faced with that problem in one of our bigger DataCore installations we decided to use ICINGA to monitor whatever we want by using the SSY-V Powershell Commandlets provided by Datacore. By using these cmdlets you can grap nearly all information from the system and use the data to transform it to whatever information you want.

The other side of the story is the performance of the cmdlets. As with most of the requests new powershell processes are spawned we use plenty of ressources on the DataCore hosts to get the needed information. Things are getting worse if you lower the time between the requests.

We found a good way for balancing between request interval and needed ressources so we currently have a pretty good monitoring solution for SSY-V based on ICINGA.

This morning we got an alert from the monitoring system:

"Connect-DcsServer : Timed out while waiting for a controller to become available".

 

Looking at the GUI all seems to be fine. We checked for outstanding requests and saw a lot of them in the monitoring queue. To identify where the problem occured we started the check scripts with minimal settings and saw that even the first logon from the cmdlets to the DataCore server group fail. Using "Connect-DcsServer" throws the same timeout error. Even the start of the powershell took quite a minute.

As there is no special service or process we could restart we tried to kill all open powershell tasks and retried but still the connect timed out. So we used our "last line of defense" before calling DataCore support and did a livestop on the first DCS. After restart of the DataCore Management services the connection still timed out. So we did the same restart on the second node and tried again. 

Voila, the powershell cmdlets started fast and could connect to the server group.

So if you ever face this kind of timeout problem do a livestopn on BOTH resp. ALL DCS in the server group and this should fix the problem.

 

Leave your comments

Post comment as a guest

0
Your comments are subjected to administrator's moderation.
  • No comments found
Powered by Komento
joomla templatesfree joomla templatestemplate joomla
2017  v-strange.de   globbers joomla template