- Published: Tuesday, 29 July 2014 15:44
Once in a while it come to a problem where the SSY-V GUI doesn't show the actual status of all components within a server group. The last time I saw this problem was after a driver/firmware update on both DCS in a server group and a normal reboot of both systems. After the second node was updated and rebooted, the GUI shows all iSCSI ports as "not started" or even "not found". Checking the device manager all NICs had the DataCore iSCSI driver attached to them. Starting the server virtualization didn't help either. The FC mirror ports on the other hand showed up as connected and the vDisks started to resync after a few minutes running to 100% and changed the state to "Not defined". So something really strange was happening here.
Rebooting the server several times also didn't make any change.
Now the most important thing was to check the traces with the DCSTRaceConsole.msc MMC plugin you can find in the SANsymphony installation folder. Checking the state of the iSCSI HBAs (identified by the MAC addresses and the FC HBAs identified by WWNs) they appear to work as expected on both DCS. All ports came up and accepted connections from the application servers but were still shown as "down" in the GUI.
To be sure the application servers really had connection we checked them and saw that all pathes were available. So it seemed to be only a GUI problem.
As such a problem is hard to check for non-DataCore support staff the best thing to do is to involve DataCore support. They identified the problem as a GUI thing quite fast and advised to do a livestop simultaneously on both DCS. The exact procedure is:
a. On both Datacore Servers open the Windows services panel, open the properties dialog for DataCore Executive Service then choose the recovery tab. Change the action for all failures to "Take no Action" (click ok)
b. Close the SANsymphony-V management console UI on all DataCore Servers as well any remote management console machine (if there is one).
c. On both DataCore servers simultaneously open a cmd prompt ( run as administrator ), navigate to the SANsymphony program folder and issue the following command:
Allow the above command to complete successfully on both DataCore servers before proceeding to the next step
d.Issue the command on both DataCore servers:
Allow the above command to complete successfully on both DataCore servers before proceeding to the next step.
e. Open the SANsymphony-V management console (UI) and check that the configuration appears as expected.
f. Set the recovery option for the DataCore Executive Service back to "Restart the service" for any failures on both servers
One thing to note here is that the forementioned procedure should only be done when advised by DataCore support. If you are in a test environment feel free to do it at your own risk but as the livestop feature doesn't have any effect on the storage virtualization layer I don't see any problems here.
After the procedure we saw the GUI reflecting every status as OK and such the problem was solved.
So if you ever see such a behavior of your GUI, give the livestop feature a try.