Fun with Fusion-IO cards in HP servers

Recently I installed some HP IO Accelerators in HP ProLiant DL580G7 systems. These cards are relabeled Fusion-IO PCIe cards. As this was the 1.2TB Gen1 model, we talk about the ioDrive Duo.

Beside the other aspects you should keep in mind when using those cards in a DataCore system, we stumbled across another BIG problem while setting them up.

The story started by installing Windows Server 2012, installed all drivers from HP as well as the Fusion-IO drivers. After a reboot both cards were successfully detected and installed. Then we installed SSY-V on top and shutdown the system to make the recommended BIOS settings. According to DataCores FAQ 1467 (login required) you have to make some BIOS settings to give SSY-V best hardware performance and stability. One of the settings you should make is enabling node interleaving. This will cause a NUMA architecture to work just like a UMA architecture which is optimal for Datacore's cache handling.

Said, done, starting the server and waiting for the OS to boot..1min....3mins....10mins.... the system hang at the Windows splash screen. No way to get the system up and running.

We removed one of the Fusion-IO cards and the server booted up. Shutting the server down, installing the second card once more and booting the server gave us the same splash screen experience as before. Okay, stopping the system, removing the first card and only using the second card, booting the server....nothing! The system was unable to boot.

We removed the second card, installed it in a desktop machine and tested it. Everything fine. Running the server with only the first card installed was also fine.

Playing a bit around with IRQ assignments in the BIOS, disabling all unneeded devices like USB or serial to free up some IRQ.... nothing. All the time you use two cards, the system was unable to boot.

The we found a note from HP that some Fusion-IO cards need more power than the 25W a PCIe slot normally provides. So we raised the limit to 50W but the problem still exists.

As a last step we set all BIOS settings that could cause such an issue in ANY way back to default and VOILA, the system was able to boot.

After a long way of problem checks we finally found out that (probably) in this specific hardware setup (HP ProLiant DL580G7 with 6 DualPort FC HBAs, one 2port NIC and two PCIe Accelerator cards) the node interleaving setting causes the system to freeze.

As node interleaving shouldn't have a big impact on the performance of a SSY-V server, we left it disabled and now we have a stable and decent performance system.

So for all Fusion-IO card users that have boot up problems in HP servers, try disabling node interleaving and you should be fine.

 

Leave your comments

Post comment as a guest

0
Your comments are subjected to administrator's moderation.
  • No comments found
Powered by Komento
joomla templatesfree joomla templatestemplate joomla
2017  v-strange.de   globbers joomla template