Using GPUs for arithmetic-intensive workloads is getting a more and more common setup. With GPUs from NVIDIA like Tesla or Keppler based boards, the performance of GPU-based computing is getting extremely high and thus getting more and more interesting for customers.
Beside the high power input rate and the needed cooling for these cards inside modern server systems there is another hard limitation I wasn't aware of since today.
For HP Proliant servers (and probably all other vendors too as it is more a general than a specific vendor problem) there is a hard limit on the amount of memory one can use inside a server system when using NVIDIA based GPUs. Because of memory addressing limitations inside the NVIDIA GPU the maximum amount of useable RAM is limited to less than 1TB.
1TB sounds quite a lot and in fact, it is. But with memory capacities raising steadily and costs dropping the time for systems with more than 1 TB is near. Even a standard HP Proliant DL380 Gen9 system is capable of up to 1.5TB of RAM. Additionally in HPC environments where GPUs are used, memory is often a key factor.
So don't be fooled and think that amount of RAM is only seen in very rare and special configurations. It isn't....
There is an advisory from HP linking to some NVIDIA documentation for all of you who want to get a closer look at this limitation.