next up previous [pdf]

Next: Acronyms Quick Reference Up: Appendix Previous: Appendix

Tesla S1070 system setup

This section details the procedure to install and configure the hardware for the Tesla S1070 GPGPU system.

During my early work, I configured the ProLiant system to run Ubuntu 8.10 and Nvidia 180.22 drivers. This required recompiling Nvidia Debian kernel modules (.ko files). The recompiled modules successfully connected to the S1070 system, but incorrectly identified it as an Nvidia C1060. Downgrading the operating system to Ubuntu 8.04 enabled the modules to correctly connect and recognize the Nvidia S1070, but also produced an unstable system, which occasionally crashed. Following advice from Nvidia, I switched the ProLiant operating system to CENTOS and had significantly more success. However, the Nvidia 180.22 drivers have not been fully tested on the 1U rack-mount S1070 systems with four GPUs. There were several system hang-ups and unexpected, non-repeatable crashes. It should be noted that the GPGPU driver for the 1U Tesla system interferes with some automatic configuration of the Linux operating system (specifically graphic configuration for X11). This happens because the S1070 appears to X11 to be a video accelerator and display driver even though it cannot be connected to a physical display monitor.

Another potential configuration problem arises from the presence of two Host Interconnects on the S1070 1U unit. The Nvidia documentation mentions that these interconnects allow the S1070 to optionally connect to two separate host CPU systems. However, even though only one host is used in our system, we found that both interconnects should be used because Connecting and configuring only one card results in access to only 2 out of the 4 available Tesla T10 GPUs on the S1070 1U server. Using both interconnects allows access to all four GPUs, and also doubles the PCI-e bandwidth available to the S1070 memory controller.

The final system environment runs CENTOS 5.2 for x86_64 and using the Nvidia Tesla Driver (Linux x86_64 - 177.70.11). Two Host Interconnect Cards (HIC) are installed and configured in the ProLiant. Both cards are connected to the S1070 unit via two PCI-e cables. This setup produces a reliable and functional system for GPGPU computational acceleration. Much difficulty can be avoided by using exactly these system and driver versions.


next up previous [pdf]

Next: Acronyms Quick Reference Up: Appendix Previous: Appendix

2009-05-05