VSC-3 – a summary of the experiences with an HPC system immersed in oil
| dc.contributor.author | Zabloudil, Jan | |
| dc.contributor.author | Haunschmid, Ernst | |
| dc.date.accessioned | 2021-12-14T10:57:30Z | |
| dc.date.available | 2021-12-14T10:57:30Z | |
| dc.date.issued | 2021 | |
| dc.description.abstract | The electric energy consumed by computer systems is fully converted into heat, which needs to be removed from the system. As the internal temperature of processor chips is typically 80°C in operation and the temperature of the outside air hardly exceeds 45°C, a physicist would not easily understand, why heat pumps, consuming extra energy, are needed to remove heat to the outside. Traditional computer centers nevertheless consume copious amounts of extra electric energy to operate heat pumps. Consequently, various options for the efficient cooling of computers have been considered for several decades. Immersion cooling is an intriguingly simple option. You just find the right fluid and submerge the whole system in it. Then you only cool the fluid, which should be simple. There are, however, challenges to be met, which will be discussed. After several years of experiments with a small system, VSC-3 was procured as a full-scale immersion cooled system. Already at installation, several issues needed to be addressed, mostly concerning the compatibility of various submerged components with the fluid. Nevertheless, with some adaptions, the system got up and running. Due to the very simple hardware surrounding the nodes, the system had a very good price/performance ratio and, from a point of view of energy efficiency, the whole installation surpassed the expectations with an average PoE of 1.03. Additionally, roughly 10% of the energy consumption of a conventional system were saved by the absence of cooling fans. There were, however, serious downsides. First and foremost, the maintenance of VSC-3 was challenging to say the least. While compute nodes had hardly any problem, many other components suffered from the prolonged contact with hot fluid. Cables became stiff and some wicked oil outside the containers. Legions of InfiniBand switches failed for reasons, that will probably never be fully understood. Also, the containers and the oil installation were prototypes, leading to leakage. Ultimately, the question of the future of immersion cooled systems remains. Let’s assume, engineering problems can be solved without destroying the economic appeal of immersion cooling. The energy efficiency aspect will become more important in a greener future. The main problem, which remains to be solved, is the compatibility of computing components with the immersion fluid. Looking at VSC-3 as our main system for several years, we must however say, it served us very well. During it’s 6.5 years it delivered more than 1.25 billion core-hours to more than 700 projects. | en |
| dc.identifier.doi | 10.18420/informatik2021-049 | |
| dc.identifier.isbn | 978-3-88579-708-1 | |
| dc.identifier.pissn | 1617-5468 | |
| dc.identifier.uri | https://dl.gi.de/handle/20.500.12116/37714 | |
| dc.language.iso | en | |
| dc.publisher | Gesellschaft für Informatik, Bonn | |
| dc.relation.ispartof | INFORMATIK 2021 | |
| dc.relation.ispartofseries | Lecture Notes in Informatics (LNI) - Proceedings, Volume P-314 | |
| dc.subject | HPC | |
| dc.subject | Immersion Cooling | |
| dc.subject | Energy Efficiency | |
| dc.title | VSC-3 – a summary of the experiences with an HPC system immersed in oil | en |
| gi.citation.startPage | 609 | |
| gi.conference.date | 27. September - 1. Oktober 2021 | |
| gi.conference.location | Berlin | |
| gi.conference.sessiontitle | Workshop: Energie- und Ressourceneffizienz von Rechenzentren (E2R21) |
Dateien
Originalbündel
1 - 1 von 1
Vorschaubild nicht verfügbar
- Name:
- E1-5.pdf
- Größe:
- 160.53 KB
- Format:
- Adobe Portable Document Format