PARS-Mitteilungen 2024

Auflistung nach:

1 - 1 von 1

Zeitschriftenartikel
Automatic Code Transformation of NetCDF Code for I/O Optimisation
(PARS-Mitteilungen: Vol. 36, 2024) Squar, Jannek; Fuchs, Anna; Kuhn, Michael; Ludwig, Thomas
Even small improvements to applications can have a huge impact when running on massive parallel systems. Domain experts often lack sufficient computer science expertise or face significant challenges when trying to implement new features such as data compression or parallel I/O. We present anextension to CATO, a code transformation tool that automatically inserts new features and optimisations into scientific code to demonstrate their use and benefits. It helps to overcome initial barriers and supports guided self-learning in a user-friendly way. In this work we implement and evaluate an LLVM pass to automatically find, analyse and transform an application using the netCDF API to optimise the runtime and memory as well as the storage footprint during the I/O phase of the application by inserting parallelisation and compression. Our evaluation shows good speedup and near-optimal memory usage when the modified application is run on distributed hardware using Lustre as the parallel file system backend.