Zeitschriftenartikel
Automatic Code Transformation of NetCDF Code for I/O Optimisation
Lade...
Volltext URI
Dokumententyp
Text/Journal Article
Dateien
Zusatzinformation
Datum
2024
Autor:innen
Zeitschriftentitel
ISSN der Zeitschrift
Bandtitel
Quelle
Verlag
Gesellschaft für Informatik e.V., Fachgruppe PARS
Zusammenfassung
Even small improvements to applications can have a huge impact when running on massive parallel systems. Domain experts often lack sufficient computer science expertise or face significant challenges when trying to implement new features such as data compression or parallel I/O. We present anextension to CATO, a code transformation tool that automatically inserts new features and optimisations into scientific code to demonstrate their use and benefits. It helps to overcome initial barriers and supports guided self-learning in a user-friendly way. In this work we implement and evaluate an LLVM pass to automatically find, analyse and transform an application using the netCDF API to optimise the runtime and memory as well as the storage footprint during the I/O phase of the application by inserting parallelisation and compression. Our evaluation shows good speedup and near-optimal memory usage when the modified application is run on distributed hardware using Lustre as the parallel file system backend.