Ways to organize parallel access to structured data

A. O. Ignatyev, S. Yu. Mokshin, D. V. Ivankov, E. A. Beketov

Russian Federal Nuclear Center E. I. Zababakhin All-Russian Scientific Research Institute of Technical Physics

Abstract: This paper explores ways to achieve the highest possible exchange performance with files containing structured data. The research was carried out on file systems with supercomputer systems parallel access designed to solve problems of physical and mathematical modeling of various processes and objects. For example, parallel access to raw data is considered using the Lustre file system. The article suggests a way to organize parallel access to structured data based on a specially developed PSIO storage format and the psio access library. A comparative analysis of the I/O performance of the developed data storage format and the HDF5 parallel version format is performed.

Keywords: computing system, structured data, high-performance computing simulation, mathematical simulation subsystem, data format, HDF, HPC, SIO

DOI: 10.15514/ISPRAS-2023-35(2)-8