Other
HDF5 (Hierarchical Data Format v5)
ActivePublished
Key
innovation
A hierarchical, self-describing format for efficient, portable storage of large numeric data.
Category
Other
Abstraction level
Building block
Operation level
TrainingSystem
How it works
An HDF5 file has a filesystem-like structure: groups (like directories) contain datasets (arrays) and attributes. The library lets you read/write arbitrary slices without loading the whole file, apply compression and chunking, and stay portable across languages (C, Python/h5py and others).
Problem solved
Large, multimodal data (images, point clouds, time series) must be stored efficiently, with metadata and fast partial reading, independent of platform.