An Overview of the Data-Loader Landscape: Comparative Performance Analysis

Abstract

The efficiency of Deep Learning (DL) training jobs is critically dependent on dataloaders, which facilitate the transfer of data from storage to DL-accelerated hardware during training. Recent advancements in data loading technology have demonstrated significant improvements, not only in reducing training times but also in introducing capabilities such as seamless integration with cloud storage. This paper examines the dataloader as a distinct component within the DL workflow, offering a detailed analysis of its structure and functionalities. We present a systematic evaluation of various dataloading libraries, investigating their performance across different configurations, including worker count, batch size, GPU scaling, data access patterns and remote loading. The evaluation highlights trade-offs in functionality, usability, and performance. Additionally, we examine the impact of dataset characteristics on data loading performance, showing that throughput decreases exponentially with image resolution. To support ongoing research and practical advancements, we introduce the first open-source benchmarking suite for DL data loading, which allows the community to replicate, extend, and build upon our experiments.