Acceleration of large-scale graph neural networks training by improving page utilization of solid-state drive

Abstract

The size of graph data used in graph neural network models increases rapidly every year, but the memory size of the computational unit, CPU and GPU, has not improved much. Therefore, recently, graphs have been stored in solid-state drive (SSD) storage with good capacity-to-price efficiency to train graph neural network models. However, due to the limitations of Von-Neumann's architecture (memory wall), transferring graph data from storage to computational units is a significant bottleneck. In order to overcome this limitation, in-storage processing (ISP) research has been studied. However, since most components of graph data are much smaller than pages, which are the smallest units of SSD, a problem arises that the page utilization decreases, slowing the entire training process. This thesis applies a partial page read scheme to SSD to solve this problem to increase page utilization. This thesis also proposes a page allocation strategy, a mapping scheme for the layout of graph data to fit the size of partial pages, and a customized flash translation layer (FTL) mapping table for optimizing the partial page read scheme. Additionally, this thesis implements a request manager to optimize data requests and proposes an architecture for standalone in-storage processing. The proposed method achieves, on average, 11.6$\times$ and 11.2$\times$ speedup and 14.8$\times$ and 16.8$\times$ lower energy consumption than the CPU and GPU systems.