Application of graph neural network for the analysis of resting-state functional magnetic resonance image

Abstract

We address the application of the graph neural networks to the analysis of resting-state functional magnetic resonance image data. Graph neural networks rely on graph operations that include neural network training for various graph related tasks. Several attempts have been made to apply the graph neural networks to functional magnetic resonance image data. Despite the recent progresses, a common limitation is its difficulty to explain the classification results in a neuroscientifically explainable way. Here, we develop a framework for analyzing the functional magnetic resonance image data using the graph isomorphism network, which was recently proposed as a powerful graph neural network for graph classification. We deduce the theoretical observation that the spatial aggregation operation of the graph isomorphism network is a dual representation of convolutional neural network in the graph space where the shift operation is defined using the adjacency matrix. This understanding enables us to exploit the saliency map techniques built for convolutional neural networks for use in our model, which we tailor with the one-hot encoding of the node features to directly visualize the important regions of the brain. We validate our proposed framework using large-scale resting-state functional magnetic resonance image data for classifying the sex of the subject based on the graph structure of the brain. The experiment was consistent with our expectation such that the obtained saliency map show high correspondence with previous neuroimaging evidences related to sex differences. One limitation of the forementioned method using graph isomorphism network is that it is applicable only to the analysis of static functional connectivity. We further extend the idea to propose a graph neural network framework for analyzing the dynamic functional connectivity, which considers the temporal fluctuation of the functional connectivity. The proposed spatio-temporal attention graph isomorphism network uses learned timestamping, graph-attention READOUT module, and the Transformer encoder to embed the whole graph representation of the dynamic functional connectivity, and makes frequency analysis and spatial region mapping possible with the learned attention values. We perform comparative studies to demonstrate the exceptional performance of the spatio-temporal attention graph isomorphism network, and discuss the spatial and temporal characteristic of the brain based on the attention analyses.