Bayesian approach to shrinkage estimation for large scale vector autoregressive models

Abstract

A vector autoregressive (VAR) model is a statistical model that describes linear dependencies among vectors of current and previous values for multivariate time series data. Recently, there has been increasing demand to employ high-dimensional VAR models that can process large numbers of time series variables from areas such as systems biology, econometrics, and computational neuroscience. However, when the number of variables is too large compared to the limited length of the time series, computational obstacles, including singular matrix problems and overfitting, are encountered. Several methods have been proposed in literature to handle high-dimensional sparse data problems, but most have limited applications because of heavy computational costs and incorrect assumptions about data. In this thesis we propose a Bayesian approach for modeling VAR processes in order to incorporate proper dependence assumptions and deal with a large dimensionality of data with low computational costs. For the selection of the shrinkage parameter, which is regarded as a prior hyperparameter, we propose a new score function related to the limit of a marginal posterior distribution for the model coefficients. The proposed shrinkage is computationally carried out by using a variation of cross validation. Experimental results based on simulated data demonstrate that the suggested method performs better than the other methods reported in the literature when (1) the number of variables is large and the length of time series is small, or (2) there are strong cross correlations between the time series variables. The proposed method is applied to real world data from systems biology and computational neuroscience. In both cases, the time series data contain limited numbers of observations with relatively large numbers of variables. Once a VAR model is estimated by the proposed method, the model structure that is determined based on nonzero VAR coefficients is discovered by further pruni...