In this article, we propose a new framework for addressing multivariate time-varying volatilities. By employing methods of differential geometry, our model respects the geometric structure of the covariance space, i.e., symmetry and positive definiteness, in a way that is independent of any local coordinate parametrization. Its parsimonious specification makes it particularly suitable for large dimensional systems. Simulation studies suggest that our model embraces much of the nonlinear behaviour of the covariance dynamics. Applied to the US and the UK stock markets, the model performs well, especially when applied to risk measurement. In a broad context, our framework presents a new approach treating nonlinear properties observed in the financial market, and numerous areas of application can be further considered.