Streaming regularization parameter selection via stochastic gradient descent

Ricardo Pio Monti, Romy Lorenz, Robert Leech, Christoforos Anagnostopoulos, Giovanni Montana

Published 2015-11-06Version 1

We propose a framework to perform streaming covariance selection. Our approach employs regularization constraints where a time-varying sparsity parameter is iteratively estimated via stochastic gradient descent. This allows for the regularization parameter to be efficiently learnt in an online manner. The proposed framework is developed for linear regression models and extended to graphical models via neighbourhood selection. We demonstrate the capabilities of such an approach using both synthetic data as well as neuroimaging data.