Proximal Algorithms in Statistics and Machine Learning

Nicholas G. Polson, James G. Scott, Brandon T. Willard

Published 2015-02-11Version 1

In this paper we develop proximal methods for statistical learning. Proximal point algorithms are useful for optimisation in machine learning and statistics for obtaining solutions with composite objective functions. Our approach exploits a generalised Moreau envelope and closed form solutions of proximal operators to develop novel proximal algorithms. We illustrate our methodology with regularized logistic and poisson regression and provide solutions for non-convex bridge penalties and fused lasso norms. We also provide a survey of convergence of non-descent algorithms with acceleration. Finally, we provide directions for future research.