Hyungjoon Soh, Dongyeob Kim, Juno Hwang, Junghyo Jo
Published 2022-11-29Version 1
Mirror descent is a gradient descent method that uses a dual space of parametric models. The great idea has been developed in convex optimization, but not yet widely applied in machine learning. In this study, we provide a possible way that the mirror descent can help data-driven parameter initialization of neural networks. We adopt the Hopfield model as a prototype of neural networks, we demonstrate that the mirror descent can train the model more effectively than the usual gradient descent with random parameter initialization.
On the Power of Over-parametrization in Neural Networks with Quadratic Activation
Measuring abstract reasoning in neural networks
ECG Segmentation by Neural Networks: Errors and Correction
![QR code for arXiv:2211.15880 [cs.LG]](http://oss.arxitics.com/images/favicon.svg)