A Deep Reinforcement Learning Approach to Efficient Distributed Optimization

Daokuan Zhu, Jie Lu

Published 2023-11-15Version 1

In distributed optimization, the practical problem-solving performance is essentially sensitive to algorithm selection, parameter setting, problem type and data pattern. Thus, it is often laborious to acquire a highly efficient method for a given specific problem. In this paper, we propose a learning-based method to achieve efficient distributed optimization over networked systems. Specifically, a deep reinforcement learning (DRL) framework is developed for adaptive configuration within a parameterized unifying algorithmic form, which incorporates an abundance of first-order and second-order optimization algorithms that can be implemented in a decentralized fashion. We exploit the local consensus and objective information to represent the regularities of problem instances and trace the solving progress, which constitute the states observed by an RL agent. The framework is trained using Proximal Policy Optimization (PPO) on a number of practical problem instances of similar structures yet different problem data. Experiments on various smooth and non-smooth classes of objective functions demonstrate that our proposed learning-based method outperforms several state-of-the-art distributed optimization algorithms in terms of convergence speed and solution accuracy.