Search ResultsShowing 1-2 of 2

1. arXiv:2502.04788 (Published 2025-02-07)

   A non-zero-sum game with reinforcement learning under mean-variance framework

   Junyi Guo, Xia Han, Hao Wang, Kam Chuen Yuen
   Categories: math.OC

   Keywords: reinforcement learning, non-zero-sum game, mean-variance framework, gaussian mean return model, time-consistent nash equilibrium strategy
   • Show abstract
   • View PDF
   • Download source
   • Export citation

   In this paper, we investigate a competitive market involving two agents who consider both their own wealth and the wealth gap with their opponent. Both agents can invest in a financial market consisting of a risk-free asset and a risky asset, under conditions where model parameters are partially or completely unknown. This setup gives rise to a non-zero-sum differential game within the framework of reinforcement learning (RL). Each agent aims to maximize his own Choquet-regularized, time-inconsistent mean-variance objective. Adopting the dynamic programming approach, we derive a time-consistent Nash equilibrium strategy in a general incomplete market setting. Under the additional assumption of a Gaussian mean return model, we obtain an explicit analytical solution, which facilitates the development of a practical RL algorithm. Notably, the proposed algorithm achieves uniform convergence, even though the conventional policy improvement theorem does not apply to the equilibrium policy. Numerical experiments demonstrate the robustness and effectiveness of the algorithm, underscoring its potential for practical implementation.

2. arXiv:2306.11035 (Published 2023-06-19)

   Adversarial Training Should Be Cast as a Non-Zero-Sum Game

   Alexander Robey, Fabian Latorre, George J. Pappas, Hamed Hassani, Volkan Cevher
   Categories: cs.LG, math.OC, stat.ML

   Keywords: non-zero-sum game, cases outperforms state-of-the-art attacks, novel non-zero-sum bilevel formulation, simple algorithmic framework, deep neural networks
   • Show abstract
   • View PDF
   • Download source
   • Export citation

   One prominent approach toward resolving the adversarial vulnerability of deep neural networks is the two-player zero-sum paradigm of adversarial training, in which predictors are trained against adversarially-chosen perturbations of data. Despite the promise of this approach, algorithms based on this paradigm have not engendered sufficient levels of robustness, and suffer from pathological behavior like robust overfitting. To understand this shortcoming, we first show that the commonly used surrogate-based relaxation used in adversarial training algorithms voids all guarantees on the robustness of trained classifiers. The identification of this pitfall informs a novel non-zero-sum bilevel formulation of adversarial training, wherein each player optimizes a different objective function. Our formulation naturally yields a simple algorithmic framework that matches and in some cases outperforms state-of-the-art attacks, attains comparable levels of robustness to standard adversarial training algorithms, and does not suffer from robust overfitting.