{
  "id": "2207.12738",
  "version": "v1",
  "published": "2022-07-26T08:43:36.000Z",
  "updated": "2022-07-26T08:43:36.000Z",
  "title": "Quantitative propagation of chaos for mean field Markov decision process with common noise",
  "authors": [
    "Médéric Motte",
    "Huyên Pham"
  ],
  "categories": [
    "math.OC",
    "math.PR"
  ],
  "abstract": "We investigate propagation of chaos for mean field Markov Decision Process with common noise (CMKV-MDP), and when the optimization is performed over randomized open-loop controls on infinite horizon. We first state a rate of convergence of order $M_N^\\gamma$, where $M_N$ is the mean rate of convergence in Wasserstein distance of the empirical measure, and $\\gamma \\in (0,1]$ is an explicit constant, in the limit of the value functions of $N$-agent control problem with asymmetric open-loop controls, towards the value function of CMKV-MDP. Furthermore, we show how to explicitly construct $(\\epsilon+\\mathcal{O}(M_N^\\gamma))$-optimal policies for the $N$-agent model from $\\epsilon$-optimal policies for the CMKV-MDP. Our approach relies on sharp comparison between the Bellman operators in the $N$-agent problem and the CMKV-MDP, and fine coupling of empirical measures.",
  "revisions": [
    {
      "version": "v1",
      "updated": "2022-07-26T08:43:36.000Z"
    }
  ],
  "analyses": {
    "keywords": [
      "mean field markov decision process",
      "common noise",
      "quantitative propagation",
      "open-loop controls",
      "optimal policies"
    ],
    "note": {
      "typesetting": "TeX",
      "pages": 0,
      "language": "en",
      "license": "arXiv",
      "status": "editable"
    }
  }
}