{
  "id": "2301.08028",
  "version": "v1",
  "published": "2023-01-19T12:01:41.000Z",
  "updated": "2023-01-19T12:01:41.000Z",
  "title": "A Survey of Meta-Reinforcement Learning",
  "authors": [
    "Jacob Beck",
    "Risto Vuorio",
    "Evan Zheran Liu",
    "Zheng Xiong",
    "Luisa Zintgraf",
    "Chelsea Finn",
    "Shimon Whiteson"
  ],
  "categories": [
    "cs.LG"
  ],
  "abstract": "While deep reinforcement learning (RL) has fueled multiple high-profile successes in machine learning, it is held back from more widespread adoption by its often poor data efficiency and the limited generality of the policies it produces. A promising approach for alleviating these limitations is to cast the development of better RL algorithms as a machine learning problem itself in a process called meta-RL. Meta-RL is most commonly studied in a problem setting where, given a distribution of tasks, the goal is to learn a policy that is capable of adapting to any new task from the task distribution with as little data as possible. In this survey, we describe the meta-RL problem setting in detail as well as its major variations. We discuss how, at a high level, meta-RL research can be clustered based on the presence of a task distribution and the learning budget available for each individual task. Using these clusters, we then survey meta-RL algorithms and applications. We conclude by presenting the open problems on the path to making meta-RL part of the standard toolbox for a deep RL practitioner.",
  "revisions": [
    {
      "version": "v1",
      "updated": "2023-01-19T12:01:41.000Z"
    }
  ],
  "analyses": {
    "keywords": [
      "meta-reinforcement learning",
      "task distribution",
      "survey meta-rl algorithms",
      "poor data efficiency",
      "deep rl practitioner"
    ],
    "note": {
      "typesetting": "TeX",
      "pages": 0,
      "language": "en",
      "license": "arXiv",
      "status": "editable"
    }
  }
}