{
  "id": "2306.06370",
  "version": "v1",
  "published": "2023-06-10T07:27:00.000Z",
  "updated": "2023-06-10T07:27:00.000Z",
  "title": "AutoSAM: Adapting SAM to Medical Images by Overloading the Prompt Encoder",
  "authors": [
    "Tal Shaharabany",
    "Aviad Dahan",
    "Raja Giryes",
    "Lior Wolf"
  ],
  "categories": [
    "cs.CV"
  ],
  "abstract": "The recently introduced Segment Anything Model (SAM) combines a clever architecture and large quantities of training data to obtain remarkable image segmentation capabilities. However, it fails to reproduce such results for Out-Of-Distribution (OOD) domains such as medical images. Moreover, while SAM is conditioned on either a mask or a set of points, it may be desirable to have a fully automatic solution. In this work, we replace SAM's conditioning with an encoder that operates on the same input image. By adding this encoder and without further fine-tuning SAM, we obtain state-of-the-art results on multiple medical images and video benchmarks. This new encoder is trained via gradients provided by a frozen SAM. For inspecting the knowledge within it, and providing a lightweight segmentation solution, we also learn to decode it into a mask by a shallow deconvolution network.",
  "revisions": [
    {
      "version": "v1",
      "updated": "2023-06-10T07:27:00.000Z"
    }
  ],
  "analyses": {
    "keywords": [
      "medical images",
      "prompt encoder",
      "adapting sam",
      "image segmentation capabilities",
      "lightweight segmentation solution"
    ],
    "note": {
      "typesetting": "TeX",
      "pages": 0,
      "language": "en",
      "license": "arXiv",
      "status": "editable"
    }
  }
}