Self-learning how to swim at low Reynolds number

Alan Cheng Hou Tsang, Pun Wai Tong, Shreyes Nallan, On Shun Pak

Published 2018-08-23Version 1

Synthetic microswimmers show great promise in biomedical applications such as drug delivery and microsurgery. However, their locomotion is subject not only to stringent constraints due to physical laws at microscales but also to uncontrolled environmental factors in realistic biological media. Successful applications of these synthetics are contingent upon their ability to adapt their locomotory gaits across varying biological environments. Here, we present a machine learning framework to design a new class of self-learning, adaptive (or "smart") swimmers at low Reynolds numbers. Unlike the conventional approach of designing synthetic microswimmers, we do not specify any locomotory gaits $\textit{a priori}$ but allow the swimmer to self-learn its own propulsion policy based on its interactions with the surrounding medium via reinforcement learning. We showcase the capabilities of these smart swimmers to identify effective propulsion policies, progressively improve these policies, and adapt their locomotory gaits to traverse media with vastly different properties. Further, these swimmers perform robustly under the influence of random noises, and the learning algorithm is scalable to complex designs with multiple degrees of freedom. We demonstrate these novel features theoretically via a simple reconfigurable system amenable to future experimental implementation. Our studies lay the groundwork for designing the next-generation of smart micro-robots with robust locomotive capabilities.