Bacterial cells use run-and-tumble motion to climb up attractant concentration gradient in their environment. By extending the uphill runs and shortening the downhill runs the cells migrate towards the higher attractant zones. Motivated by this, we formulate a reinforcement learning (RL) algorithm where an agent moves in one dimension in the presence of an attractant gradient. The agent can perform two actions: either persistent motion in the same direction or reversal of direction. We assign costs for these actions based on the recent history of the agent's trajectory. We ask the question: which RL strategy works best in different types of attractant environment. We quantify efficiency of the RL strategy by the ability of the agent (a) to localize in the favorable zones after large times, and (b) to learn about its complete environment. Depending on the attractant profile and the initial condition, we find an optimum balance is needed between exploration and exploitation to ensure the most efficient performance.

Artificial Intelligence (AI) has witnessed significant advances with the emergence of powerful neural network (NN) models. Examples include large language models [1] and image generation models such as DALL-E [2], Imagen [3], and Parti [4]. Each has achieved previously unseen capabilities as powerful individuals through recent technical breakthroughs. On the other hand, the biological evolutionary strategy focuses more on the direction of collective intelligence compared to individual ability, especially for species living in populations [5]. Unlike individual intelligence, which deals with challenges independently, collective intelligence necessitates the ability to process information, operate in a decentralized manner, and adaptively integrate information based on context. This distinction is evident in social insects, such as ants and bees, where collective behavior with role differentiation emerges not from highly complex individuals but through simple interactions among members.

Single-cell organisms and various cell types use a range of motility modes when following a chemical gradient, but it is unclear which mode is best suited for different gradients. Here, we model directional decision-making in chemotactic amoeboid cells as a stimulus-dependent actin recruitment contest. Pseudopods extending from the cell body compete for a finite actin pool to push the cell in their direction until one pseudopod wins and determines the direction of movement. Our minimal model provides a quantitative understanding of the strategies cells use to reach the physical limit of accurate chemotaxis, aligning with data without explicit gradient sensing or cellular memory for persistence. To generalize our model, we employ reinforcement learning optimization to study the effect of pseudopod suppression, a simple but effective cellular algorithm by which cells can suppress possible directions of movement. Different pseudopod-based chemotaxis strategies emerge naturally depending on the environment and its dynamics. For instance, in static gradients, cells can react faster at the cost of pseudopod accuracy, which is particularly useful in noisy, shallow gradients where it paradoxically increases chemotactic accuracy. In contrast, in dynamics gradients, cells form \textit{de novo} pseudopods. Overall, our work demonstrates mechanical intelligence for high chemotaxis performance with minimal cellular regulation.

We investigate the boundary between chemotaxis driven by spatial estimation of gradients and chemotaxis driven by temporal estimation. While it is well known that spatial chemotaxis becomes disadvantageous for small organisms at high noise levels, it is unclear whether there is a discontinuous switch of optimal strategies or a continuous transition exists. Here, we employ deep reinforcement learning to study the possible integration of spatial and temporal information in an a priori unconstrained manner. We parameterize such a combined chemotactic policy by a recurrent neural network and evaluate it using a minimal theoretical model of a chemotactic cell. By comparing with constrained variants of the policy, we show that it converges to purely temporal and spatial strategies at small and large cell sizes, respectively. We find that the transition between the regimes is continuous, with the combined strategy outperforming in the transition region both the constrained variants as well as models that explicitly integrate spatial and temporal information. Finally, by utilizing the attribution method of integrated gradients, we show that the policy relies on a non-trivial combination of spatially and temporally derived gradient information in a ratio that varies dynamically during the chemotactic trajectories.

We train recurrent networks to control chemotaxis in a computer model of the nematode C. elegans. The model presented is based closely on the body mechanics, behavioral analyses, neuroanatomy and neurophysiology of C. elegans, each imposing constraints rel(cid:173) evant for information processing. Simulated worms moving au(cid:173) tonomously in simulated chemical environments display a variety of chemotaxis strategies similar to those of biological worms.

By imitating biological microswimmers, microrobots can be designed to accomplish targeted delivery of cargos and biomedical manipulations at microscale. However, it is still a great challenge to enable microrobots to maneuver in a complex environment. Machine learning algorithms offer a tool to boost mobility and flexibility of a synthetic microswimmer, hence could help us design truly smart microrobots. In this work, we investigate how a model of sea urchin sperm cell can self-learn chemotactic motion in a chemoattractant concentration field. We employ an artificial neural network to act as a decision-making agent and facilitate the sperm cell to discover efficient maneuver strategies through a deep reinforcement learning (DRL) algorithm. Our results show that chemotactic behaviours, very similar to the realistic ones, can be achieved by the DRL utilizing only limited environmental information. In most cases, the DRL algorithm discovers more efficient strategies than the human-devised one. Furthermore, the DRL can even utilize an external disturbance to facilitate the chemotactic motion if the extra flow information is also taken into account by the artificial neural network. Our results provide insights to the chemotactic process of sea urchin sperm cells and also prepare guidance for the intelligent maneuver of microrobots.

The coupled problem of hydrodynamics and solute transport for the Najafi-Golestanian three-sphere swimmer is studied, with the Reynolds number set to zero and P\'eclet numbers (Pe) ranging from 0.06 to 60. The adopted method is the numerical simulation of the problem with a finite element code based upon the FEniCS library. For the swimmer executing the optimal locomotion gait, we report the Sherwood number as a function of Pe in homogeneous fluids and confirm that little gain in solute flux is achieved by swimming unless Pe is significantly larger than 10. We also consider the swimmer as an learning agent moving inside a fluid that has a concentration gradient. The outcomes of Q-learning processes show that learning locomotion (with the displacement as reward) is significantly easier than learning chemotaxis (with the increase of solute flux as reward). The chemotaxis problem, even at low Pe, has a varying environment that renders learning more difficult. Further, the learning difficulty increases severely with the P\'eclet number. The results demonstrate the challenges that natural and artificial swimmers need to overcome to migrate efficiently when exposed to chemical inhomogeneities.

We train recurrent networks to control chemotaxis in a computer model of the nematode C. elegans. The model presented is based closely on the body mechanics, behavioral analyses, neuroanatomy and neurophysiology of C. elegans, each imposing constraints relevant for information processing. Simulated worms moving autonomously in simulated chemical environments display a variety of chemotaxis strategies similar to those of biological worms. 1 INTRODUCTION The nematode C. elegans provides a unique opportunity to study the neuronal basis of neural computation in an animal capable of complex goal-oriented behaviors. The adult hermaphrodite is only 1 mm long, and has exactly 302 neurons and 95 muscle cells. The morphology of every cell and the location of most electrical and chemical synapses are known precisely (White et al., 1986), making C. elegans especially attractive for study.

We train recurrent networks to control chemotaxis in a computer model of the nematode C. elegans. The model presented is based closely on the body mechanics, behavioral analyses, neuroanatomy and neurophysiology of C. elegans, each imposing constraints relevant for information processing. Simulated worms moving autonomously in simulated chemical environments display a variety of chemotaxis strategies similar to those of biological worms. 1 INTRODUCTION The nematode C. elegans provides a unique opportunity to study the neuronal basis of neural computation in an animal capable of complex goal-oriented behaviors. The adult hermaphrodite is only 1 mm long, and has exactly 302 neurons and 95 muscle cells. The morphology of every cell and the location of most electrical and chemical synapses are known precisely (White et al., 1986), making C. elegans especially attractive for study.

Thomas C. Ferree, Ben A. Marcotte, Shawn R. Lockery Institute of Neuroscience, University of Oregon, Eugene, Oregon 97403 Abstract We train recurrent networks to control chemotaxis in a computer model of the nematode C. elegans. The model presented is based closely on the body mechanics, behavioral analyses, neuroanatomy and neurophysiology of C. elegans, each imposing constraints relevant forinformation processing. Simulated worms moving autonomously insimulated chemical environments display a variety of chemotaxis strategies similar to those of biological worms. 1 INTRODUCTION The nematode C. elegans provides a unique opportunity to study the neuronal basis ofneural computation in an animal capable of complex goal-oriented behaviors. The adult hermaphrodite is only 1 mm long, and has exactly 302 neurons and 95 muscle cells. The morphology of every cell and the location of most electrical and chemical synapses are known precisely (White et al., 1986), making C. elegans especially attractivefor study. Whole-cell recordings are now being made on identified neurons in the nerve ring of C. elegans to determine electrophysiological properties which underly information processing in this animal (Lockery and Goodman, unpublished).