Training an agent to achieve particular goals or perform desired behaviors is often accomplished through reinforcement learning, especially in the absence of expert demonstrations. However, supporting novel goals or behaviors through reinforcement learning requires the ad-hoc design of appropriate reward functions, which quickly becomes intractable. To address this challenge, we propose Text-Aware Diffusion for Policy Learning (TADPoLe), which uses a pretrained, frozen text-conditioned diffusion model to compute dense zero-shot reward signals for text-aligned policy learning. We hypothesize that large-scale pretrained generative models encode rich priors that can supervise a policy to behave not only in a text-aligned manner, but also in alignment with a notion of naturalness summarized from internet-scale training data. In our experiments, we demonstrate that TADPoLe is able to learn policies for novel goal-achievement and continuous locomotion behaviors specified by natural language, in both Humanoid and Dog environments. The behaviors are learned zero-shot without ground-truth rewards or expert demonstrations, and are qualitatively more natural according to human evaluation. We further show that TADPoLe performs competitively when applied to robotic manipulation tasks in the Meta-World environment, without having access to any in-domain demonstrations.

They're born as tiny'toadlets.' Breakthroughs, discoveries, and DIY tips sent every weekday. A frog's lifecycle is likely one of the earliest bits of science that many of us remember learning. They start as eggs, hatch into tadpoles, and soon grow into the recognizable adult amphibians. While that remains true for the vast majority of the planet's nearly 8,000 known frog species, a handful of the amphibians have evolved a more streamlined reproductive process.

Training an agent to achieve particular goals or perform desired behaviors is often accomplished through reinforcement learning, especially in the absence of expert demonstrations. However, supporting novel goals or behaviors through reinforcement learning requires the ad-hoc design of appropriate reward functions, which quickly becomes intractable. To address this challenge, we propose Text-Aware Diffusion for Policy Learning (TADPoLe), which uses a pretrained, frozen text-conditioned diffusion model to compute dense zero-shot reward signals for text-aligned policy learning. We hypothesize that large-scale pretrained generative models encode rich priors that can supervise a policy to behave not only in a text-aligned manner, but also in alignment with a notion of naturalness summarized from internet-scale training data. In our experiments, we demonstrate that TADPoLe is able to learn policies for novel goal-achievement and continuous locomotion behaviors specified by natural language, in both Humanoid and Dog environments.

Training an agent to achieve particular goals or perform desired behaviors is often accomplished through reinforcement learning, especially in the absence of expert demonstrations. However, supporting novel goals or behaviors through reinforcement learning requires the ad-hoc design of appropriate reward functions, which quickly becomes intractable. To address this challenge, we propose Text-Aware Diffusion for Policy Learning (TADPoLe), which uses a pretrained, frozen text-conditioned diffusion model to compute dense zero-shot reward signals for text-aligned policy learning. We hypothesize that large-scale pretrained generative models encode rich priors that can supervise a policy to behave not only in a text-aligned manner, but also in alignment with a notion of naturalness summarized from internet-scale training data. In our experiments, we demonstrate that TADPoLe is able to learn policies for novel goal-achievement and continuous locomotion behaviors specified by natural language, in both Humanoid and Dog environments. The behaviors are learned zero-shot without ground-truth rewards or expert demonstrations, and are qualitatively more natural according to human evaluation. We further show that TADPoLe performs competitively when applied to robotic manipulation tasks in the Meta-World environment.

Pre-training has shown success in different areas of machine learning, such as Computer Vision, Natural Language Processing (NLP), and medical imaging. However, it has not been fully explored for clinical data analysis. An immense amount of clinical records are recorded, but still, data and labels can be scarce for data collected in small hospitals or dealing with rare diseases. In such scenarios, pre-training on a larger set of unlabelled clinical data could improve performance. In this paper, we propose novel unsupervised pre-training techniques designed for heterogeneous, multi-modal clinical data for patient outcome prediction inspired by masked language modeling (MLM), by leveraging graph deep learning over population graphs. To this end, we further propose a graph-transformer-based network, designed to handle heterogeneous clinical data. By combining masking-based pre-training with a transformer-based network, we translate the success of masking-based pre-training in other domains to heterogeneous clinical data. We show the benefit of our pre-training method in a self-supervised and a transfer learning setting, utilizing three medical datasets TADPOLE, MIMIC-III, and a Sepsis Prediction Dataset. We find that our proposed pre-training methods help in modeling the data at a patient and population level and improve performance in different fine-tuning tasks on all datasets.

We present the design and operation of tadpole-mimetic robots prepared for a study of the parenting behaviors of poison frogs, which pair bond and raise their offspring. The mission of these robots is to convince poison frog parents that they are tadpoles, which need to be fed. Tadpoles indicate this need, at least in part, by wriggling with a characteristic frequency and amplitude. While the study is in progress, preliminary indications are that the TadBots have passed their test, at least for father frogs. We discuss the design and operational requirements for producing convincing TadBots and provide some details of the study design and plans for future work.

To fight the invasive fish, the international team, composed of biologists and engineers from Australia, the U.S., and Italy, turned to its natural predator -- the largemouth bass (Micropterus salmoides) -- for inspiration. They crafted a robotic fish that mimics the appearance and simulates the movements of the real predator. Aided by computer vision, the robot strikes when it spots the mosquitofish approaching tadpoles of an Australian species (Litoria moorei), which is threatened by mosquitofish in the wild. Scared and stressed, the mosquitofish showed fearful behaviors and experienced weight loss, changes in body shape, and a reduction in fertility, all of which impair their survival and reproduction. "Mosquitofish is one of the 100 world's worst invasive species, and current methods to eradicate it are too expensive and time-consuming to effectively contrast its spread," says first author Giovanni Polverino (@GioPolverino) of the University of Western Australia.

Robotic fish might help solve an ecological problem by scaring an invasive fish species so profoundly that it is put off breeding. Eastern mosquitofish (Gambusia holbrooki) were introduced in many parts of the world to eat mosquito larvae and keep the disease-spreading insects under control. But they have had a negative and unintended consequence on local fauna: they chew the tails of native freshwater fish and tadpoles, then leave them to die. Reducing numbers of eastern mosquitofish without harming other wildlife is a difficult prospect, but Giovanni Polverino at the University of Western Australia and his colleagues have come up with a potential solution. They designed a robotic version of the largemouth bass (Micropterus salmoides), which naturally preys on mosquitofish.

Life finds a way, and the same goes for even robots, according to a group of scientists who say the first living robotic life forms can reproduce. In January 2020, a team of scientists from the University of Vermont, Tufts University and Harvard University took stem cells from African clawed frog embryos and formed them into tiny living creatures called xenobots. The xenobots, which are less than 0.04 inches wide, were able to move on their own, communicate amongst each other and heal themselves from an injury, making them the first-ever living robots. But over one year later, the computer-designed creatures have begun to do "something that's never been observed before." What the team of scientists discovered was the xenobots would move around their environment and find single cells.