The proofs of these propositions are extended from Berlekamp (1968). Note that both oracle's preference feedback and We adopt the environment setting created by Rothfuss et al. (2019). MuJoCo locomotion tasks, where the reward function are varied to create a multi-task setting. The training and testing tasks are randomly generated by a fixed random seed. During meta-training, the meta-RL algorithm has the full access to the environmental interaction.

Emerging Artificial Intelligence of Things (AIoT) applications desire online prediction using deep neural network (DNN) models on mobile devices. However, due to the movement of devices, unfamiliar test samples constantly appear, significantly affecting the prediction accuracy of a pre-trained DNN. In addition, unstable network connection calls for local model inference. In this paper, we propose a light-weight scheme, called Anole, to cope with the local DNN model inference on mobile devices. The core idea of Anole is to first establish an army of compact DNN models, and then adaptively select the model fitting the current test sample best for online inference. The key is to automatically identify model-friendly scenes for training scene-specific DNN models. To this end, we design a weakly-supervised scene representation learning algorithm by combining both human heuristics and feature similarity in separating scenes. Moreover, we further train a model classifier to predict the best-fit scene-specific DNN model for each test sample. We implement Anole on different types of mobile devices and conduct extensive trace-driven and real-world experiments based on unmanned aerial vehicles (UAVs). The results demonstrate that Anole outwits the method of using a versatile large DNN in terms of prediction accuracy (4.5% higher), response time (33.1% faster) and power consumption (45.1% lower).

Learning new task-specific skills from a few trials is a fundamental challenge for artificial intelligence. Meta reinforcement learning (meta-RL) tackles this problem by learning transferable policies that support few-shot adaptation to unseen tasks. Despite recent advances in meta-RL, most existing methods require the access to the environmental reward function of new tasks to infer the task objective, which is not realistic in many practical applications. To bridge this gap, we study the problem of few-shot adaptation in the context of human-in-the-loop reinforcement learning. We develop a meta-RL algorithm that enables fast policy adaptation with preference-based feedback. The agent can adapt to new tasks by querying human's preference between behavior trajectories instead of using per-step numeric rewards. By extending techniques from information theory, our approach can design query sequences to maximize the information gain from human interactions while tolerating the inherent error of non-expert human oracle. In experiments, we extensively evaluate our method, Adaptation with Noisy OracLE (ANOLE), on a variety of meta-RL benchmark tasks and demonstrate substantial improvement over baseline algorithms in terms of both feedback efficiency and error tolerance.

Scientists usually think of natural selection as a slow process, unfolding over generations of incremental change. But, as a study published today in Nature suggests, sometimes this system can take a more rapid approach, especially after a sudden event like a hurricane. As these disasters become more frequent thanks to anthropogenic climate change, understanding how hurricanes affect the species who live in the places they make landfall is vital. This study, which was mainly the result of good timing, offers evidence that, for one family of lizards at least, hurricanes may initiate a rapid natural selection process for certain traits. Just four days before Hurricane Irma reached the Turks and Caicos in 2017, ecologist Colin Donihue completed a survey of the local anole species (Anolis scriptus, a family of lizards) on two remote islands.

Natural selection shapes behavior as well as morphology -- but it can have different effects in males and females! Assessment of risk-taking behaviors of the Bahaman brown anole (Anolis sagrei): Exploratory behavior is favored in the absence of predators, whereas avoidance of the ground is favored in their presence. Scientific research long ago established that natural selection can change an animal's physical appearance. For example, animals with longer limbs are faster runners whereas animals with shorter limbs are more efficient climbers. But does natural selection shape an animal's behavior, too?