Learned world models summarize an agent's experience to facilitate learning complex behaviors. While learning world models from high-dimensional sensory inputs is becoming feasible through deep learning, there are many potential ways for deriving behaviors from them. We present Dreamer, a reinforcement learning agent that solves long-horizon tasks from images purely by latent imagination. We efficiently learn behaviors by propagating analytic gradients of learned state values back through trajectories imagined in the compact state space of a learned world model. On 20 challenging visual control tasks, Dreamer exceeds existing approaches in data-efficiency, computation time, and final performance.

One of the most ambitious use cases of computer-assisted learning is to build a recommendation system for lifelong learning. Most recommender algorithms exploit similarities between content and users, overseeing the necessity to leverage sensible learning trajectories for the learner. Lifelong learning thus presents unique challenges, requiring scalable and transparent models that can account for learner knowledge and content novelty simultaneously, while also retaining accurate learners representations for long periods of time. We attempt to build a novel educational recommender, that relies on an integrative approach combining multiple drivers of learners engagement. Our first step towards this goal is TrueLearn, which models content novelty and background knowledge of learners and achieves promising performance while retaining a human interpretable learner model.

One of the most difficult tasks in higher-order theorem proving is the instantiation of set variables [ 2, 3 ]. An important class of theorem proving problems requiring instantia tion of a set variable are those requiring induction [ 5 ]. Instantiating a set variable often requires synthesizing a formula satisfying some properties. In our work we apply machine le arning to the task of synthesizing formulas satisfying a collection of semantic properties . Previous work applying machine learning to induction theorem proving can be found in [ 7 ].

An architecture and a learning procedure where: An agent is made up of many experts All experts share the same communication policy (expert policy), but have different internal memory states There are two levels of learning, an inner loop (with a communication stage) and an outer lo op In ner loop - Agent's behavior and adaptation should emerge as a result of e xperts communicating between each other. Expert s send messag es (of any complexity) to each other and update their internal states based on observations/messages and their internal state fr om the previous time-step. Expert policy is fixed and does not c hange during the inner loop Inner loop loss need not even be a proper loss function. It can be any kind of structured feedback guiding the adaptation during th e age nt's lifetime Outer loop - An expert policy is discovered over generations of agents, ensuring that strategies that find solutions to prob lems in divers e environments can quickly emerge in the inner loop Agent's objective is to adapt fast to novel tasks Exhibiting the following novel properties: Roles of experts and connectivity among them assigned dynamically at inference time Learned communication protocol with context dependent messages of varied complexity Generalizes to different numbers and types of inputs/ou tputs Ca n be trained to handle variations in architecture during bot h training and testing Initial empirical results show generalization and scalability along the spectrum of learning types.

While many methods for learning vector space embeddings have been proposed in the field of Natural Language Processing, these methods typically do not distinguish between categories and individuals. Intuitively, if individuals are represented as vectors, we can think of categories as (soft) regions in the embedding space. Unfortunately, meaningful regions can be difficult to estimate, especially since we often have few examples of individuals that belong to a given category. To address this issue, we rely on the fact that different categories are often highly interdependent. In particular, categories often have conceptual neighbors, which are disjoint from but closely related to the given category (e.g.\ fruit and vegetable). Our hypothesis is that more accurate category representations can be learned by relying on the assumption that the regions representing such conceptual neighbors should be adjacent in the embedding space. We propose a simple method for identifying conceptual neighbors and then show that incorporating these conceptual neighbors indeed leads to more accurate region based representations.

We present SafeLife, a publicly available reinforcement learning environment that tests the safety of reinforcement learning agents. It contains complex, dynamic, tunable, procedurally generated levels with many opportunities for unsafe behavior. Agents are graded both on their ability to maximize their explicit reward and on their ability to operate safely without unnecessary side effects. We train agents to maximize rewards using proximal policy optimization and score them on a suite of benchmark levels. The resulting agents are performant but not safe---they tend to cause large side effects in their environments---but they form a baseline against which future safety research can be measured.

To build AI-based systems that users and the public can justifiably trust one needs to understand how machine learning technologies impact trust put in these services. To guide technology developments, this paper provides a systematic approach to relate social science concepts of trust with the technologies used in AI-based services and products. We conceive trust as discussed in the ABI (Ability, Benevolence, Integrity) framework and use a recently proposed mapping of ABI on qualities of technologies. We consider four categories of machine learning technologies, namely these for Fairness, Explainability, Auditability and Safety (FEAS) and discuss if and how these possess the required qualities. Trust can be impacted throughout the life cycle of AI-based systems, and we introduce the concept of Chain of Trust to discuss technological needs for trust in different stages of the life cycle. FEAS has obvious relations with known frameworks and therefore we relate FEAS to a variety of international Principled AI policy and technology frameworks that have emerged in recent years.

This paper outlines BenchCouncil's view on the challenges, rules, and vision of benchmarking modern workloads like Big Data, AI or machine learning, and Internet Services. We conclude the challenges of benchmarking modern workloads as FIDSS (Fragmented, Isolated, Dynamic, Service-based, and Stochastic), and propose the PRDAERS benchmarking rules that the benchmarks should be specified in a paper-and-pencil manner, relevant, diverse, containing different levels of abstractions, specifying the evaluation metrics and methodology, repeatable, and scaleable. We believe proposing simple but elegant abstractions that help achieve both efficiency and general-purpose is the final target of benchmarking in future, which may be not pressing. In the light of this vision, we shortly discuss BenchCouncil's related projects.

Complex node interactions are common in knowledge graphs, and these interactions also contain rich knowledge information. However, traditional methods usually treat a triple as a training unit during the knowledge representation learning (KRL) procedure, neglecting contextualized information of the nodes in knowledge graphs (KGs). We generalize the modeling object to a very general form, which theoretically supports any subgraph extracted from the knowledge graph, and these subgraphs are fed into a novel transformer-based model to learn the knowledge embeddings. To broaden usage scenarios of knowledge, pre-trained language models are utilized to build a model that incorporates the learned knowledge representations. Experimental results demonstrate that our model achieves the state-of-the-art performance on several medical NLP tasks, and improvement above TransE indicates that our KRL method captures the graph contextualized information effectively.

Dogs have lived alongside humans for at least 40,000 years, but proximity doesn't automatically lead to understanding. According to a new study from the Max Planck Institute for Evolutionary Anthropology, the key to understanding dogs largely depends on where you're from. The researchers, led by Federica Amici, a behavioral ecologist, tested 89 adults and 77 children from distinct cultural backgrounds to test their ability to read the facial expression of dogs. Specifically, subjects were taken from Europe, where dogs are considered close family companions that live indoors alongside humans, and Muslim-majority countries where dogs more commonly live outside and aren't necessarily thought of as surrogate family members. Researchers showed the test subjects photographs of dogs, chimpanzees, and humans and asked them to distinguish expressions of anger, happiness, sadness, fear, and neutral expressions.