Policy gradient reinforcement learning (RL) algorithms have achieved impressive performance in challenging learning tasks such as continuous control, but suffer from high sample complexity. Experience replay is a commonly used approach to improve sample efficiency, but gradient estimators using past trajectories typically have high variance. Existing sampling strategies for experience replay like uniform sampling or prioritised experience replay do not explicitly try to control the variance of the gradient estimates. In this paper, we propose an online learning algorithm, adaptive experience selection (AES), to adaptively learn an experience sampling distribution that explicitly minimises this variance. Using a regret minimisation approach, AES iteratively updates the experience sampling distribution to match the performance of a competitor distribution assumed to have optimal variance. Sample non-stationarity is addressed by proposing a dynamic (i.e. time changing) competitor distribution for which a closed-form solution is proposed. We demonstrate that AES is a low-regret algorithm with reasonable sample complexity. Empirically, AES has been implemented for deep deterministic policy gradient and soft actor critic algorithms, and tested on 8 continuous control tasks from the OpenAI Gym library. Ours results show that AES leads to significantly improved performance compared to currently available experience sampling strategies for policy gradient.

We study an online learning problem subject to the constraint of individual fairness, which requires that similar individuals are treated similarly. Unlike prior work on individual fairness, we do not assume the similarity measure among individuals is known, nor do we assume that such measure takes a certain parametric form. Instead, we leverage the existence of an auditor who detects fairness violations without enunciating the quantitative measure. In each round, the auditor examines the learner's decisions and attempts to identify a pair of individuals that are treated unfairly by the learner. We provide a general reduction framework that reduces online classification in our model to standard online classification, which allows us to leverage existing online learning algorithms to achieve sub-linear regret and number of fairness violations. Surprisingly, in the stochastic setting where the data are drawn independently from a distribution, we are also able to establish PAC-style fairness and accuracy generalization guarantees (Yona and Rothblum [2018]), despite only having access to a very restricted form of fairness feedback. Our fairness generalization bound qualitatively matches the uniform convergence bound of Yona and Rothblum [2018], while also providing a meaningful accuracy generalization guarantee. Our results resolve an open question by Gillen et al. [2018] by showing that online learning under an unknown individual fairness constraint is possible even without assuming a strong parametric form of the underlying similarity measure.

Abstract reasoning refers to the ability to analyze information, discover rules at an intangible level, and solve problems in innovative ways. Raven's Progressive Matrices (RPM) test is typically used to examine the capability of abstract reasoning. In the test, the subject is asked to identify the correct choice from the answer set to fill the missing panel at the bottom right of RPM (e.g., a 3$\times$3 matrix), following the underlying rules inside the matrix. Recent studies, taking advantage of Convolutional Neural Networks (CNNs), have achieved encouraging progress to accomplish the RPM test problems. Unfortunately, simply relying on the relation extraction at the matrix level, they fail to recognize the complex attribute patterns inside or across rows/columns of RPM. To address this problem, in this paper we propose a Hierarchical Rule Induction Network (HriNet), by intimating human induction strategies. HriNet extracts multiple granularity rule embeddings at different levels and integrates them through a gated embedding fusion module. We further introduce a rule similarity metric based on the embeddings, so that HriNet can not only be trained using a tuplet loss but also infer the best answer according to the similarity score. To comprehensively evaluate HriNet, we first fix the defects contained in the very recent RAVEN dataset and generate a new one named Balanced-RAVEN. Then extensive experiments are conducted on the large-scale dataset PGM and our Balanced-RAVEN, the results of which show that HriNet outperforms the state-of-the-art models by a large margin.

To prepare all students with the creative, collaborative and digital problem-solving skills of the future, schools must teach computer science as part of the core curriculum. Computer science is not just about coding. It is also about computational thinking, interface design, data analysis, machine learning, cybersecurity, networking and robotics. Learning computer science encourages creativity, problem-solving, ethics and collaboration – skills which aren't just important for technical careers in the developed world, but valuable for every career in all economies. What's more, in a study of how students felt about their classes, computer science and engineering trailed only the arts in terms of classes they liked the most.

Here on Codementor I usually see lots of students and developers trying to get into Machine Learning confused with complicated topics they are facing at the very beginning of their journey. I want to make a deep yet understandable introduction to the algorithm which is so simple and elegant that you would like it. If you are a Machine Learning engineer but have a limited understanding of this one, it could be also useful to read it. I was working as a software developer for years and everyone around me was talking about this brand new data science and machine learning thing (I understood that there is nothing new on this planet later), so I've decided to take masters studies in the University to get known to it. Our first module was a general introductory course to Data Science and I remember myself sitting and trying to understand what's going on.

I recently asked the Twitter community about their biggest machine learning pain points and what work their teams plan to focus on in 2020. One of the most frequently mentioned pain points was deploying machine learning models. More specifically, "How do you deploy machine learning models in an automated, reproducible, and auditable manner?" The topic of ML deployment is rarely discussed when machine learning is taught. Boot camps, data science graduate programs, and online courses tend to focus on training algorithms and neural network architectures because these are "core" machine learning ideas.

So we don't have to construct the actual algorithms used to process data and train models. However, there's still a level of math that you have to grapple with when dabbling in Machine Learning. You need to first be able to process data to pass into ML algorithms and models. You also need to have some knowledge of ML framework settings and configuration. Most of the work done by data scientists is involved in preparing the data.

The Open University is the UK's largest university, a world leader in flexible part-time education combining a mission to widen access to higher education with research excellence, transforming lives through education. We are seeking to appoint a Lecturer or Senior Lecturer in Artificial Intelligence to join the School of Languages and Applied Linguistics. The candidate will work in one or more of the topics of central importance to the School of Languages and Applied Linguistics, with special attention to workplace communication and/or the interface between social media and AI. The successful candidate will have a master's degree or equivalent in linguistics or a related field, at Senior Lecturer level you will also have a PHD or equivalent in interactional linguistics, preferably ethnomethodological conversation analysis and/or interactional linguistics. Evidence of an emerging research profile in AI and human communication is also essential for this post.

Shell has a broader strategy to embed AI across its operations, a move that has helped the oil giant lower costs and avoid downtime. Other oil-and-gas companies that have tapped AI to improve operations and reduce costs include Exxon Mobil Corp., BP PLC and Chevron Corp. "Artificial intelligence enables us to process the vast quantity of data across our businesses to generate new insights which can keep us ahead of the competition," said Yuri Sebregts, Shell's chief technology officer, in an email. The initiative at Shell expands a 2019 yearlong pilot program with Udacity, based in Mountain View, Calif., that included about 250 Shell data scientists and software engineers. They picked up AI skills such as reinforcement learning, a type of machine learning where algorithms learn the correct way to perform an action based on trial-and-error and observations. Shell employees could use AI expertise, for example, to better predict equipment failures and automatically identify areas within a facility to reduce carbon emissions, said Dan Jeavons, Shell's general manager of data science.

I have a Mechanical Engineering (ME) background as all of my degrees are in ME. After my Bachelor, I was doing my higher education in the field of Robotics when the Data revolution took shape. People were more familiar with the word "Big Data" at the time rather than Data Science. Then, I got hooked up with Machine Learning and started steering my career path towards Data Science since. I had a bit of good start with a Robotics background, especially in programming, so I didn't have to start from scratch.