Continual learning aims to learn new tasks without forgetting previously learned ones. This is especially challenging when one cannot access data from previous tasks and when the model has a fixed capacity. Current regularization-based continual learning algorithms need an external representation and extra computation to measure the parameters' importance. In contrast, we propose Uncertainty-guided Continual Bayesian Neural Networks (UCB), where the learning rate adapts according to the uncertainty defined in the probability distribution of the weights in networks. Uncertainty is a natural way to identify what to remember and what to change as we continually learn, allowing to mitigate catastrophic forgetting. We also show a variant of our model, which uses uncertainty for weight pruning and retains task performance after pruning by saving binary masks per tasks. We evaluate our UCB approach extensively on diverse object classification datasets with short and long sequences of tasks and report superior or on-par performance compared to existing approaches. Additionally, we show that our model does not necessarily need task information at test time, i.e. it does not presume knowledge of which task a sample belongs to.

Before the meeting, the head of state was told about the academic process at School 21 and had a brief conversation with students. The President was informed about the school by Head of Sberbank German Gref and school Principal Svetlana Infimovskaya. The students of the school can study the following areas: Algorithms, Graphics, Mobile Development, Computer Security, Robot Technology, and Artificial Intelligence to name a few. The school has 940 students today. On average, students are expected to study for 2–3.5 years. The course includes two practical training sessions in relevant companies for six months or more. Today I suggest that we discuss concrete steps that will form the foundation for our National Strategy on the development of artificial intelligence technologies. We have repeatedly spoken about the need for such a comprehensive document. I also mentioned it in this year's Address to the Federal Assembly. This is indeed one of the key areas of technological development that determines and will continue to determine the future of the entire world. The artificial intelligence mechanisms will allow for quick real-time decision-making based on analysing vast amounts of information known as big data, which provides tremendous advantages in terms of quality and performance. In addition, such mechanisms are unparalleled in history in terms of their impact on the economy and productivity, the effectiveness of management, education, healthcare and daily life. However, vying for technological leadership, primarily, in the sphere of artificial intelligence – and you are all very well aware of this, colleagues – has already lead to global competition. New products and solutions are being created at an exponential growth rate. I have said it before and I will say it now: he who can establish a monopoly in artificial intelligence – we are aware of the consequences – will rule the world. It is no accident that many developed countries of the world have already adopted action plans to develop such technologies. Of course, we must ensure technological sovereignty in the realm of artificial intelligence. This is the most important prerequisite for the viability of our businesses and the economy, the quality of life for Russian citizens, security and, finally, our defence capability. Here, we are not just talking about algorithms for addressing individual and highly specialised problems; what we need are universal solutions, the use of which gives the optimum effect in any industry. In order to achieve such an ambitious goal in AI technology, we are objectively positioned to have a good start and we have a serious competitive edge. Today, Russia boasts one of the world's highest penetration rates for mobile communications and internet access, as well as for the development of electronic services.

MONTREAL, June 4, 2019 /CNW Telbec/ - Today's students are graduating into a world that is in a significant state of transformation due to developments in Artificial Intelligence (AI) and related information technologies. "Our current and future students are the ones who will be facing these challenges and opportunities when they enter university or the work force. We are committed to offering new, updated and upgraded classes and learning opportunities to help prepare our students adequately," said Richard Filion, Director General of Dawson College. As a sign of that commitment to its students, Dawson College has decided to make an investment of over a million dollars in a comprehensive Artificial Intelligence initiative, the largest investment in an AI project by a cégep in Quebec. Today the College announced that a three-year strategic plan for the academic years 2019-2022 has been adopted and that $1,050,000 has been budgeted for its implementation, hoping to establish Dawson College as the centre of excellence in AI in college education.

FREMONT, CA: Machine learning (ML) has showcased unprecedented growth and benefits in firms that have strategically implemented it to leverage its essence to the fullest, especially in e-learning. Traditional corporate and institutional learning programs are not up to the level of the modern business landscape. To broaden the talent pool, keep employees up-to-speed on the latest technical skills, and efficiently shape a leading workforce, companies need to think out-of-the-box about possible methods of enhancing development and training the workforce. In a scenario with a shortage of highly skilled workers and a surplus of mid-low skilled workers, companies focus on training the workers available in excess, so that the ML talent gap is not widened further. The IT talent pool can be enriched by drawing employees with less experience in data science analytics and providing them with internal learning.

In this thesis, we draw inspiration from both classical system identification and modern machine learning in order to solve estimation problems for real-world, physical systems. The main approach to estimation and learning adopted is optimization based. Concepts such as regularization will be utilized for encoding of prior knowledge and basis-function expansions will be used to add nonlinear modeling power while keeping data requirements practical. The thesis covers a wide range of applications, many inspired by applications within robotics, but also extending outside this already wide field. Usage of the proposed methods and algorithms are in many cases illustrated in the real-world applications that motivated the research. Topics covered include dynamics modeling and estimation, model-based reinforcement learning, spectral estimation, friction modeling and state estimation and calibration in robotic machining. In the work on modeling and identification of dynamics, we develop regularization strategies that allow us to incorporate prior domain knowledge into flexible, overparameterized models. We make use of classical control theory to gain insight into training and regularization while using flexible tools from modern deep learning. A particular focus of the work is to allow use of modern methods in scenarios where gathering data is associated with a high cost. In the robotics-inspired parts of the thesis, we develop methods that are practically motivated and ensure that they are implementable also outside the research setting. We demonstrate this by performing experiments in realistic settings and providing open-source implementations of all proposed methods and algorithms.

Identifying statistical dependence between the features and the label is a fundamental problem in supervised learning. This paper presents a framework for estimating dependence between numerical features and a categorical label using generalized Gini distance, an energy distance in reproducing kernel Hilbert spaces (RKHS). Two Gini distance based dependence measures are explored: Gini distance covariance and Gini distance correlation. Unlike Pearson covariance and correlation, which do not characterize independence, the above Gini distance based measures define dependence as well as independence of random variables. The test statistics are simple to calculate and do not require probability density estimation. Uniform convergence bounds and asymptotic bounds are derived for the test statistics. Comparisons with distance covariance statistics are provided. It is shown that Gini distance statistics converge faster than distance covariance statistics in the uniform convergence bounds, hence tighter upper bounds on both Type I and Type II errors. Moreover, the probability of Gini distance covariance statistic under-performing the distance covariance statistic in Type II error decreases to 0 exponentially with the increase of the sample size. Extensive experimental results are presented to demonstrate the performance of the proposed method.

We study the problem of inverse reinforcement learning (IRL) with the added twist that the learner is assisted by a helpful teacher. More formally, we tackle the following algorithmic question: How could a teacher provide an informative sequence of demonstrations to an IRL learner to speed up the learning process? We present an interactive teaching framework where a teacher adaptively chooses the next demonstration based on learner's current policy. In particular, we design teaching algorithms for two concrete settings: an omniscient setting where a teacher has full knowledge about the learner's dynamics and a blackbox setting where the teacher has minimal knowledge. Then, we study a sequential variant of the popular MCE-IRL learner and prove convergence guarantees of our teaching algorithm in the omniscient setting. Extensive experiments with a car driving simulator environment show that the learning progress can be speeded up drastically as compared to an uninformative teacher.

The multi-armed bandit (MAB) problem is one of the most fundamental problems in online learning, with diverse applications ranging from pricing and online advertising to clinical trails. Over the past several decades, it has been a very active research area spanning different disciplines, including computer science, operations research, statistics and economics. In a traditional multi-armed bandit problem, the learner (i.e., decision-maker) is allowed to switch freely between actions, and an effective learning policy may incur frequent switching -- indeed, the learner's task is to balance the exploration-exploitation tradeoff, and both exploration (i.e., acquiring new information) and exploitation (i.e., optimizing decisions based on up-to-date information) require switching. However, in many real-world scenarios, it is costly to switch between different alternatives, and a learning policy with limited switching behavior is preferred. The learner thus has to consider the cost of switching in her learning task.

Many machine learning problems reduce to the problem of minimizing an expected risk, defined as the sum of a large number of, often convex, component functions. Iterative gradient methods are popular techniques for the above problems. However, they are in general slow to converge, in particular for large data sets. In this work, we develop analysis for selecting a subset (or sketch) of training data points with their corresponding learning rates in order to provide faster convergence to a close neighbordhood of the optimal solution. We show that subsets that minimize the upper-bound on the estimation error of the full gradient, maximize a submodular facility location function. As a result, by greedily maximizing the facility location function we obtain subsets that yield faster convergence to a close neighborhood of the optimum solution. We demonstrate the real-world effectiveness of our algorithm, SIG, confirming our analysis, through an extensive set of experiments on several applications, including logistic regression and training neural networks. We also include a method that provides a deliberate deterministic ordering of the data subset that is quite effective in practice. We observe that our method, while achieving practically the same loss, speeds up gradient methods by up to 10x for convex and 3x for non-convex (deep) functions.

Differential Private Learning and Online Learning are two well-studied areas in machine learning. While at a first glance these two subjects may seem disparate, recent works gathered a growing amount of evidence which suggests otherwise. For example, Adaptive Data Analysis [15, 14, 24, 19, 3] shares strong similarities with adversarial frameworks studied in online learning, and on the other hand exploits ideas and tools from differential privacy. A more formal relation between private and online learning is manifested by the following fact: Every privately learnable class is online learnable. This implication and variants of it were derived by several recent works [20, 9, 1] (see the related work section for more details). One caveat of the latter results is that they are non-constructive: they show that every privately learnable class has a finite Littlestone dimension. Then, since the Littlestone dimension is known to capture online learnability [26, 5], it follows that privately learnable classes are indeed online learnable. Consequently, the implied online learner is not necessarily efficient, even if the assumed private learner is.