Knowledge tracing (KT) which aims at predicting learner's knowledge mastery plays an important role in the computer-aided educational system. Given learners' exercise records, a knowledge tracing model can trace their hidden knowledge state dynamically. In recent years, many deep learning models have been applied to tackle the KT task, which has shown promising results. However, they still have limitations. Most existing methods simplify the exercising records as knowledge sequence, which fails to explore rich information existed in exercise texts. Besides, the latent hierarchical graph nature of exercises and knowledge remain unexplored. Thus, in this paper, we propose a hierarchical graph knowledge tracing model framework (HGKT) which could leverage the advantages of hierarchical exercise graph and sequence model to enhance the ability of knowledge tracing. Besides, we introduce the concept of problem schema to better represent a group of similar exercises and propose a hierarchical graph neural network to learn representations of problem schemas. Moreover, in the sequence model, we employ two attention mechanisms to highlight important historical states of students. In the testing stage, we present a K&S diagnosis matrix that could trace the transition of mastery of knowledge and problem schema, which could more easily be applied to different applications. Finally, we conduct extensive experiments to evaluate the model on a large scale real-world dataset. The results prove the effectiveness of our model and the diversity of its application scenarios.

Knowledge distillation (KD) is commonly deemed as an effective model compression technique in which a compact model (student) is trained under the supervision of a larger pretrained model or an ensemble of models (teacher). Various techniques have been proposed since the original formulation, which mimic different aspects of the teacher such as the representation space, decision boundary, or intra-data relationship. Some methods replace the one-way knowledge distillation from a static teacher with collaborative learning between a cohort of students. Despite the recent advances, a clear understanding of where knowledge resides in a deep neural network and an optimal method for capturing knowledge from teacher and transferring it to student remains an open question. In this study, we provide an extensive study on nine different KD methods which covers a broad spectrum of approaches to capture and transfer knowledge. We demonstrate the versatility of the KD framework on different datasets and network architectures under varying capacity gaps between the teacher and student. The study provides intuition for the effects of mimicking different aspects of the teacher and derives insights from the performance of the different distillation approaches to guide the design of more effective KD methods. Furthermore, our study shows the effectiveness of the KD framework in learning efficiently under varying severity levels of label noise and class imbalance, consistently providing generalization gains over standard training. We emphasize that the efficacy of KD goes much beyond a model compression technique and it should be considered as a general-purpose training paradigm which offers more robustness to common challenges in the real-world datasets compared to the standard training procedure.

In this paper we provide a general framework for designing, analyzing and implementing such algorithms in the episodic reinforcement learning problem. This framework is built upon Lagrangian duality, and demonstrates that every model-optimistic algorithm that constructs an optimistic MDP has an equivalent representation as a value-optimistic dynamic programming algorithm. Typically, it was thought that these two classes of algorithms were distinct, with model-optimistic algorithms benefiting from a cleaner probabilistic analysis while value-optimistic algorithms are easier to implement and thus more practical. With the framework developed in this paper, we show that it is possible to get the best of both worlds by providing a class of algorithms which have a computationally efficient dynamic-programming implementation and also a simple probabilistic analysis. Besides being able to capture many existing algorithms in the tabular setting, our framework can also address largescale problems under realizable function approximation, where it enables a simple model-based analysis of some recently proposed methods.

We consider an online learning problem where the learner interacts with a Markov decision process in a sequence of episodes, where the reward function is allowed to change between episodes in an adversarial manner and the learner only gets to observe the rewards associated with its actions. We allow the state space to be arbitrarily large, but we assume that all action-value functions can be represented as linear functions in terms of a known low-dimensional feature map, and that the learner has access to a simulator of the environment that allows generating trajectories from the true MDP dynamics. Our main contribution is developing a computationally efficient algorithm that we call MDP-LinExp3, and prove that its regret is bounded by $\widetilde{\mathcal{O}}\big(H^2 T^{2/3} (dK)^{1/3}\big)$, where $T$ is the number of episodes, $H$ is the number of steps in each episode, $K$ is the number of actions, and $d$ is the dimension of the feature map. We also show that the regret can be improved to $\widetilde{\mathcal{O}}\big(H^2 \sqrt{TdK}\big)$ under much stronger assumptions on the MDP dynamics. To our knowledge, MDP-LinExp3 is the first provably efficient algorithm for this problem setting.

In continuing tasks, average-reward reinforcement learning may be a more appropriate problem formulation than the more common discounted reward formulation. As usual, learning an optimal policy in this setting typically requires a large amount of training experiences. Reward shaping is a common approach for incorporating domain knowledge into reinforcement learning in order to speed up convergence to an optimal policy. However, to the best of our knowledge, the theoretical properties of reward shaping have thus far only been established in the discounted setting. This paper presents the first reward shaping framework for average-reward learning and proves that, under standard assumptions, the optimal policy under the original reward function can be recovered. In order to avoid the need for manual construction of the shaping function, we introduce a method for utilizing domain knowledge expressed as a temporal logic formula. The formula is automatically translated to a shaping function that provides additional reward throughout the learning process. We evaluate the proposed method on three continuing tasks. In all cases, shaping speeds up the average-reward learning rate without any reduction in the performance of the learned policy compared to relevant baselines.

Dr. Andrew Ng is a globally recognized leader in artificial intelligence. He was until recently chief scientist at Baidu, where he led the company's approximately 1,300-person AI group and was responsible for driving its global AI strategy and infrastructure. He was also the founding lead of the Google Brain team. In addition, Dr. Ng is co-chairman and cofounder of Coursera, the world's leading MOOC (massive open online course) platform, and an adjunct professor of computer science at Stanford University. He has authored or coauthored over 100 research papers in machine learning, robotics, and related fields.

Digital tools have become one of the only means by which consumers can communicate with their banks and other financial services, even when opening brand new accounts. The pandemic has put trust in remote digital onboarding centre stage. Government benefits, health services, online education, dating companies and gaming are just some of the sectors witnessing a huge surge in demand for digital know-your-customer (KYC) services. This is expanding the use of digital authentication at an unprecedented scale. Unfortunately, at the same time, the outbreak is proving fertile ground for fraudsters looking to exploit this global rise in digital metamorphosis.

Bioengineers have designed a glove that is able to translate American Sign Language to speech in real time. The glove has thin, stretchable sensors inside which run to the fingertips. These sensors are able to pick up and motions and finger placement through electrically conducting yarns. Those sensors are then connected to a small piece of circuit board – approximately the size of a coin – that users wear on their wrists. As such, when people move their hands and fingers to sign words, the glove is able to translate the individual letters, numbers, words and phrases into spoken words.

Artificial intelligence is an oxymoron (Technology quarterly, June 13th). Intelligence is an attribute of living things, and can best be defined as the use of information to further survival and reproduction. When a computer resists being switched off, or a robot worries about the future for its children, then, and only then, may intelligence flow. I acknowledge Richard Sutton's "bitter lesson", that attempts to build human understanding into computers rarely work, although there is nothing new here. I was aware of the folly of anthropomorphism as an AI researcher in the mid-1980s.

This post is part of "AI education", a series of posts that review and explore educational content on data science and machine learning. Mastering machine learning is not easy, even if you're a crack programmer. I've seen many people come from a solid background of writing software in different domains (gaming, web, multimedia, etc.) thinking that adding machine learning to their roster of skills is another walk in the park. And every single one of them has been dismayed. I see two reasons for why the challenges of machine learning are misunderstood. First, as the name suggests, machine learning is software that learns by itself as opposed to being instructed on every single rule by a developer.