Time series is the most prevalent form of input data for educational prediction tasks. The vast majority of research using time series data focuses on hand-crafted features, designed by experts for predictive performance and interpretability. However, extracting these features is labor-intensive for humans and computers. In this paper, we propose an approach that utilizes irregular multivariate time series modeling with graph neural networks to achieve comparable or better accuracy with raw time series clickstreams in comparison to hand-crafted features. Furthermore, we extend concept activation vectors for interpretability in raw time series models. We analyze these advances in the education domain, addressing the task of early student performance prediction for downstream targeted interventions and instructional support. Our experimental analysis on 23 MOOCs with millions of combined interactions over six behavioral dimensions show that models designed with our approach can (i) beat state-of-the-art educational time series baselines with no feature extraction and (ii) provide interpretable insights for personalized interventions. Source code: https://github.com/epfl-ml4ed/ripple/.

We tackle the problem of aligning pre-trained large language models (LMs) with human preferences. If we view text generation as a sequential decision-making problem, reinforcement learning (RL) appears to be a natural conceptual framework. However, using RL for LM-based generation faces empirical challenges, including training instability due to the combinatorial action space, as well as a lack of open-source libraries and benchmarks customized for LM alignment. Thus, a question rises in the research community: is RL a practical paradigm for NLP? To help answer this, we first introduce an open-source modular library, RL4LMs (Reinforcement Learning for Language Models), for optimizing language generators with RL. The library consists of on-policy RL algorithms that can be used to train any encoder or encoder-decoder LM in the HuggingFace library (Wolf et al. 2020) with an arbitrary reward function. Next, we present the GRUE (General Reinforced-language Understanding Evaluation) benchmark, a set of 6 language generation tasks which are supervised not by target strings, but by reward functions which capture automated measures of human preference. GRUE is the first leaderboard-style evaluation of RL algorithms for NLP tasks. Finally, we introduce an easy-to-use, performant RL algorithm, NLPO (Natural Language Policy Optimization) that learns to effectively reduce the combinatorial action space in language generation. We show 1) that RL techniques are generally better than supervised methods at aligning LMs to human preferences; and 2) that NLPO exhibits greater stability and performance than previous policy gradient methods (e.g., PPO (Schulman et al. 2017)), based on both automatic and human evaluations.

Markov Decision Process (MDP) presents a mathematical framework to formulate the learning processes of agents in reinforcement learning. MDP is limited by the Markovian assumption that a reward only depends on the immediate state and action. However, a reward sometimes depends on the history of states and actions, which may result in the decision process in a non-Markovian environment. In such environments, agents receive rewards via temporally-extended behaviors sparsely, and the learned policies may be similar. This leads the agents acquired with similar policies generally overfit to the given task and can not quickly adapt to perturbations of environments. To resolve this problem, this paper tries to learn the diverse policies from the history of state-action pairs under a non-Markovian environment, in which a policy dispersion scheme is designed for seeking diverse policy representation. Specifically, we first adopt a transformer-based method to learn policy embeddings. Then, we stack the policy embeddings to construct a dispersion matrix to induce a set of diverse policies. Finally, we prove that if the dispersion matrix is positive definite, the dispersed embeddings can effectively enlarge the disagreements across policies, yielding a diverse expression for the original policy embedding distribution. Experimental results show that this dispersion scheme can obtain more expressive diverse policies, which then derive more robust performance than recent learning baselines under various learning environments.

Way back in 2019--an eon ago in AI time--the New York Times reported an AI milestone: Aristo, a natural-language processing and reasoning system scored over 90% on parts of the New York Regents 8th Grade Science Exam, and over 83% on parts of the corresponding Grade 12 Science Exam. Aristo, the Times proclaimed, "is ready for high school science. I argued this at the time: "The truth is that while these systems perform well on specific language-processing tests, they can only take the test. None come anywhere close to matching humans in reading comprehension or other general abilities that the test was designed to measure." Moreover, such systems lack the basic commonsense understanding of the world that is assumed of humans taking the same tests.

We've joined the AI-hype bandwagon here at the Seattle Search Network. We live and breathe search engine marketing and jump at every Google hiccup, even if we've been okay ignoring Bing all these years. Maybe that's about to change. From random thoughts to early experiments with the tools, there's no shortage of differences of opinion, but that's what keeps us on our toes as digital marketers. We'd love to hear what you think.

Controversial AI tool ChatGPT has already been banned in schools across the world over fears it encourages cheating and laziness. But the International Baccalaureate (IB), which offers an alternative to A-levels, is bucking this trend by permitting the use of ChatGPT to write essays. Students undertaking IB programmes will be able to quote passages generated by the chatbot - as long as they do not try to pass it off as their own words. Created by San Francisco-based company OpenAI, the tool has been trained on a massive amount of text so it can generate human-like responses to questions. A university student has already used ChatGPT to write a 2,000-word essay that got a 2:2 grade, although the lecturer called the language used'fishy'.

Abstract: We investigate the extent to which offline demonstration data can improve online learning. It is natural to expect some improvement, but the question is how, and by how much? We show that the degree of improvement must depend on the quality of the demonstration data. To generate portable insights, we focus on Thompson sampling (TS) applied to a multi-armed bandit as a prototypical online learning algorithm and model. The demonstration data is generated by an expert with a given competence level, a notion we introduce.

Meet Sylvia Christie, our education partnerships manager who's played a leading role in expanding our scholarship programme, which has just celebrated its five-year anniversary. Before DeepMind, I worked for a social purpose startup that increased access to mental healthcare. Then I got a job at a university alongside academics and students. At that point, I realised I was looking for a'Goldilocks' role that brought together everything I loved about these different environments – the speed and excitement of a tech startup, impact-focussed goals, and the fascination of working with brilliant researchers. It seemed impossible to combine all these things.

Abstract: Suppose we are given access to n independent samples from distribution μ and we wish to output one of them with the goal of making the output distributed as close as possible to a target distribution ν. In this work we show that the optimal total variation distance as a function of n is given by Θ (Df′(n)) over the class of all pairs ν,μ with a bounded f-divergence Df(ν μ) D. Previously, this question was studied only for the case when the Radon-Nikodym derivative of ν with respect to μ is uniformly bounded. We then consider an application in the seemingly very different field of smoothed online learning, where we show that recent results on the minimax regret and the regret of oracle-efficient algorithms still hold even under relaxed constraints on the adversary (to have bounded f-divergence, as opposed to bounded Radon-Nikodym derivative). Finally, we also study efficacy of importance sampling for mean estimates uniform over a function class and compare importance sampling with rejection sampling. Abstract: We initiate a study of computable online (c-online) learning, which we analyze under varying requirements for "optimality" in terms of the mistake bound.

Jacques Viljoen 24 February 2023 Integrating artificial intelligence (AI) into education has sparked a dynamic discussion about the potential gains and drawbacks of this technology. Benefits could include more personalised and efficient learning experiences and increased accessibility to information. Challenges may well centre on privacy concerns, ethical considerations and the potential for AI systems to preserve existing biases. Given the complex issues surrounding AI in education, it is crucial to carefully weigh these benefits and challenges to make informed decisions, for instance, about incorporating ChatGPT into the education system. ChatGPT ChatGPT, like any AI language model, has limitations regarding its accuracy and handling of complex or abstract concepts.