Instructional Material
Online machine-learning forecast uncertainty estimation for sequential data assimilation
Sacco, Maximiliano A., Pulido, Manuel, Ruiz, Juan J., Tandeo, Pierre
Quantifying forecast uncertainty is a key aspect of state-of-the-art numerical weather prediction and data assimilation systems. Ensemble-based data assimilation systems incorporate state-dependent uncertainty quantification based on multiple model integrations. However, this approach is demanding in terms of computations and development. In this work a machine learning method is presented based on convolutional neural networks that estimates the state-dependent forecast uncertainty represented by the forecast error covariance matrix using a single dynamical model integration. This is achieved by the use of a loss function that takes into account the fact that the forecast errors are heterodastic. The performance of this approach is examined within a hybrid data assimilation method that combines a Kalman-like analysis update and the machine learning based estimation of a state-dependent forecast error covariance matrix. Observing system simulation experiments are conducted using the Lorenz'96 model as a proof-of-concept. The promising results show that the machine learning method is able to predict precise values of the forecast covariance matrix in relatively high-dimensional states. Moreover, the hybrid data assimilation method shows similar performance to the ensemble Kalman filter outperforming it when the ensembles are relatively small.
What Students Can Learn About Artificial Intelligence -- Recommendations for K-12 Computing Education
Michaeli, Tilman, Seegerer, Stefan, Romeike, Ralf
Technological advances in the context of digital transformation are the basis for rapid developments in the field of artificial intelligence (AI). Although AI is not a new topic in computer science (CS), recent developments are having an immense impact on everyday life and society. In consequence, everyone needs competencies to be able to adequately and competently analyze, discuss and help shape the impact, opportunities, and limits of artificial intelligence on their personal lives and our society. As a result, an increasing number of CS curricula are being extended to include the topic of AI. However, in order to integrate AI into existing CS curricula, what students can and should learn in the context of AI needs to be clarified. This has proven to be particularly difficult, considering that so far CS education research on central concepts and principles of AI lacks sufficient elaboration. Therefore, in this paper, we present a curriculum of learning objectives that addresses digital literacy and the societal perspective in particular. The learning objectives can be used to comprehensively design curricula, but also allow for analyzing current curricula and teaching materials and provide insights into the central concepts and corresponding competencies of AI.
Data, Trees, and Forests -- Decision Tree Learning in K-12 Education
Michaeli, Tilman, Seegerer, Stefan, Kerber, Lennard, Romeike, Ralf
Closely linked to the topic of ML is data science, which is of particular interest for approaches in machine learning and As a consequence of the increasing influence of thus also reflected in multiple AI curricula. Corresponding machine learning on our lives, everyone needs methods are also used to gain knowledge in a wide variety competencies to understand corresponding phenomena, of scientific disciplines. Data analysis and artificial intelligence but also to get involved in shaping our are often referred to as the fourth pillar of science world and making informed decisions regarding (Riedel et al., 2008; Tolle et al., 2011). This is becoming the influences on our society. Therefore, in K-increasingly relevant for K-12 education as well, as this shift 12 education, students need to learn about core in the scientific disciplines is also reflected in corresponding ideas and principles of machine learning.
Towards Scalable Adaptive Learning with Graph Neural Networks and Reinforcement Learning
Vassoyan, Jean, Vie, Jill-Jênn, Lemberger, Pirmin
Adaptive learning is an area of educational technology that consists in delivering personalized learning experiences to address the unique needs of each learner. An important subfield of adaptive learning is learning path personalization: it aims at designing systems that recommend sequences of educational activities to maximize students' learning outcomes. Many machine learning approaches have already demonstrated significant results in a variety of contexts related to learning path personalization. However, most of them were designed for very specific settings and are not very reusable. This is accentuated by the fact that they often rely on non-scalable models, which are unable to integrate new elements after being trained on a specific set of educational resources. In this paper, we introduce a flexible and scalable approach towards the problem of learning path personalization, which we formalize as a reinforcement learning problem. Our model is a sequential recommender system based on a graph neural network, which we evaluate on a population of simulated learners. Our results demonstrate that it can learn to make good recommendations in the small-data regime.
University of Florida offers class examining 'white terror' in Frankenstein, other classic texts
New Jersey parents Christina Balestriere and Kristen Cobo discuss being sued by a school librarian for speaking out against'inappropriate books' on'Jesse Watters Primetime.' The University of Florida offers a class that examines race in the "genre of horror and its trends with a particular focus on representations of racial Otherness and racism," including "white terror" in literary classics, like Frankenstein. As part of the African American Studies class, titled "Black Horror, White Terror," students are instructed to analyze horror books and movies through the lens of "racial identity and oppression" using materials about "the power and horror of whiteness," "black feminism" and "queering personhood," according to a fall 2022 syllabus obtained by The College Fix. "We will also consider the relationship between horror and Black literary modes and traditions focusing on key moments that depict fears of Blackness and/or the terror associated with being Black in America," the syllabus reads. "This course will study the works of Black authors and producers as a way to explore racial identity and oppression."
A Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networks
Ahmadilivani, Mohammad Hasan, Taheri, Mahdi, Raik, Jaan, Daneshtalab, Masoud, Jenihhin, Maksim
Artificial Intelligence (AI) and, in particular, Machine Learning (ML) have emerged to be utilized in various applications due to their capability to learn how to solve complex problems. Over the last decade, rapid advances in ML have presented Deep Neural Networks (DNNs) consisting of a large number of neurons and layers. DNN Hardware Accelerators (DHAs) are leveraged to deploy DNNs in the target applications. Safety-critical applications, where hardware faults/errors would result in catastrophic consequences, also benefit from DHAs. Therefore, the reliability of DNNs is an essential subject of research. In recent years, several studies have been published accordingly to assess the reliability of DNNs. In this regard, various reliability assessment methods have been proposed on a variety of platforms and applications. Hence, there is a need to summarize the state of the art to identify the gaps in the study of the reliability of DNNs. In this work, we conduct a Systematic Literature Review (SLR) on the reliability assessment methods of DNNs to collect relevant research works as much as possible, present a categorization of them, and address the open challenges. Through this SLR, three kinds of methods for reliability assessment of DNNs are identified including Fault Injection (FI), Analytical, and Hybrid methods. Since the majority of works assess the DNN reliability by FI, we characterize different approaches and platforms of the FI method comprehensively. Moreover, Analytical and Hybrid methods are propounded. Thus, different reliability assessment methods for DNNs have been elaborated on their conducted DNN platforms and reliability evaluation metrics. Finally, we highlight the advantages and disadvantages of the identified methods and address the open challenges in the research area.
Logic for Explainable AI
A central quest in explainable AI relates to understanding the decisions made by (learned) classifiers. There are three dimensions of this understanding that have been receiving significant attention in recent years. The first dimension relates to characterizing conditions on instances that are necessary and sufficient for decisions, therefore providing abstractions of instances that can be viewed as the "reasons behind decisions." The next dimension relates to characterizing minimal conditions that are sufficient for a decision, therefore identifying maximal aspects of the instance that are irrelevant to the decision. The last dimension relates to characterizing minimal conditions that are necessary for a decision, therefore identifying minimal perturbations to the instance that yield alternate decisions. We discuss in this tutorial a comprehensive, semantical and computational theory of explainability along these dimensions which is based on some recent developments in symbolic logic. The tutorial will also discuss how this theory is particularly applicable to non-symbolic classifiers such as those based on Bayesian networks, decision trees, random forests and some types of neural networks.
Reinforced MOOCs Concept Recommendation in Heterogeneous Information Networks
Gong, Jibing, Wan, Yao, Liu, Ye, Li, Xuewen, Zhao, Yi, Wang, Cheng, Lin, Yuting, Fang, Xiaohan, Feng, Wenzheng, Zhang, Jingyi, Tang, Jie
Massive open online courses (MOOCs), which offer open access and widespread interactive participation through the internet, are quickly becoming the preferred method for online and remote learning. Several MOOC platforms offer the service of course recommendation to users, to improve the learning experience of users. Despite the usefulness of this service, we consider that recommending courses to users directly may neglect their varying degrees of expertise. To mitigate this gap, we examine an interesting problem of concept recommendation in this paper, which can be viewed as recommending knowledge to users in a fine-grained way. We put forward a novel approach, termed HinCRec-RL, for Concept Recommendation in MOOCs, which is based on Heterogeneous Information Networks and Reinforcement Learning. In particular, we propose to shape the problem of concept recommendation within a reinforcement learning framework to characterize the dynamic interaction between users and knowledge concepts in MOOCs. Furthermore, we propose to form the interactions among users, courses, videos, and concepts into a heterogeneous information network (HIN) to learn the semantic user representations better. We then employ an attentional graph neural network to represent the users in the HIN, based on meta-paths. Extensive experiments are conducted on a real-world dataset collected from a Chinese MOOC platform, XuetangX, to validate the efficacy of our proposed HinCRec-RL. Experimental results and analysis demonstrate that our proposed HinCRec-RL performs well when comparing with several state-of-the-art models.
Lifelong Learning on Evolving Graphs Under the Constraints of Imbalanced Classes and New Classes
Galke, Lukas, Vagliano, Iacopo, Franke, Benedikt, Zielke, Tobias, Hoffmann, Marcel, Scherp, Ansgar
Lifelong graph learning deals with the problem of continually adapting graph neural network (GNN) models to changes in evolving graphs. We address two critical challenges of lifelong graph learning in this work: dealing with new classes and tackling imbalanced class distributions. The combination of these two challenges is particularly relevant since newly emerging classes typically resemble only a tiny fraction of the data, adding to the already skewed class distribution. We make several contributions: First, we show that the amount of unlabeled data does not influence the results, which is an essential prerequisite for lifelong learning on a sequence of tasks. Second, we experiment with different label rates and show that our methods can perform well with only a tiny fraction of annotated nodes. Third, we propose the gDOC method to detect new classes under the constraint of having an imbalanced class distribution. The critical ingredient is a weighted binary cross-entropy loss function to account for the class imbalance. Moreover, we demonstrate combinations of gDOC with various base GNN models such as GraphSAGE, Simplified Graph Convolution, and Graph Attention Networks. Lastly, our k-neighborhood time difference measure provably normalizes the temporal changes across different graph datasets. With extensive experimentation, we find that the proposed gDOC method is consistently better than a naive adaption of DOC to graphs. Specifically, in experiments using the smallest history size, the out-of-distribution detection score of gDOC is 0.09 compared to 0.01 for DOC. Furthermore, gDOC achieves an Open-F1 score, a combined measure of in-distribution classification and out-of-distribution detection, of 0.33 compared to 0.25 of DOC (32% increase).
Code Execution with Pre-trained Language Models
Liu, Chenxiao, Lu, Shuai, Chen, Weizhu, Jiang, Daxin, Svyatkovskiy, Alexey, Fu, Shengyu, Sundaresan, Neel, Duan, Nan
Code execution is a fundamental aspect of programming language semantics that reflects the exact behavior of the code. However, most pre-trained models for code intelligence ignore the execution trace and only rely on source code and syntactic structures. In this paper, we investigate how well pre-trained models can understand and perform code execution. We develop a mutation-based data augmentation technique to create a large-scale and realistic Python dataset and task for code execution, which challenges existing models such as Codex. We then present CodeExecutor, a Transformer model that leverages code execution pre-training and curriculum learning to enhance its semantic comprehension. We evaluate CodeExecutor on code execution and show its promising performance and limitations. We also demonstrate its potential benefits for code intelligence tasks such as zero-shot code-to-code search and text-to-code generation. Our analysis provides insights into the learning and generalization abilities of pre-trained models for code execution.