Education
Online Multi-Agent Forecasting with Interpretable Collaborative Graph Neural Network
Li, Maosen, Chen, Siheng, Shen, Yanning, Liu, Genjia, Tsang, Ivor W., Zhang, Ya
This paper considers predicting future statuses of multiple agents in an online fashion by exploiting dynamic interactions in the system. We propose a novel collaborative prediction unit (CoPU), which aggregates the predictions from multiple collaborative predictors according to a collaborative graph. Each collaborative predictor is trained to predict the status of an agent by considering the impact of another agent. The edge weights of the collaborative graph reflect the importance of each predictor. The collaborative graph is adjusted online by multiplicative update, which can be motivated by minimizing an explicit objective. With this objective, we also conduct regret analysis to indicate that, along with training, our CoPU achieves similar performance with the best individual collaborative predictor in hindsight. This theoretical interpretability distinguishes our method from many other graph networks. To progressively refine predictions, multiple CoPUs are stacked to form a collaborative graph neural network. Extensive experiments are conducted on three tasks: online simulated trajectory prediction, online human motion prediction and online traffic speed prediction, and our methods outperform state-of-the-art works on the three tasks by 28.6%, 17.4% and 21.0% on average, respectively.
Research Mathematical Statistician
BASIC EDUCATION REQUIREMENT for Research Mathematical Statistician, GS-1529: Applicants must meet A or B below to satisfy the basic education requirement for Research Mathematical Statistician at all grade levels. A. Possess a degree that included 24 semester hours of mathematics and statistics, of which at least 12 semester hours were in mathematics and 6 semester hours were in statistics. B. A combination of education and experience -- at least 24 semester hours of mathematics and statistics, including at least 12 hours in mathematics and 6 hours in statistics, as shown in A above, plus appropriate experience or additional education. Courses in mathematical statistics or probability theory with a prerequisite of elementary calculus or more advanced courses will be accepted toward meeting the mathematics requirements, with the provision that the same course cannot be counted toward both the mathematics and the statistics requirement. Evaluation of Experience: The experience offered in combination with educational courses to meet the requirements in paragraph B above should include evidence of statistical work such as (a) sampling, (b) collecting, computing, and analyzing statistical data, and (c) applying known statistical techniques to data such as measurement of central tendency, dispersion, skewness, sampling error, simple and multiple correlation, analysis of variance, and tests of significance.
Developing AI Applications on Azure
This course introduces the concepts of Artificial Intelligence and Machine learning. This course introduces the concepts of Artificial Intelligence and Machine learning. We'll discuss machine learning types and tasks, and machine learning algorithms. You'll explore Python as a popular programming language for machine learning solutions, including using some scientific ecosystem packages which will help you implement machine learning. Next, this course introduces the machine learning tools available in Microsoft Azure.
Grokking self-supervised (representation) learning: how it works in computer vision and why
Self-Supervised Learning (SSL) is a pre-training alternative to transfer learning. Even though SSL emerged from massive NLP datasets, it has also shown significant progress in computer vision. Self-supervised learning in computer vision started from pretext tasks like rotation, jigsaw puzzles or even video ordering. All of these methods were formulating hand-crafted classification problems to generate labels without human annotators. Because many application domains are deprived of human labels. To this end, self-supervised learning is one way to transfer weights. By pretraining your model on labels that are artificially produced from the data.
BRIEF: Everything We Know About 1970s Mainframe RPGs We Can No Longer Play
A PLATO terminal in a museum case at the University of Illinois; photo taken by the author in 2013. This entry summarizes a series of 1970s mainframe games that have been so lost we don't even have screenshots. I also asked several dozen PLATO authors, administrators, and former CRPG Addict contributors--everyone I could find--for any additional recollections about the games. I stopped only when I was confident there was nothing left to learn. If you have any new or conflicting information about any of the games below, I welcome your comments below or an e-mail to crpgaddict@gmail.com. I will update the information below with any new material discovered. However, please do not take it upon yourself to try to track down and contact any of the people listed here on my behalf; it is likely that I have already reached out and they either declined to respond or already told me all they could. Except for Don Daglow's Dungeon, all the games listed below were written in a language called TUTOR for the PLATO educational mainframe hosted by the University of Illinois Urbana-Champaign. Many of the games written on this system have been preserved and are playable today at Cyber1.
Deployment of Machine Learning Models
By the end of the course you will have a comprehensive overview of the entire research, development and deployment lifecycle of a machine learning model, and understood the best coding practices, and things to consider to put a model in production. You will also have a better understanding of the tools available to you to deploy your models, and will be well placed to take the deployment of the models in any direction that serves the needs of your organization. What else should you know? This course will help you take the first steps towards putting your models in production. You will learn how to go from a Jupyter notebook to a fully deployed machine learning model, considering CI/CD, and deploying to cloud platforms and infrastructure. But, there is a lot more to model deployment, like model monitoring, advanced deployment orchestration with Kubernetes, and scheduled workflows with Airflow, as well as various testing paradigms such as shadow deployments that are not covered in this course.
On Bridging Generic and Personalized Federated Learning
Federated learning is promising for its ability to collaboratively train models with multiple clients without accessing their data, but vulnerable when clients' data distributions diverge from each other. This divergence further leads to a dilemma: "Should we prioritize the learned model's generic performance (for future use at the server) or its personalized performance (for each client)?" These two, seemingly competing goals have divided the community to focus on one or the other, yet in this paper we show that it is possible to approach both at the same time. Concretely, we propose a novel federated learning framework that explicitly decouples a model's dual duties with two prediction tasks. On the one hand, we introduce a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them. On the other hand, we formulate the personalized predictor as a lightweight adaptive module that is learned to minimize each client's empirical risk on top of the generic predictor. With this two-loss, two-predictor framework which we name Federated Robust Decoupling Fed-RoD, the learned model can simultaneously achieve state-of-the-art generic and personalized performance, essentially bridging the two tasks.
Impact Remediation: Optimal Interventions to Reduce Inequality
Bynum, Lucius E. J., Loftus, Joshua R., Stoyanovich, Julia
A significant body of research in the data sciences considers unfair discrimination against social categories such as race or gender that could occur or be amplified as a result of algorithmic decisions. Simultaneously, real-world disparities continue to exist, even before algorithmic decisions are made. In this work, we draw on insights from the social sciences and humanistic studies brought into the realm of causal modeling and constrained optimization, and develop a novel algorithmic framework for tackling pre-existing real-world disparities. The purpose of our framework, which we call the "impact remediation framework," is to measure real-world disparities and discover the optimal intervention policies that could help improve equity or access to opportunity for those who are underserved with respect to an outcome of interest. We develop a disaggregated approach to tackling pre-existing disparities that relaxes the typical set of assumptions required for the use of social categories in structural causal models. Our approach flexibly incorporates counterfactuals and is compatible with various ontological assumptions about the nature of social categories. We demonstrate impact remediation with a real-world case study and compare our disaggregated approach to an existing state-of-the-art approach, comparing its structure and resulting policy recommendations. In contrast to most work on optimal policy learning, we explore disparity reduction itself as an objective, explicitly focusing the power of algorithms on reducing inequality.
CLINE: Contrastive Learning with Semantic Negative Examples for Natural Language Understanding
Wang, Dong, Ding, Ning, Li, Piji, Zheng, Hai-Tao
Despite pre-trained language models have proven useful for learning high-quality semantic representations, these models are still vulnerable to simple perturbations. Recent works aimed to improve the robustness of pre-trained models mainly focus on adversarial training from perturbed examples with similar semantics, neglecting the utilization of different or even opposite semantics. Different from the image processing field, the text is discrete and few word substitutions can cause significant semantic changes. To study the impact of semantics caused by small perturbations, we conduct a series of pilot experiments and surprisingly find that adversarial training is useless or even harmful for the model to detect these semantic changes. To address this problem, we propose Contrastive Learning with semantIc Negative Examples (CLINE), which constructs semantic negative examples unsupervised to improve the robustness under semantically adversarial attacking. By comparing with similar and opposite semantic examples, the model can effectively perceive the semantic changes caused by small perturbations. Empirical results show that our approach yields substantial improvements on a range of sentiment analysis, reasoning, and reading comprehension tasks. And CLINE also ensures the compactness within the same semantics and separability across different semantics in sentence-level.