Deep learning on graphs has contributed to breakthroughs in biology [1, 2], chemistry [3, 4], physics [5, 6], and the social sciences [7]. The predominant use of graph neural networks [8] is to learn representations of various graph components--such as nodes, edges, subgraphs, and entire graphs--based on neural message passing strategies. The learned representations are used for downstream tasks, including label prediction via semi-supervised learning [9], self-supervised learning [10], and graph design and generation [11, 12]. In most existing applications, datasets explicitly describe graphs in the form of nodes, edges, and additional information representing contextual knowledge, such as node, edge, and graph attributes. Modeling complex systems requires measurements that describe the same objects from different perspectives, at different scales, or through multiple modalities, such as images, sensor readings, language sequences, and compact mathematical statements. Multimodal learning [13] studies how such heterogeneous, complex descriptors can be optimized to create learning systems that are broadly generalizable, robust to changes in the underlying data distributions, and can train more with less labeled data.

Federated learning is a distributed and privacy-preserving approach to train a statistical model collaboratively from decentralized data of different parties. However, when datasets of participants are not independent and identically distributed (non-IID), models trained by naive federated algorithms may be biased towards certain participants, and model performance across participants is non-uniform. This is known as the fairness problem in federated learning. In this paper, we formulate fairness-controlled federated learning as a dynamical multi-objective optimization problem to ensure fair performance across all participants. To solve the problem efficiently, we study the convergence and bias of Adam as the server optimizer in federated learning, and propose Adaptive Federated Adam (AdaFedAdam) to accelerate fair federated learning with alleviated bias. We validated the effectiveness, Pareto optimality and robustness of AdaFedAdam in numerical experiments and show that AdaFedAdam outperforms existing algorithms, providing better convergence and fairness properties of the federated scheme.

The Winograd Schema Challenge - a set of twin sentences involving pronoun reference disambiguation that seem to require the use of commonsense knowledge - was proposed by Hector Levesque in 2011. By 2019, a number of AI systems, based on large pre-trained transformer-based language models and fine-tuned on these kinds of problems, achieved better than 90% accuracy. In this paper, we review the history of the Winograd Schema Challenge and discuss the lasting contributions of the flurry of research that has taken place on the WSC in the last decade. We discuss the significance of various datasets developed for WSC, and the research community's deeper understanding of the role of surrogate tasks in assessing the intelligence of an AI system.

Heart sound auscultation has been demonstrated to be beneficial in clinical usage for early screening of cardiovascular diseases. Due to the high requirement of well-trained professionals for auscultation, automatic auscultation benefiting from signal processing and machine learning can help auxiliary diagnosis and reduce the burdens of training professional clinicians. Nevertheless, classic machine learning is limited to performance improvement in the era of big data. Deep learning has achieved better performance than classic machine learning in many research fields, as it employs more complex model architectures with stronger capability of extracting effective representations. Deep learning has been successfully applied to heart sound analysis in the past years. As most review works about heart sound analysis were given before 2017, the present survey is the first to work on a comprehensive overview to summarise papers on heart sound analysis with deep learning in the past six years 2017--2022. We introduce both classic machine learning and deep learning for comparison, and further offer insights about the advances and future research directions in deep learning for heart sound analysis.

The field of natural language processing has reached breakthroughs with the advent of transformers. They have remained state-of-the-art since then, and there also has been much research in analyzing, interpreting, and evaluating the attention layers and the underlying embedding space. In addition to the self-attention layers, the feed-forward layers in the transformer are a prominent architectural component. From extensive research, we observe that its role is under-explored. We focus on the latent space, known as the Activation Space, that consists of the neuron activations from these feed-forward layers. In this survey paper, we review interpretability methods that examine the learnings that occurred in this activation space. Since there exists only limited research in this direction, we conduct a detailed examination of each work and point out potential future directions of research. We hope our work provides a step towards strengthening activation space analysis.

Self-adaptive software can assess and modify its behavior when the assessment indicates that the program is not performing as intended or when improved functionality or performance is available. Since the mid-1960s, the subject of system adaptivity has been extensively researched, and during the last decade, many application areas and technologies involving self-adaptation have gained prominence. All of these efforts have in common the introduction of self-adaptability through software. Thus, it is essential to investigate systematic software engineering methods to create self-adaptive systems that may be used across different domains. The primary objective of this research is to summarize current advances in awareness requirements for adaptive strategies based on an examination of state-of-the-art methods described in the literature. This paper presents a review of self-adaptive systems in the context of requirement awareness and summarizes the most common methodologies applied. At first glance, it gives a review of the previous surveys and works about self-adaptive systems. Afterward, it classifies the current self-adaptive systems based on six criteria. Then, it presents and evaluates the most common self-adaptive approaches. Lastly, an evaluation among the self-adaptive models is conducted based on four concepts (requirements description, monitoring, relationship, dependency/impact, and tools).

Quality education, one of the seventeen sustainable development goals (SDGs) identified by the United Nations General Assembly, stands to benefit enormously from the adoption of artificial intelligence (AI) driven tools and technologies. The concurrent boom of necessary infrastructure, digitized data and general social awareness has propelled massive research and development efforts in the artificial intelligence for education (AIEd) sector. In this review article, we investigate how artificial intelligence, machine learning and deep learning methods are being utilized to support students, educators and administrative staff. We do this through the lens of a novel categorization approach. We consider the involvement of AI-driven methods in the education process in its entirety - from students admissions, course scheduling etc. in the proactive planning phase to knowledge delivery, performance assessment etc. in the reactive execution phase. We outline and analyze the major research directions under proactive and reactive engagement of AI in education using a representative group of 194 original research articles published in the past two decades i.e., 2003 - 2022. We discuss the paradigm shifts in the solution approaches proposed, i.e., in the choice of data and algorithms used over this time. We further dive into how the COVID-19 pandemic challenged and reshaped the education landscape at the fag end of this time period. Finally, we pinpoint existing limitations in adopting artificial intelligence for education and reflect on the path forward.

Over the last several years use cases for robotics based solutions have diversified from factory floors to domestic applications. In parallel, Deep Learning approaches are replacing traditional techniques in Computer Vision, Natural Language Processing, Speech processing etc. and are delivering robust results. Our goal is to survey a number of research internship projects in the broad area of "Deep Learning as applied to Robotics" and present a concise view for the benefit of aspiring student interns. In this paper, we survey the research work done by Robotic Institute Summer Scholars (RISS), CMU. We particularly focus on papers that use deep learning to solve core robotic problems and also robotic solutions. We trust this would be useful particularly for internship aspirants for the Robotics Institute, CMU.

We present the Verse library with the aim of making hybrid system verification more usable for multi-agent scenarios. In Verse, decision making agents move in a map and interact with each other through sensors. The decision logic for each agent is written in a subset of Python and the continuous dynamics is given by a black-box simulator. Multiple agents can be instantiated and they can be ported to different maps for creating scenarios. Verse provides functions for simulating and verifying such scenarios using existing reachability analysis algorithms. We illustrate several capabilities and use cases of the library with heterogeneous agents, incremental verification, different sensor models, and the flexibility of plugging in different subroutines for post computations.

In today's multilingual lexical databases, the majority of the world's languages are under-represented. Beyond a mere issue of resource incompleteness, we show that existing lexical databases have structural limitations that result in a reduced expressivity on culturally-specific words and in mapping them across languages. In particular, the lexical meaning space of dominant languages, such as English, is represented more accurately while linguistically or culturally diverse languages are mapped in an approximate manner. Our paper assesses state-of-the-art multilingual lexical databases and evaluates their strengths and limitations with respect to their expressivity on lexical phenomena of linguistic diversity.