Education
Can Learning Deteriorate Control? Analyzing Computational Delays in Gaussian Process-Based Event-Triggered Online Learning
Dai, Xiaobing, Lederer, Armin, Yang, Zewen, Hirche, Sandra
When the dynamics of systems are unknown, supervised machine learning techniques are commonly employed to infer models from data. Gaussian process (GP) regression is a particularly popular learning method for this purpose due to the existence of prediction error bounds. Moreover, GP models can be efficiently updated online, such that event-triggered online learning strategies can be pursued to ensure specified tracking accuracies. However, existing trigger conditions must be able to be evaluated at arbitrary times, which cannot be achieved in practice due to non-negligible computation times. Therefore, we first derive a delay-aware tracking error bound, which reveals an accuracy-delay trade-off. Based on this result, we propose a novel event trigger for GP-based online learning with computational delays, which we show to offer advantages over offline trained GP models for sufficiently small computation times. Finally, we demonstrate the effectiveness of the proposed event trigger for online learning in simulations.
Origami-inspired robots can sense, analyze and act in challenging environments
However, the rigid computer chips traditionally needed to enable advanced robot capabilities -- sensing, analyzing and responding to the environment -- add extra weight to the thin sheet materials and makes them harder to fold. The semiconductor-based components therefore have to be added after a robot has taken its final shape. Now, a multidisciplinary team led by researchers at the UCLA Samueli School of Engineering has created a new fabrication technique for fully foldable robots that can perform a variety of complex tasks without relying on semiconductors. A study detailing the research findings was published in Nature Communications. By embedding flexible and electrically conductive materials into a pre-cut, thin polyester film sheet, the researchers created a system of information-processing units, or transistors, which can be integrated with sensors and actuators.
DAC-MR: Data Augmentation Consistency Based Meta-Regularization for Meta-Learning
Shu, Jun, Yuan, Xiang, Meng, Deyu, Xu, Zongben
Abstract--Meta learning recently has been heavily researched and helped advance the contemporary machine learning. However, achieving well-performing meta-learning model requires a large amount of training tasks with high-quality meta-data representing the underlying task generalization goal, which is sometimes difficult and expensive to obtain for real applications. Current meta-data-driven meta-learning approaches, however, are fairly hard to train satisfactory meta-models with imperfect training tasks. To address this issue, we suggest a meta-knowledge informed meta-learning (MKIML) framework to improve meta-learning by additionally integrating compensated meta-knowledge into meta-learning process. We preliminarily integrate meta-knowledge into meta-objective via using an appropriate meta-regularization (MR) objective to regularize capacity complexity of the meta-model function class to facilitate better generalization on unseen tasks. As a practical implementation, we introduce data augmentation consistency to encode invariance as meta-knowledge for instantiating MR objective, denoted by DAC-MR. The proposed DAC-MR is hopeful to learn well-performing meta-models from training tasks with noisy, sparse or unavailable meta-data. We theoretically demonstrate that DAC-MR can be treated as a proxy meta-objective used to evaluate meta-model without high-quality meta-data. Besides, meta-data-driven meta-loss objective combined with DAC-MR is capable of achieving better meta-level generalization. Fine performance of DAC-MR are obtained across all settings, and are well-aligned with our theoretical insights. This implies that our DAC-MR is problem-agnostic, and hopeful to be readily applied to extensive meta-learning problems and tasks.
Pivotal Role of Language Modeling in Recommender Systems: Enriching Task-specific and Task-agnostic Representation Learning
Shin, Kyuyong, Kwak, Hanock, Kim, Wonjae, Jeong, Jisu, Jung, Seungjae, Kim, Kyung-Min, Ha, Jung-Woo, Lee, Sang-Woo
Recent studies have proposed unified user modeling frameworks that leverage user behavior data from various applications. Many of them benefit from utilizing users' behavior sequences as plain texts, representing rich information in any domain or system without losing generality. Hence, a question arises: Can language modeling for user history corpus help improve recommender systems? While its versatile usability has been widely investigated in many domains, its applications to recommender systems still remain underexplored. We show that language modeling applied directly to task-specific user histories achieves excellent results on diverse recommendation tasks. Also, leveraging additional task-agnostic user histories delivers significant performance benefits. We further demonstrate that our approach can provide promising transfer learning capabilities for a broad spectrum of real-world recommender systems, even on unseen domains and services.
Towards Mechatronics Approach of System Design, Verification and Validation for Autonomous Vehicles
Samak, Chinmay Vilas, Samak, Tanmay Vilas, Krovi, Venkat
Modern-day autonomous vehicles are increasingly becoming complex multidisciplinary systems composed of mechanical, electrical, electronic, computing and information sub-systems. Furthermore, the individual constituent technologies employed for developing autonomous vehicles have started maturing up to a point, where it seems beneficial to start looking at the synergistic integration of these components into sub-systems, systems, and potentially, system-of-systems. Hence, this work applies the principles of mechatronics approach of system design, verification and validation for the development of autonomous vehicles. Particularly, we discuss leveraging multidisciplinary co-design practices along with virtual, hybrid and physical prototyping and testing within a concurrent engineering framework to develop and validate a scaled autonomous vehicle using the AutoDRIVE ecosystem. We also describe a case-study of autonomous parking application using a modular probabilistic framework to illustrate the benefits of the proposed approach.
data2vec-aqc: Search for the right Teaching Assistant in the Teacher-Student training setup
Lodagala, Vasista Sai, Ghosh, Sreyan, Umesh, S.
In this paper, we propose a new Self-Supervised Learning (SSL) algorithm called data2vec-aqc, for speech representation learning from unlabeled speech data. Our goal is to improve SSL for speech in domains where both unlabeled and labeled data are limited. Building on the recently introduced data2vec, we introduce additional modules to the data2vec framework that leverage the benefit of data augmentations, quantized representations, and clustering. The interaction between these modules helps solve the cross-contrastive loss as an additional self-supervised objective. data2vec-aqc achieves up to 14.1% and 20.9% relative WER improvement over the existing state-of-the-art data2vec system over the test-clean and test-other sets, respectively of LibriSpeech, without the use of any language model (LM). Our proposed model also achieves up to 17.8\% relative WER gains over the baseline data2vec when fine-tuned on a subset of the Switchboard dataset. Code: https://github.com/Speech-Lab-IITM/data2vec-aqc.
Natural Language Reasoning, A Survey
Yu, Fei, Zhang, Hongbo, Tiwari, Prayag, Wang, Benyou
This survey paper proposes a clearer view of natural language reasoning in the field of Natural Language Processing (NLP), both conceptually and practically. Conceptually, we provide a distinct definition for natural language reasoning in NLP, based on both philosophy and NLP scenarios, discuss what types of tasks require reasoning, and introduce a taxonomy of reasoning. Practically, we conduct a comprehensive literature review on natural language reasoning in NLP, mainly covering classical logical reasoning, natural language inference, multi-hop question answering, and commonsense reasoning. The paper also identifies and views backward reasoning, a powerful paradigm for multi-step reasoning, and introduces defeasible reasoning as one of the most important future directions in natural language reasoning research. We focus on single-modality unstructured natural language text, excluding neuro-symbolic techniques and mathematical reasoning.
Scalable Educational Question Generation with Pre-trained Language Models
Bulathwela, Sahan, Muse, Hamze, Yilmaz, Emine
The automatic generation of educational questions will play a key role in scaling online education, enabling self-assessment at scale when a global population is manoeuvring their personalised learning journeys. We develop \textit{EduQG}, a novel educational question generation model built by adapting a large language model. Our extensive experiments demonstrate that \textit{EduQG} can produce superior educational questions by further pre-training and fine-tuning a pre-trained language model on the scientific text and science question data.
'Why would we employ people?' Experts on five ways AI will change work
In 1965, the political scientist and Nobel laureate Herbert Simon declared: "Machines will be capable, within 20 years, of doing any work a man can do." Today, in what is increasingly referred to as the fourth industrial revolution, the arrival of artificial intelligence (AI) in the workplace is igniting similar concerns. The European parliament's forthcoming Artificial Intelligence Act is likely to deem the use of AI across education, law enforcement and worker management to be "high risk". Geoffrey Hinton, known as the "godfather of AI", recently resigned from his position at Google, citing concerns about the technology's impact on the job market. And, in early May, striking members of the Writers Guild of America promised executives: "AI will replace you before it replaces us."
Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy
Smith, Michael J., Geach, James E.
In this review, we explore the historical development and future prospects of artificial intelligence (AI) and deep learning in astronomy. We trace the evolution of connectionism in astronomy through its three waves, from the early use of multilayer perceptrons, to the rise of convolutional and recurrent neural networks, and finally to the current era of unsupervised and generative deep learning methods. With the exponential growth of astronomical data, deep learning techniques offer an unprecedented opportunity to uncover valuable insights and tackle previously intractable problems. As we enter the anticipated fourth wave of astronomical connectionism, we argue for the adoption of GPT-like foundation models fine-tuned for astronomical applications. Such models could harness the wealth of high-quality, multimodal astronomical data to serve state-of-the-art downstream tasks. To keep pace with advancements driven by Big Tech, we propose a collaborative, open-source approach within the astronomy community to develop and maintain these foundation models, fostering a symbiotic relationship between AI and astronomy that capitalizes on the unique strengths of both fields.