Statistical Learning
Practical applications of machine-learned flows on gauge fields
Abbott, Ryan, Albergo, Michael S., Boyda, Denis, Hackett, Daniel C., Kanwar, Gurtej, Romero-López, Fernando, Shanahan, Phiala E., Urban, Julian M.
Numerical lattice quantum chromodynamics (QCD) is an integral part of the modern particle and nuclear theory toolkit [1-9]. In this framework, the discretized path integral is computed using Monte Carlo methods. Computationally, this is very expensive, and grows more so as physical limits of interest are approached [10-12]. Consequently, algorithmic developments are an important driver of progress. For example, resolving topological freezing [12-14]--an issue that arises in sampling gauge field configurations with state-of-the-art Markov chain Monte Carlo (MCMC) algorithms like heatbath [15-19] or Hybrid/Hamiltonian Monte Carlo (HMC) [20-22]--would provide access to finer lattice spacings than presently affordable. To such ends, recent work has explored how emerging machine learning (ML) techniques may be applied to lattice QCD [23, 24]. Of particular interest has been the possibility of accelerating gauge-field sampling [25-34] using normalizing flows [35-37], a class of generative statistical models with tractable density functions. In this framework, a flow is a learned, invertible (diffeomorphic) map between gauge fields. Abstractly, flows may be thought of as bridges between different distributions over gauge fields (or, equivalently, different theories or choices of action parameters).
Calibrating Bayesian Learning via Regularization, Confidence Minimization, and Selective Inference
Huang, Jiayi, Park, Sangwoo, Simeone, Osvaldo
The application of artificial intelligence (AI) models in fields such as engineering is limited by the known difficulty of quantifying the reliability of an AI's decision. A well-calibrated AI model must correctly report its accuracy on in-distribution (ID) inputs, while also enabling the detection of out-of-distribution (OOD) inputs. A conventional approach to improve calibration is the application of Bayesian ensembling. However, owing to computational limitations and model misspecification, practical ensembling strategies do not necessarily enhance calibration. This paper proposes an extension of variational inference (VI)-based Bayesian learning that integrates calibration regularization for improved ID performance, confidence minimization for OOD detection, and selective calibration to ensure a synergistic use of calibration regularization and confidence minimization. The scheme is constructed successively by first introducing calibration-regularized Bayesian learning (CBNN), then incorporating out-of-distribution confidence minimization (OCM) to yield CBNN-OCM, and finally integrating also selective calibration to produce selective CBNN-OCM (SCBNN-OCM). Selective calibration rejects inputs for which the calibration performance is expected to be insufficient. Numerical results illustrate the trade-offs between ID accuracy, ID calibration, and OOD calibration attained by both frequentist and Bayesian learning methods. Among the main conclusions, SCBNN-OCM is seen to achieve best ID and OOD performance as compared to existing state-of-the-art approaches at the cost of rejecting a sufficiently large number of inputs.
Inductive Cognitive Diagnosis for Fast Student Learning in Web-Based Online Intelligent Education Systems
Liu, Shuo, Shen, Junhao, Qian, Hong, Zhou, Aimin
Cognitive diagnosis aims to gauge students' mastery levels based on their response logs. Serving as a pivotal module in web-based online intelligent education systems (WOIESs), it plays an upstream and fundamental role in downstream tasks like learning item recommendation and computerized adaptive testing. WOIESs are open learning environment where numerous new students constantly register and complete exercises. In WOIESs, efficient cognitive diagnosis is crucial to fast feedback and accelerating student learning. However, the existing cognitive diagnosis methods always employ intrinsically transductive student-specific embeddings, which become slow and costly due to retraining when dealing with new students who are unseen during training. To this end, this paper proposes an inductive cognitive diagnosis model (ICDM) for fast new students' mastery levels inference in WOIESs. Specifically, in ICDM, we propose a novel student-centered graph (SCG). Rather than inferring mastery levels through updating student-specific embedding, we derive the inductive mastery levels as the aggregated outcomes of students' neighbors in SCG. Namely, SCG enables to shift the task from finding the most suitable student-specific embedding that fits the response logs to finding the most suitable representations for different node types in SCG, and the latter is more efficient since it no longer requires retraining. To obtain this representation, ICDM consists of a construction-aggregation-generation-transformation process to learn the final representation of students, exercises and concepts. Extensive experiments across real-world datasets show that, compared with the existing cognitive diagnosis methods that are always transductive, ICDM is much more faster while maintains the competitive inference performance for new students.
Predictive Model Development to Identify Failed Healing in Patients after Non-Union Fracture Surgery
Donié, Cedric, Reumann, Marie K., Hartung, Tony, Braun, Benedikt J., Histing, Tina, Endo, Satoshi, Hirche, Sandra
Bone non-union is among the most severe complications associated with trauma surgery, occurring in 10-30% of cases after long bone fractures. Treating non-unions requires a high level of surgical expertise and often involves multiple revision surgeries, sometimes even leading to amputation. Thus, more accurate prognosis is crucial for patient well-being. Recent advances in machine learning (ML) hold promise for developing models to predict non-union healing, even when working with smaller datasets, a commonly encountered challenge in clinical domains. To demonstrate the effectiveness of ML in identifying candidates at risk of failed non-union healing, we applied three ML models (logistic regression, support vector machine, and XGBoost) to the clinical dataset TRUFFLE, which includes 797 patients with long bone non-union. The models provided prediction results with 70% sensitivity, and the specificities of 66% (XGBoost), 49% (support vector machine), and 43% (logistic regression). These findings offer valuable clinical insights because they enable early identification of patients at risk of failed non-union healing after the initial surgical revision treatment protocol.
SemEval-2024 Task 1: Semantic Textual Relatedness for African and Asian Languages
Ousidhoum, Nedjma, Muhammad, Shamsuddeen Hassan, Abdalla, Mohamed, Abdulmumin, Idris, Ahmad, Ibrahim Said, Ahuja, Sanchit, Aji, Alham Fikri, Araujo, Vladimir, Beloucif, Meriem, De Kock, Christine, Hourrane, Oumaima, Shrivastava, Manish, Solorio, Thamar, Surange, Nirmal, Vishnubhotla, Krishnapriya, Yimam, Seid Muhie, Mohammad, Saif M.
We present the first shared task on Semantic Textual Relatedness (STR). While earlier shared tasks primarily focused on semantic similarity, we instead investigate the broader phenomenon of semantic relatedness across 14 languages: Afrikaans, Algerian Arabic, Amharic, English, Hausa, Hindi, Indonesian, Kinyarwanda, Marathi, Moroccan Arabic, Modern Standard Arabic, Punjabi, Spanish, and Telugu. These languages originate from five distinct language families and are predominantly spoken in Africa and Asia -- regions characterised by the relatively limited availability of NLP resources. Each instance in the datasets is a sentence pair associated with a score that represents the degree of semantic textual relatedness between the two sentences. Participating systems were asked to rank sentence pairs by their closeness in meaning (i.e., their degree of semantic relatedness) in the 14 languages in three main tracks: (a) supervised, (b) unsupervised, and (c) crosslingual. The task attracted 163 participants. We received 70 submissions in total (across all tasks) from 51 different teams, and 38 system description papers. We report on the best-performing systems as well as the most common and the most effective approaches for the three different tracks.
A Text Classification Framework for Simple and Effective Early Depression Detection Over Social Media Streams
Burdisso, Sergio G., Errecalde, Marcelo, Montes-y-Gómez, Manuel
With the rise of the Internet, there is a growing need to build intelligent systems that are capable of efficiently dealing with early risk detection (ERD) problems on social media, such as early depression detection, early rumor detection or identification of sexual predators. These systems, nowadays mostly based on machine learning techniques, must be able to deal with data streams since users provide their data over time. In addition, these systems must be able to decide when the processed data is sufficient to actually classify users. Moreover, since ERD tasks involve risky decisions by which people's lives could be affected, such systems must also be able to justify their decisions. However, most standard and state-of-the-art supervised machine learning models are not well suited to deal with this scenario. This is due to the fact that they either act as black boxes or do not support incremental classification/learning. In this paper we introduce SS3, a novel supervised learning model for text classification that naturally supports these aspects. SS3 was designed to be used as a general framework to deal with ERD problems. We evaluated our model on the CLEF's eRisk2017 pilot task on early depression detection. Most of the 30 contributions submitted to this competition used state-of-the-art methods. Experimental results show that our classifier was able to outperform these models and standard classifiers, despite being less computationally expensive and having the ability to explain its rationale.
Exploring Key Point Analysis with Pairwise Generation and Graph Partitioning
Li, Xiao, Jiang, Yong, Huang, Shen, Xie, Pengjun, Cheng, Gong, Huang, Fei
Key Point Analysis (KPA), the summarization of multiple arguments into a concise collection of key points, continues to be a significant and unresolved issue within the field of argument mining. Existing models adapt a two-stage pipeline of clustering arguments or generating key points for argument clusters. This approach rely on semantic similarity instead of measuring the existence of shared key points among arguments. Additionally, it only models the intra-cluster relationship among arguments, disregarding the inter-cluster relationship between arguments that do not share key points. To address these limitations, we propose a novel approach for KPA with pairwise generation and graph partitioning. Our objective is to train a generative model that can simultaneously provide a score indicating the presence of shared key point between a pair of arguments and generate the shared key point. Subsequently, to map generated redundant key points to a concise set of key points, we proceed to construct an arguments graph by considering the arguments as vertices, the generated key points as edges, and the scores as edge weights. We then propose a graph partitioning algorithm to partition all arguments sharing the same key points to the same subgraph. Notably, our experimental findings demonstrate that our proposed model surpasses previous models when evaluated on both the ArgKP and QAM datasets.
Prediction of Unmanned Surface Vessel Motion Attitude Based on CEEMDAN-PSO-SVM
Geng, Zhuoya, Chen, Jianmei, Zhu, Wanqiang
Unmanned boats, while navigating at sea, utilize active compensation systems to mitigate wave disturbances experienced by onboard instruments and equipment. However, there exists a lag in the measurement of unmanned boat attitudes, thus introducing unmanned boat motion attitude prediction to compensate for the lag in the signal acquisition process. This paper, based on the basic principles of waves, derives the disturbance patterns of waves on unmanned boats from the wave energy spectrum. Through simulation analysis of unmanned boat motion attitudes, motion attitude data is obtained, providing experimental data for subsequent work. A combined prediction model based on Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (CEEMDAN), Particle Swarm Optimization (PSO), and Support Vector Machine (SVM) is designed to predict the motion attitude of unmanned boats. Simulation results validate its superior prediction accuracy compared to traditional prediction models. For example, in terms of mean absolute error, it improves by 17% compared to the EMD-PSO-SVM model.
Demystifying Legalese: An Automated Approach for Summarizing and Analyzing Overlaps in Privacy Policies and Terms of Service
Soneji, Shikha, Hoesing, Mitchell, Koujalgi, Sujay, Dodge, Jonathan
This calls into question the reasonableness of expecting users to make informed decisions given that they do not comprehend the terms. A technology that can automate the simplification and categorization of popular ToS documents would be immensely beneficial, enhancing user understanding of accepted policies and facilitating the identification of concerning changes. We envision an automated system that begins with the text of a ToS document for a new product or service. The prospective user copies and pastes the text into an automated tool, which extracts key concepts and then presents some information in a format that is shorter and easier to read, such as a numeric/letter score alongside a bullet list of the most important concepts. Our work focuses on extracting key concepts from a data corpus we scraped from Terms of Service; Didn't Read (ToS;DR) [38].
Enhancing Argument Summarization: Prioritizing Exhaustiveness in Key Point Generation and Introducing an Automatic Coverage Evaluation Metric
Khosravani, Mohammad, Huang, Chenyang, Trabelsi, Amine
The proliferation of social media platforms has given rise to the amount of online debates and arguments. Consequently, the need for automatic summarization methods for such debates is imperative, however this area of summarization is rather understudied. The Key Point Analysis (KPA) task formulates argument summarization as representing the summary of a large collection of arguments in the form of concise sentences in bullet-style format, called key points. A sub-task of KPA, called Key Point Generation (KPG), focuses on generating these key points given the arguments. This paper introduces a novel extractive approach for key point generation, that outperforms previous state-of-the-art methods for the task. Our method utilizes an extractive clustering based approach that offers concise, high quality generated key points with higher coverage of reference summaries, and less redundant outputs. In addition, we show that the existing evaluation metrics for summarization such as ROUGE are incapable of differentiating between generated key points of different qualities. To this end, we propose a new evaluation metric for assessing the generated key points by their coverage. Our code can be accessed online.