South America
Image-based Novel Fault Detection with Deep Learning Classifiers using Hierarchical Labels
Sergin, Nurettin, Huang, Jiayu, Chang, Tzyy-Shuh, Yan, Hao
Many manufacturing systems are instrumented with image-sensing systems to monitor process performance and product quality. The low cost and rich information of the image-based sensing systems have led to high-dimensional data streams that provide distinctive opportunities for performance improvement. Among these, accurate process monitoring and fault classification are among the benefits gained from the rich information these image sensors can provide. In literature, process monitoring often refers to the step of detecting and isolating abnormal samples in a certain process. Normally, after process monitoring, fault classification is performed, and the isolated fault is classified into one or more known types of fault. Fault classification is an essential step within the process monitoring loop, at which point the type of detected and identified faults are determined (Chiang et al., 2001). Accurate fault classification can provide engineers with favorable information to isolate and diagnose system faults and anomalies to improve quality and maximize system efficiency. However, fault classification in manufacturing systems typically assumes a fixed set of fault modes. In this case, the existing fault classification model may make overconfident decisions or fail silently and, at certain times, dangerously for new unseen fault types.
Sparse Logistic Regression with High-order Features for Automatic Grammar Rule Extraction from Treebanks
Herrera, Santiago, Corro, Caio, Kahane, Sylvain
Descriptive grammars are highly valuable, but writing them is time-consuming and difficult. Furthermore, while linguists typically use corpora to create them, grammar descriptions often lack quantitative data. As for formal grammars, they can be challenging to interpret. In this paper, we propose a new method to extract and explore significant fine-grained grammar patterns and potential syntactic grammar rules from treebanks, in order to create an easy-to-understand corpus-based grammar. More specifically, we extract descriptions and rules across different languages for two linguistic phenomena, agreement and word order, using a large search space and paying special attention to the ranking order of the extracted rules. For that, we use a linear classifier to extract the most salient features that predict the linguistic phenomena under study. We associate statistical information to each rule, and we compare the ranking of the model's results to those of other quantitative and statistical measures.
UCxn: Typologically Informed Annotation of Constructions Atop Universal Dependencies
Weissweiler, Leonie, Böbel, Nina, Guiller, Kirian, Herrera, Santiago, Scivetti, Wesley, Lorenzi, Arthur, Melnik, Nurit, Bhatia, Archna, Schütze, Hinrich, Levin, Lori, Zeldes, Amir, Nivre, Joakim, Croft, William, Schneider, Nathan
The Universal Dependencies (UD) project has created an invaluable collection of treebanks with contributions in over 140 languages. However, the UD annotations do not tell the full story. Grammatical constructions that convey meaning through a particular combination of several morphosyntactic elements -- for example, interrogative sentences with special markers and/or word orders -- are not labeled holistically. We argue for (i) augmenting UD annotations with a 'UCxn' annotation layer for such meaning-bearing grammatical constructions, and (ii) approaching this in a typologically informed way so that morphosyntactic strategies can be compared across languages. As a case study, we consider five construction families in ten languages, identifying instances of each construction in UD treebanks through the use of morphosyntactic patterns. In addition to findings regarding these particular constructions, our study yields important insights on methodology for describing and identifying constructions in language-general and language-particular ways, and lays the foundation for future constructional enrichment of UD treebanks.
SciCapenter: Supporting Caption Composition for Scientific Figures with Machine-Generated Captions and Ratings
Hsu, Ting-Yao, Huang, Chieh-Yang, Huang, Shih-Hong, Rossi, Ryan, Kim, Sungchul, Yu, Tong, Giles, C. Lee, Huang, Ting-Hao K.
Crafting effective captions for figures is important. Readers heavily depend on these captions to grasp the figure's message. However, despite a well-developed set of AI technologies for figures and captions, these have rarely been tested for usefulness in aiding caption writing. This paper introduces SciCapenter, an interactive system that puts together cutting-edge AI technologies for scientific figure captions to aid caption composition. SciCapenter generates a variety of captions for each figure in a scholarly article, providing scores and a comprehensive checklist to assess caption quality across multiple critical aspects, such as helpfulness, OCR mention, key takeaways, and visual properties reference. Users can directly edit captions in SciCapenter, resubmit for revised evaluations, and iteratively refine them. A user study with Ph.D. students indicates that SciCapenter significantly lowers the cognitive load of caption writing. Participants' feedback further offers valuable design insights for future systems aiming to enhance caption writing.
AE SemRL: Learning Semantic Association Rules with Autoencoders
Karabulut, Erkan, Degeler, Victoria, Groth, Paul
Association Rule Mining (ARM) is the task of learning associations among data features in the form of logical rules. Mining association rules from high-dimensional numerical data, for example, time series data from a large number of sensors in a smart environment, is a computationally intensive task. In this study, we propose an Autoencoder-based approach to learn and extract association rules from time series data (AE SemRL). Moreover, we argue that in the presence of semantic information related to time series data sources, semantics can facilitate learning generalizable and explainable association rules. Despite enriching time series data with additional semantic features, AE SemRL makes learning association rules from high-dimensional data feasible. Our experiments show that semantic association rules can be extracted from a latent representation created by an Autoencoder and this method has in the order of hundreds of times faster execution time than state-of-the-art ARM approaches in many scenarios. We believe that this study advances a new way of extracting associations from representations and has the potential to inspire more research in this field.
Identification of Craving Maps among Marijuana Users via the Analysis of Functional Brain Networks with High-Order Attention Graph Neural Networks
Ding, Jun-En, Yang, Shihao, Zilverstand, Anna, Liu, Feng
The excessive consumption of marijuana can induce substantial psychological and social consequences. In this investigation, we propose an elucidative framework termed high-order graph attention neural networks (HOGANN) for the classification of Marijuana addiction, coupled with an analysis of localized brain network communities exhibiting abnormal activities among chronic marijuana users. HOGANN integrates dynamic intrinsic functional brain networks, estimated from resting-state functional magnetic resonance imaging (rs-fMRI), using long short-term memory (LSTM) to capture temporal network dynamics. We employ a high-order attention module for information fusion and message passing among neighboring nodes, enhancing the network community analysis. Our model is validated across two distinct data cohorts, yielding substantially higher classification accuracy than benchmark algorithms. Furthermore, we discern the most pertinent subnetworks and cognitive regions affected by persistent marijuana consumption, indicating adverse effects on functional brain networks, particularly within the dorsal attention and frontoparietal networks. Intriguingly, our model demonstrates superior performance in cohorts exhibiting prolonged dependence, implying that prolonged marijuana usage induces more pronounced alterations in brain networks. The model proficiently identifies craving brain maps, thereby delineating critical brain regions for analysis.
A Design Space for Intelligent and Interactive Writing Assistants
Lee, Mina, Gero, Katy Ilonka, Chung, John Joon Young, Shum, Simon Buckingham, Raheja, Vipul, Shen, Hua, Venugopalan, Subhashini, Wambsganss, Thiemo, Zhou, David, Alghamdi, Emad A., August, Tal, Bhat, Avinash, Choksi, Madiha Zahrah, Dutta, Senjuti, Guo, Jin L. C., Hoque, Md Naimul, Kim, Yewon, Knight, Simon, Neshaei, Seyed Parsa, Sergeyuk, Agnia, Shibani, Antonette, Shrivastava, Disha, Shroff, Lila, Stark, Jessi, Sterman, Sarah, Wang, Sitong, Bosselut, Antoine, Buschek, Daniel, Chang, Joseph Chee, Chen, Sherol, Kreminski, Max, Park, Joonsuk, Pea, Roy, Rho, Eugenia H., Shen, Shannon Zejiang, Siangliulue, Pao
In our era of rapid technological advancement, the research landscape for writing assistants has become increasingly fragmented across various research communities. We seek to address this challenge by proposing a design space as a structured way to examine and explore the multidimensional space of intelligent and interactive writing assistants. Through a large community collaboration, we explore five aspects of writing assistants: task, user, technology, interaction, and ecosystem. Within each aspect, we define dimensions (i.e., fundamental components of an aspect) and codes (i.e., potential options for each dimension) by systematically reviewing 115 papers. Our design space aims to offer researchers and designers a practical tool to navigate, comprehend, and compare the various possibilities of writing assistants, and aid in the envisioning and design of new writing assistants.
FENICE: Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction
Scirè, Alessandro, Ghonim, Karim, Navigli, Roberto
Recent advancements in text summarization, particularly with the advent of Large Language Models (LLMs), have shown remarkable performance. However, a notable challenge persists as a substantial number of automatically-generated summaries exhibit factual inconsistencies, such as hallucinations. In response to this issue, various approaches for the evaluation of consistency for summarization have emerged. Yet, these newly-introduced metrics face several limitations, including lack of interpretability, focus on short document summaries (e.g., news articles), and computational impracticality, especially for LLM-based metrics. To address these shortcomings, we propose Factuality Evaluation of summarization based on Natural language Inference and Claim Extraction (FENICE), a more interpretable and efficient factuality-oriented metric. FENICE leverages an NLI-based alignment between information in the source document and a set of atomic facts, referred to as claims, extracted from the summary. Our metric sets a new state of the art on AGGREFACT, the de-facto benchmark for factuality evaluation. Moreover, we extend our evaluation to a more challenging setting by conducting a human annotation process of long-form summarization.
Sabi\'a-2: A New Generation of Portuguese Large Language Models
Almeida, Thales Sales, Abonizio, Hugo, Nogueira, Rodrigo, Pires, Ramon
We introduce Sabi\'a-2, a family of large language models trained on Portuguese texts. The models are evaluated on a diverse range of exams, including entry-level tests for Brazilian universities, professional certification exams, and graduate-level exams for various disciplines such as accounting, economics, engineering, law and medicine. Our results reveal that our best model so far, Sabi\'a-2 Medium, matches or surpasses GPT-4's performance in 23 out of 64 exams and outperforms GPT-3.5 in 58 out of 64 exams. Notably, specialization has a significant impact on a model's performance without the need to increase its size, allowing us to offer Sabi\'a-2 Medium at a price per token that is 10 times cheaper than GPT-4. Finally, we identified that math and coding are key abilities that need improvement.
ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?
Huang, Fan, Kwak, Haewoon, Park, Kunwoo, An, Jisun
As AI becomes more integral in our lives, the need for transparency and responsibility grows. While natural language explanations (NLEs) are vital for clarifying the reasoning behind AI decisions, evaluating them through human judgments is complex and resource-intensive due to subjectivity and the need for fine-grained ratings. This study explores the alignment between ChatGPT and human assessments across multiple scales (i.e., binary, ternary, and 7-Likert scale). We sample 300 data instances from three NLE datasets and collect 900 human annotations for both informativeness and clarity scores as the text quality measurement. We further conduct paired comparison experiments under different ranges of subjectivity scores, where the baseline comes from 8,346 human annotations. Our results show that ChatGPT aligns better with humans in more coarse-grained scales. Also, paired comparisons and dynamic prompting (i.e., providing semantically similar examples in the prompt) improve the alignment. This research advances our understanding of large language models' capabilities to assess the text explanation quality in different configurations for responsible AI development.