Oceania
Explain, Adapt and Retrain: How to improve the accuracy of a PPM classifier through different explanation styles
Rizzi, Williams, Di Francescomarino, Chiara, Ghidini, Chiara, Maggi, Fabrizio Maria
Recent papers have introduced a novel approach to explain why a Predictive Process Monitoring (PPM) model for outcome-oriented predictions provides wrong predictions. Moreover, they have shown how to exploit the explanations, obtained using state-of-the art post-hoc explainers, to identify the most common features that induce a predictor to make mistakes in a semi-automated way, and, in turn, to reduce the impact of those features and increase the accuracy of the predictive model. This work starts from the assumption that frequent control flow patterns in event logs may represent important features that characterize, and therefore explain, a certain prediction. Therefore, in this paper, we (i) employ a novel encoding able to leverage DECLARE constraints in Predictive Process Monitoring and compare the effectiveness of this encoding with Predictive Process Monitoring state-of-the art encodings, in particular for the task of outcome-oriented predictions; (ii) introduce a completely automated pipeline for the identification of the most common features inducing a predictor to make mistakes; and (iii) show the effectiveness of the proposed pipeline in increasing the accuracy of the predictive model by validating it on different real-life datasets.
Complexity and scalability of defeasible reasoning in many-valued weighted knowledge bases with typicality
Alviano, Mario, Giordano, Laura, Duprรฉ, Daniele Theseider
Weighted knowledge bases for description logics with typicality under a "concept-wise" multi-preferential semantics provide a logical interpretation of MultiLayer Perceptrons. In this context, Answer Set Programming (ASP) has been shown to be suitable for addressing defeasible reasoning in the finitely many-valued case, providing a $\Pi^p_2$ upper bound on the complexity of the problem, nonetheless leaving unknown the exact complexity and only providing a proof-of-concept implementation. This paper fulfils the lack by providing a $P^{NP[log]}$-completeness result and new ASP encodings that deal with weighted knowledge bases with large search spaces.
Opportunities and Challenges in Neural Dialog Tutoring
Macina, Jakub, Daheim, Nico, Wang, Lingzhi, Sinha, Tanmay, Kapur, Manu, Gurevych, Iryna, Sachan, Mrinmaya
Designing dialog tutors has been challenging as it involves modeling the diverse and complex pedagogical strategies employed by human tutors. Although there have been significant recent advances in neural conversational systems using large language models (LLMs) and growth in available dialog corpora, dialog tutoring has largely remained unaffected by these advances. In this paper, we rigorously analyze various generative language models on two dialog tutoring datasets for language learning using automatic and human evaluations to understand the new opportunities brought by these advances as well as the challenges we must overcome to build models that would be usable in real educational settings. We find that although current approaches can model tutoring in constrained learning scenarios when the number of concepts to be taught and possible teacher strategies are small, they perform poorly in less constrained scenarios. Our human quality evaluation shows that both models and ground-truth annotations exhibit low performance in terms of equitable tutoring, which measures learning opportunities for students and how engaging the dialog is. To understand the behavior of our models in a real tutoring setting, we conduct a user study using expert annotators and find a significantly large number of model reasoning errors in 45% of conversations. Finally, we connect our findings to outline future work.
Zero-Shot On-the-Fly Event Schema Induction
Dror, Rotem, Wang, Haoyu, Roth, Dan
What are the events involved in a pandemic outbreak? What steps should be taken when planning a wedding? The answers to these questions can be found by collecting many documents on the complex event of interest, extracting relevant information, and analyzing it. We present a new approach in which large language models are utilized to generate source documents that allow predicting, given a high-level event definition, the specific events, arguments, and relations between them to construct a schema that describes the complex event in its entirety. Using our model, complete schemas on any topic can be generated on-the-fly without any manual data collection, i.e., in a zero-shot manner. Moreover, we develop efficient methods to extract pertinent information from texts and demonstrate in a series of experiments that these schemas are considered to be more complete than human-curated ones in the majority of examined scenarios. Finally, we show that this framework is comparable in performance with previous supervised schema induction methods that rely on collecting real texts while being more general and flexible without the need for a predefined ontology.
Predicting Adverse Neonatal Outcomes for Preterm Neonates with Multi-Task Learning
Lin, Jingyang, Chen, Junyu, Lyu, Hanjia, Khodak, Igor, Chhabra, Divya, Richardson, Colby L Day, Prelipcean, Irina, Dylag, Andrew M, Luo, Jiebo
Diagnosis of adverse neonatal outcomes is crucial for preterm survival since it enables doctors to provide timely treatment. Machine learning (ML) algorithms have been demonstrated to be effective in predicting adverse neonatal outcomes. However, most previous ML-based methods have only focused on predicting a single outcome, ignoring the potential correlations between different outcomes, and potentially leading to suboptimal results and overfitting issues. In this work, we first analyze the correlations between three adverse neonatal outcomes and then formulate the diagnosis of multiple neonatal outcomes as a multi-task learning (MTL) problem. We then propose an MTL framework to jointly predict multiple adverse neonatal outcomes. In particular, the MTL framework contains shared hidden layers and multiple task-specific branches. Extensive experiments have been conducted using Electronic Health Records (EHRs) from 121 preterm neonates. Empirical results demonstrate the effectiveness of the MTL framework. Furthermore, the feature importance is analyzed for each neonatal outcome, providing insights into model interpretability.
A New Family of Generalization Bounds Using Samplewise Evaluated CMI
Hellstrรถm, Fredrik, Durisi, Giuseppe
We present a new family of information-theoretic generalization bounds, in which the training loss and the population loss are compared through a jointly convex function. This function is upper-bounded in terms of the disintegrated, samplewise, evaluated conditional mutual information (CMI), an information measure that depends on the losses incurred by the selected hypothesis, rather than on the hypothesis itself, as is common in probably approximately correct (PAC)-Bayesian results. We demonstrate the generality of this framework by recovering and extending previously known information-theoretic bounds. Furthermore, using the evaluated CMI, we derive a samplewise, average version of Seeger's PAC-Bayesian bound, where the convex function is the binary KL divergence. In some scenarios, this novel bound results in a tighter characterization of the population loss of deep neural networks than previous bounds. Finally, we derive high-probability versions of some of these average bounds. We demonstrate the unifying nature of the evaluated CMI bounds by using them to recover average and high-probability generalization bounds for multiclass classification with finite Natarajan dimension.
Welcome to Cat Royale
Have you got questions about Cat Royale? From 22nd March โ 2nd April 2023, three cats โ Ghostbuster, Pumpkin and Clover โ will visit a utopia created by the Blast Theory artists. The cats' every need is catered for. They have dens to curl up in, high platforms to pounce from and curved walls to explore. And at the centre of the room, a robot arm trained by an Artificial Intelligence offers games to make the cats happier. This page is your go to for everything Cat Royale.
Robot recruiters: can bias be banished from AI hiring?
Michael Scott, the protagonist from the US version of The Office, is using an AI recruiter to hire a receptionist. The text-based system asks applicants five questions that delve into how they responded to past work situations, including dealing with difficult colleagues and juggling competing work demands. Potential employees type their answers into a chat-style program that resembles a responsive help desk. The real โ and unnerving โ power of AI then kicks in, sending a score and traits profile to the employer, and a personality report to the applicant. This demonstration, by Melbourne-based startup Sapia.ai,
Indoor Positioning using Wi-Fi and Machine Learning for Industry 5.0
Neupane, Inoj, Alsinglawi, Belal, Rabie, Khaled
Humans and robots working together in an environment to enhance human performance is the aim of Industry 5.0. Although significant progress in outdoor positioning has been seen, indoor positioning remains a challenge. In this paper, we introduce a new research concept by exploiting the potential of indoor positioning for Industry 5.0. We use Wi-Fi Received Signal Strength Indicator (RSSI) with trilateration using cheap and easily available ESP32 Arduino boards for positioning as well as sending effective route signals to a human and a robot working in a simulated-indoor factory environment in real-time. We utilized machine learning models to detect safe closeness between two co-workers (a human subject and a robot). Experimental data and analysis show an average deviation of less than 1m from the actual distance while the targets are mobile or stationary.
CeFlow: A Robust and Efficient Counterfactual Explanation Framework for Tabular Data using Normalizing Flows
Duong, Tri Dung, Li, Qian, Xu, Guandong
Counterfactual explanation is a form of interpretable machine learning that generates perturbations on a sample to achieve the desired outcome. The generated samples can act as instructions to guide end users on how to observe the desired results by altering samples. Although state-of-the-art counterfactual explanation methods are proposed to use variational autoencoder (VAE) to achieve promising improvements, they suffer from two major limitations: 1) the counterfactuals generation is prohibitively slow, which prevents algorithms from being deployed in interactive environments; 2) the counterfactual explanation algorithms produce unstable results due to the randomness in the sampling procedure of variational autoencoder. In this work, to address the above limitations, we design a robust and efficient counterfactual explanation framework, namely CeFlow, which utilizes normalizing flows for the mixed-type of continuous and categorical features. Numerical experiments demonstrate that our technique compares favorably to state-of-the-art methods.