Oceania
Robust Educational Dialogue Act Classifiers with Low-Resource and Imbalanced Datasets
Lin, Jionghao, Tan, Wei, Nguyen, Ngoc Dang, Lang, David, Du, Lan, Buntine, Wray, Beare, Richard, Chen, Guanliang, Gasevic, Dragan
Dialogue acts (DAs) can represent conversational actions of tutors or students that take place during tutoring dialogues. Automating the identification of DAs in tutoring dialogues is significant to the design of dialogue-based intelligent tutoring systems. Many prior studies employ machine learning models to classify DAs in tutoring dialogues and invest much effort to optimize the classification accuracy by using limited amounts of training data (i.e., low-resource data scenario). However, beyond the classification accuracy, the robustness of the classifier is also important, which can reflect the capability of the classifier on learning the patterns from different class distributions. We note that many prior studies on classifying educational DAs employ cross entropy (CE) loss to optimize DA classifiers on low-resource data with imbalanced DA distribution. The DA classifiers in these studies tend to prioritize accuracy on the majority class at the expense of the minority class which might not be robust to the data with imbalanced ratios of different DA classes. To optimize the robustness of classifiers on imbalanced class distributions, we propose to optimize the performance of the DA classifier by maximizing the area under the ROC curve (AUC) score (i.e., AUC maximization). Through extensive experiments, our study provides evidence that (i) by maximizing AUC in the training process, the DA classifier achieves significant performance improvement compared to the CE approach under low-resource data, and (ii) AUC maximization approaches can improve the robustness of the DA classifier under different class imbalance ratios.
Data Analytics Lead at carsales - Melbourne, Australia
Are you ready to be a big part of something big? For 25 years we've been evolving with the new economy to help people choose how they move today and tomorrow. When it comes to your career, we want to provide big opportunities to help you make a big impact. But for this to be possible, we strive to feel small. Small enough to quickly change tack, small enough to learn from different departments and small enough to connect authentically with leadership.
Factbox: Governments' efforts to regulate AI tools
April 12 (Reuters) - Italy's data protection agency said on Wednesday it would lift its temporary ban on OpenAI's ChatGPT artificial intelligence (AI) technology if the U.S. company complied with data protection and privacy demands by end-April. Rapid advances in AI such as Microsoft-backed OpenAI's ChatGPT are complicating governments' efforts to agree on laws governing the use of the technology. The government requested advice on how to respond to AI from Australia's main science advisory body and is considering next steps, a spokesperson for the industry and science minister said on April 12. Britain said in March it plans to split responsibility for governing AI between its regulators for human rights, health and safety, and competition, rather than creating a new body. China's cyberspace regulator on April 11 unveiled draft measures to manage generative AI services, saying it wants firms to submit security assessments to authorities before they launch offerings to the public. China's capital Beijing will support leading enterprises in building AI models that can challenge ChatGPT, its economy and information technology bureau said in February.
Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles
Farid, Alec, Veer, Sushant, Ivanovic, Boris, Leung, Karen, Pavone, Marco
In modern autonomy stacks, prediction modules are paramount to planning motions in the presence of other mobile agents. However, failures in prediction modules can mislead the downstream planner into making unsafe decisions. Indeed, the high uncertainty inherent to the task of trajectory forecasting ensures that such mispredictions occur frequently. Motivated by the need to improve safety of autonomous vehicles without compromising on their performance, we develop a probabilistic run-time monitor that detects when a "harmful" prediction failure occurs, i.e., a task-relevant failure detector. We achieve this by propagating trajectory prediction errors to the planning cost to reason about their impact on the AV. Furthermore, our detector comes equipped with performance measures on the false-positive and the false-negative rate and allows for data-free calibration. In our experiments we compared our detector with various others and found that our detector has the highest area under the receiver operator characteristic curve.
Keeping the Questions Conversational: Using Structured Representations to Resolve Dependency in Conversational Question Answering
Zaib, Munazza, Sheng, Quan Z., Zhang, Wei Emma, Mahmood, Adnan
Having an intelligent dialogue agent that can engage in conversational question answering (ConvQA) is now no longer limited to Sci-Fi movies only and has, in fact, turned into a reality. These intelligent agents are required to understand and correctly interpret the sequential turns provided as the context of the given question. However, these sequential questions are sometimes left implicit and thus require the resolution of some natural language phenomena such as anaphora and ellipsis. The task of question rewriting has the potential to address the challenges of resolving dependencies amongst the contextual turns by transforming them into intent-explicit questions. Nonetheless, the solution of rewriting the implicit questions comes with some potential challenges such as resulting in verbose questions and taking conversational aspect out of the scenario by generating self-contained questions. In this paper, we propose a novel framework, CONVSR (CONVQA using Structured Representations) for capturing and generating intermediate representations as conversational cues to enhance the capability of the QA model to better interpret the incomplete questions. We also deliberate how the strengths of this task could be leveraged in a bid to design more engaging and eloquent conversational agents. We test our model on the QuAC and CANARD datasets and illustrate by experimental results that our proposed framework achieves a better F1 score than the standard question rewriting model.
Collaborative Ground-Aerial Multi-Robot System for Disaster Response Missions with a Low-Cost Drone Add-On for Off-the-Shelf Drones
Rajapakshe, Shalutha, Wickramasinghe, Dilanka, Gurusinghe, Sahan, Ishtaweera, Deepana, Silva, Bhanuka, Jayasekara, Peshala, Panitz, Nick, Flick, Paul, Kottege, Navinda
In disaster-stricken environments, it's vital to assess the damage quickly, analyse the stability of the environment, and allocate resources to the most vulnerable areas where victims might be present. These missions are difficult and dangerous to be conducted directly by humans. Using the complementary capabilities of both the ground and aerial robots, we investigate a collaborative approach of aerial and ground robots to address this problem. With an increased field of view, faster speed, and compact size, the aerial robot explores the area and creates a 3D feature-based map graph of the environment while providing a live video stream to the ground control station. Once the aerial robot finishes the exploration run, the ground control station processes the map and sends it to the ground robot. The ground robot, with its higher operation time, static stability, payload delivery and tele-conference capabilities, can then autonomously navigate to identified high-vulnerability locations. We have conducted experiments using a quadcopter and a hexapod robot in an indoor modelled environment with obstacles and uneven ground. Additionally, we have developed a low-cost drone add-on with value-added capabilities, such as victim detection, that can be attached to an off-the-shelf drone. The system was assessed for cost-effectiveness, energy efficiency, and scalability.
StyLEx: Explaining Style Using Human Lexical Annotations
Hayati, Shirley Anugrah, Park, Kyumin, Rajagopal, Dheeraj, Ungar, Lyle, Kang, Dongyeop
Large pre-trained language models have achieved impressive results on various style classification tasks, but they often learn spurious domain-specific words to make predictions (Hayati et al., 2021). While human explanation highlights stylistic tokens as important features for this task, we observe that model explanations often do not align with them. To tackle this issue, we introduce StyLEx, a model that learns from human-annotated explanations of stylistic features and jointly learns to perform the task and predict these features as model explanations. Our experiments show that StyLEx can provide human-like stylistic lexical explanations without sacrificing the performance of sentence-level style prediction on both in-domain and out-of-domain datasets. Explanations from StyLEx show significant improvements in explanation metrics (sufficiency, plausibility) and when evaluated with human annotations. They are also more understandable by human judges compared to the widely-used saliency-based explanation baseline.
Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unit
Fitzgerald, Oisin, Perez-Concha, Oscar, Gallego-Luxan, Blanca, Metke-Jimenez, Alejandro, Rudd, Lachlan, Jorm, Louisa
Irregularly measured time series are common in many of the applied settings in which time series modelling is a key statistical tool, including medicine. This provides challenges in model choice, often necessitating imputation or similar strategies. Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations through incorporating continuous evolution of the hidden states between observations. This is achieved using a neural ordinary differential equation (ODE) or neural flow layer. In this manuscript, we give an overview of these models, including the varying architectures that have been proposed to account for issues such as ongoing medical interventions. Further, we demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting using electronic medical record and simulated data. The experiments confirm that addition of a neural ODE or neural flow layer generally improves the performance of autoregressive recurrent neural networks in the irregular measurement setting. However, several CTRNN architecture are outperformed by an autoregressive gradient boosted tree model (Catboost), with only a long short-term memory (LSTM) and neural ODE based architecture (ODE-LSTM) achieving comparable performance on probabilistic forecasting metrics such as the continuous ranked probability score (ODE-LSTM: 0.118$\pm$0.001; Catboost: 0.118$\pm$0.001), ignorance score (0.152$\pm$0.008; 0.149$\pm$0.002) and interval score (175$\pm$1; 176$\pm$1).
ChatGPT: Applications, Opportunities, and Threats
Bahrini, Aram, Khamoshifar, Mohammadsadra, Abbasimehr, Hossein, Riggs, Robert J., Esmaeili, Maryam, Majdabadkohne, Rastin Mastali, Pasehvar, Morteza
Developed by OpenAI, ChatGPT (Conditional Generative Pre-trained Transformer) is an artificial intelligence technology that is fine-tuned using supervised machine learning and reinforcement learning techniques, allowing a computer to generate natural language conversation fully autonomously. ChatGPT is built on the transformer architecture and trained on millions of conversations from various sources. The system combines the power of pre-trained deep learning models with a programmability layer to provide a strong base for generating natural language conversations. In this study, after reviewing the existing literature, we examine the applications, opportunities, and threats of ChatGPT in 10 main domains, providing detailed examples for the business and industry as well as education. We also conducted an experimental study, checking the effectiveness and comparing the performances of GPT-3.5 and GPT-4, and found that the latter performs significantly better. Despite its exceptional ability to generate natural-sounding responses, the authors believe that ChatGPT does not possess the same level of understanding, empathy, and creativity as a human and cannot fully replace them in most situations.
Medical Question Summarization with Entity-driven Contrastive Learning
Wei, Sibo, Lu, Wenpeng, Peng, Xueping, Wang, Shoujin, Wang, Yi-Fei, Zhang, Weiyu
By summarizing longer consumer health questions into shorter and essential ones, medical question answering (MQA) systems can more accurately understand consumer intentions and retrieve suitable answers. However, medical question summarization is very challenging due to obvious distinctions in health trouble descriptions from patients and doctors. Although existing works have attempted to utilize Seq2Seq, reinforcement learning, or contrastive learning to solve the problem, two challenges remain: how to correctly capture question focus to model its semantic intention, and how to obtain reliable datasets to fairly evaluate performance. To address these challenges, this paper proposes a novel medical question summarization framework using entity-driven contrastive learning (ECL). ECL employs medical entities in frequently asked questions (FAQs) as focuses and devises an effective mechanism to generate hard negative samples. This approach forces models to pay attention to the crucial focus information and generate more ideal question summarization. Additionally, we find that some MQA datasets suffer from serious data leakage problems, such as the iCliniq dataset's 33% duplicate rate. To evaluate the related methods fairly, this paper carefully checks leaked samples to reorganize more reasonable datasets. Extensive experiments demonstrate that our ECL method outperforms state-of-the-art methods by accurately capturing question focus and generating medical question summaries. The code and datasets are available at https://github.com/yrbobo/MQS-ECL.