Explicit decomposition modeling, which involves breaking down complex tasks into more straightforward and often more interpretable sub-tasks, has long been a central theme in developing robust and interpretable NLU systems. However, despite the many datasets and resources built as part of this effort, the majority have small-scale annotations and limited scope, which is insufficient to solve general decomposition tasks. In this paper, we look at large-scale intermediate pre-training of decomposition-based transformers using distant supervision from comparable texts, particularly large-scale parallel news. We show that with such intermediate pre-training, developing robust decomposition-based models for a diverse range of tasks becomes more feasible. For example, on semantic parsing, our model, DecompT5, improves 20% to 30% on two datasets, Overnight and TORQUE, over the baseline language model. We further use DecompT5 to build a novel decomposition-based QA system named DecompEntail, improving over state-of-the-art models, including GPT-3, on both HotpotQA and StrategyQA by 8% and 4%, respectively.

Russia has suspended its participation in a landmark agreement that allowed vital grain exports from Ukraine after what it said was a drone attack on Russian ships in occupied Crimea. Russia's defence ministry said Ukraine attacked the Black Sea Fleet near Sevastopol in the annexed Crimean Peninsula with 16 drones in the early hours of Saturday, and that British navy "specialists" had helped coordinate the "terrorist" attack. London bluntly rejected Moscow's claim. The Turkey and UN-brokered deal to unlock grain exports signed between Russia and Ukraine in July is critical to easing the global food crisis caused by the conflict. The agreement has already allowed more than 9 million tonnes of Ukrainian grain to be exported and was due to be renewed on November 19.

Sign up to receive the Future Tense newsletter every other Saturday. The U.N.'s COP27 climate summit kicks off on Nov. 6 in Egypt, inviting us, once again, to consider whether we're doing enough, fast enough, to stave off climate chaos and the suffering that will come with it. The scale of change required is head-spinningly drastic, so even unexpectedly rapid expansions in clean energy won't do much to curb malaise and doomsaying. Here in the U.S., the Inflation Reduction Act, the biggest climate investment in the nation's history, has been met, largely, with collective indifference, despite positive buzz about its potential effectiveness. The bill was, predictably, passed without any Republican votes, a grim reminder of the scale of climate denialism.

Universities around the world are taking steps alongside major technology companies to explore ways to bolster ethics education in the artificial intelligence field in line with an initiative supported by the Vatican. The effort seeks to help those already working or aspiring to work in the tech fields understand that the development of artificial intelligence, or AI, should benefit humanity rather than pose uncontrollable challenges to human life. Participants at a global summit at the University of Notre Dame Oct. 25-26 explored ways to encompass ethics education in coursework with speakers calling for widespread integration in both technical and nontechnical curricula. Casey Fiesler, associate professor of information science at the University of Colorado, told in person and online attendees in a session that the long-held view that ethical topics are a "specialization" within technology education must be put aside. "We should not be teaching ethics in the context of computing so that it is completely separate from everything else that we are doing," Fiesler said in calling for a culture shift in higher education that can reach across society.

Federated learning (FL) aims at optimizing a shared global model over multiple edge devices without transmitting (private) data to the central server. While it is theoretically well-known that FL yields an optimal model -- centrally trained model assuming availability of all the edge device data at the central server -- under mild condition, in practice, it often requires massive amount of iterations until convergence, especially under presence of statistical/computational heterogeneity. This paper utilizes cyclic learning rate at the server side to reduce the number of training iterations with increased performance without any additional computational costs for both the server and the edge devices. Numerical results validate that, simply plugging-in the proposed cyclic aggregation to the existing FL algorithms effectively reduces the number of training iterations with improved performance.

We introduce a new Slovak masked language model called SlovakBERT. This is to our best knowledge the first paper discussing Slovak transformers-based language models. We evaluate our model on several NLP tasks and achieve state-of-the-art results. This evaluation is likewise the first attempt to establish a benchmark for Slovak language models. We publish the masked language model, as well as the fine-tuned models for part-of-speech tagging, sentiment analysis and semantic textual similarity.

Uncertainty on human behaviors poses a significant challenge to autonomous driving in crowded urban environments. The partially observable Markov decision processes (POMDPs) offer a principled framework for planning under uncertainty, often leveraging Monte Carlo sampling to achieve online performance for complex tasks. However, sampling also raises safety concerns by potentially missing critical events. To address this, we propose a new algorithm, LEarning Attention over Driving bEhavioRs (LEADER), that learns to attend to critical human behaviors during planning. LEADER learns a neural network generator to provide attention over human behaviors in real-time situations. It integrates the attention into a belief-space planner, using importance sampling to bias reasoning towards critical events. To train the algorithm, we let the attention generator and the planner form a min-max game. By solving the min-max game, LEADER learns to perform risk-aware planning without human labeling.

We review the state of research on empathy in natural language processing and identify the following issues: (1) empathy definitions are absent or abstract, which (2) leads to low construct validity and reproducibility. Moreover, (3) emotional empathy is overemphasized, skewing our focus to a narrow subset of simplified tasks. We believe these issues hinder research progress and argue that current directions will benefit from a clear conceptualization that includes operationalizing cognitive empathy components. Our main objectives are to provide insight and guidance on empathy conceptualization for NLP research objectives and to encourage researchers to pursue the overlooked opportunities in this area, highly relevant, e.g., for clinical and educational sectors.

Building automatic speech recognition (ASR) systems is a challenging task, especially for under-resourced languages that need to construct corpora nearly from scratch and lack sufficient training data. It has emerged that several African indigenous languages, including Kiswahili, are technologically under-resourced. ASR systems are crucial, particularly for the hearing-impaired persons who can benefit from having transcripts in their native languages. However, the absence of transcribed speech datasets has complicated efforts to develop ASR models for these indigenous languages. This paper explores the transcription process and the development of a Kiswahili speech corpus, which includes both read-out texts and spontaneous speech data from native Kiswahili speakers. The study also discusses the vowels and consonants in Kiswahili and provides an updated Kiswahili phoneme dictionary for the ASR model that was created using the CMU Sphinx speech recognition toolbox, an open-source speech recognition toolkit. The ASR model was trained using an extended phonetic set that yielded a WER and SER of 18.87% and 49.5%, respectively, an improved performance than previous similar research for under-resourced languages.

This page contains ML Fundamentals glossary terms. The number of correct classification predictions divided by the total number of predictions. Binary classification provides specific names for the different categories of correct predictions and incorrect predictions. Compare and contrast accuracy with precision and recall. Although a valuable metric for some situations, accuracy is highly misleading for others. Notably, accuracy is usually a poor metric for evaluating classification models that process class-imbalanced datasets. For example, suppose snow falls only 25 days per century in a certain subtropical city. Since days without snow (the negative class) vastly outnumber days with snow (the positive class), the snow dataset for this city is class-imbalanced. Imagine a binary classification model that is supposed to predict either snow or no snow each day but simply predicts "no snow" every day. This model is highly accurate but has no predictive power. Although 99.93% accuracy seems like very a impressive percentage, the model actually has no predictive power. Precision and recall are usually more useful metrics than accuracy for evaluating models trained on class-imbalanced datasets. A function that enables neural networks to learn nonlinear (complex) relationships between features and the label. The plots of activation functions are never single straight lines. In a neural network, activation functions manipulate the weighted sum of all the inputs to a neuron. To calculate a weighted sum, the neuron adds up the products of the relevant values and weights. A non-human program or model that can solve sophisticated tasks. For example, a program or model that translates text or a program or model that identifies diseases from radiologic images both exhibit artificial intelligence. Formally, machine learning is a sub-field of artificial intelligence. However, in recent years, some organizations have begun using the terms artificial intelligence and machine learning interchangeably. A number between 0.0 and 1.0 representing a binary classification model's ability to separate positive classes from negative classes. The closer the AUC is to 1.0, the better the model's ability to separate classes from each other. For example, the following illustration shows a classifier model that separates positive classes (green ovals) from negative classes (purple rectangles) perfectly. This unrealistically perfect model has an AUC of 1.0: Conversely, the following illustration shows the results for a classifier model that generated random results.