Nonlinear dimensionality reduction methods are ubiquitously applied for visualization and preprocessing highdimensional data in machine learning [1, 2, 3, 4, 5, 6, 7, 8]. These methods assume that the intrinsic dimension of the underlying manifold is much lower than the ambient dimension of the real-world data [9, 10, 11]. Based on approximating the manifold by k nearest neighbour (kNN) graph, nonlinear dimensionality reduction projects data from high to low-dimensional space and retains the topological structure of original data. While nonlinear dimensionality reduction is effective for visualizing high-dimensional data, one major weakness is lacking interpretability of the reduced-dimension results [8]. The reduced dimensions of nonlinear dimensionality reduction have no specific meaning, compared with linear methods like Principal Component Analysis (PCA) where the dimensions of the embedding space represent the directions of the largest variance of original data. Particularly, nonlinear dimensionality reduction focuses on preserving distance between observations and thereby loses source feature information in the embedding space, resulting in failing to illustrate feature loadings that linear methods such as PCA can provide to explain the feature contribution in each dimension. In this paper, we seek to improve the interpretability of nonlinear dimensionality reduction. In addition to preserving the local topological structure between observations in the embedding space, we aim to incorporate the source features to devise an interpretable nonlinear dimensionality reduction method. The feature information is encoded in the column space of data, and we use the tangent space to locally depict the column space [12, 13].

Robots must move legibly around people for safety reasons, especially for tasks where physical contact is possible. One such task is handovers, which requires implicit communication on where and when physical contact (object transfer) occurs. In this work, we study whether the trajectory model used by a robot during the reaching phase affects the subjective perceptions of receivers for robot-to-human handovers. We conducted a user study where 32 participants were handed over three objects with four trajectory models: three were versions of a minimum jerk trajectory, and one was an ellipse-fitting-based trajectory. The start position of the handover was fixed for all trajectories, and the end position was allowed to vary randomly around a fixed position by $\pm$3 cm in all axis. The user study found no significant differences among the handover trajectories in survey questions relating to safety, predictability, naturalness, and other subjective metrics. While these results seemingly reject the hypothesis that the trajectory affects human perceptions of a handover, it prompts future research to investigate the effect of other variables, such as robot speed, object transfer position, object orientation at the transfer point, and explicit communication signals such as gaze and speech.

Text Simplification (TS) is the task of converting a text into a form that is easier to read while maintaining the meaning of the original text. A sub-task of TS is Cognitive Simplification (CS), converting text to a form that is readily understood by people with cognitive disabilities without rendering it childish or simplistic. This sub-task has yet to be explored with neural methods in NLP, and resources for it are scarcely available. In this paper, we present a method for incorporating knowledge from the cognitive accessibility domain into a TS model, by introducing an inductive bias regarding what simplification operations to use. We show that by adding this inductive bias to a TS-trained model, it is able to adapt better to CS without ever seeing CS data, and outperform a baseline model on a traditional TS benchmark. In addition, we provide a novel test dataset for CS, and analyze the differences between CS corpora and existing TS corpora, in terms of how simplification operations are applied.

Although neural networks have seen tremendous success as predictive models in a variety of domains, they can be overly confident in their predictions on out-of-distribution (OOD) data. To be viable for safety-critical applications, like autonomous vehicles, neural networks must accurately estimate their epistemic or model uncertainty, achieving a level of system self-awareness. Techniques for epistemic uncertainty quantification often require OOD data during training or multiple neural network forward passes during inference. These approaches may not be suitable for real-time performance on high-dimensional inputs. Furthermore, existing methods lack interpretability of the estimated uncertainty, which limits their usefulness both to engineers for further system development and to downstream modules in the autonomy stack. We propose the use of evidential deep learning to estimate the epistemic uncertainty over a low-dimensional, interpretable latent space in a trajectory prediction setting. We introduce an interpretable paradigm for trajectory prediction that distributes the uncertainty among the semantic concepts: past agent behavior, road structure, and social context. We validate our approach on real-world autonomous driving data, demonstrating superior performance over state-of-the-art baselines. Our code is available at: https://github.com/sisl/InterpretableSelfAwarePrediction.

Keyword spotting (KWS) has been widely used in various speech control scenarios. The training of KWS is usually based on deep neural networks and requires a large amount of data. Manufacturers often use third-party data to train KWS. However, deep neural networks are not sufficiently interpretable to manufacturers, and attackers can manipulate third-party training data to plant backdoors during the model training. An effective backdoor attack can force the model to make specified judgments under certain conditions, i.e., triggers. In this paper, we design a backdoor attack scheme based on Pitch Boosting and Sound Masking for KWS, called PBSM. Experimental results demonstrated that PBSM is feasible to achieve an average attack success rate close to 90% in three victim models when poisoning less than 1% of the training data.

We develop data-driven models to predict when a robot should feed during social dining scenarios. Being able to eat independently with friends and family is considered one of the most memorable and important activities for people with mobility limitations. While existing robotic systems for feeding people with mobility limitations focus on solitary dining, commensality, the act of eating together, is often the practice of choice. Sharing meals with others introduces the problem of socially appropriate bite timing for a robot, i.e. the appropriate timing for the robot to feed without disrupting the social dynamics of a shared meal. Our key insight is that bite timing strategies that take into account the delicate balance of social cues can lead to seamless interactions during robot-assisted feeding in a social dining scenario. We approach this problem by collecting a Human-Human Commensality Dataset (HHCD) containing 30 groups of three people eating together. We use this dataset to analyze human-human commensality behaviors and develop bite timing prediction models in social dining scenarios. We also transfer these models to human-robot commensality scenarios. Our user studies show that prediction improves when our algorithm uses multimodal social signaling cues between diners to model bite timing. The HHCD dataset, videos of user studies, and code are available at https://emprise.cs.cornell.edu/hrcom/

In this paper, we propose the task of consecutive question generation (CQG), which generates a set of logically related question-answer pairs to understand a whole passage, with a comprehensive consideration of the aspects including accuracy, coverage, and informativeness. To achieve this, we first examine the four key elements of CQG, i.e., question, answer, rationale, and context history, and propose a novel dynamic multitask framework with one main task generating a question-answer pair, and four auxiliary tasks generating other elements. It directly helps the model generate good questions through both joint training and self-reranking. At the same time, to fully explore the worth-asking information in a given passage, we make use of the reranking losses to sample the rationales and search for the best question series globally. Finally, we measure our strategy by QA data augmentation and manual evaluation, as well as a novel application of generated question-answer pairs on DocNLI. We prove that our strategy can improve question generation significantly and benefit multiple related NLP tasks.

Pretrained language models (LMs) do not capture factual knowledge very well. This has led to the development of a number of knowledge integration (KI) methods which aim to incorporate external knowledge into pretrained LMs. Even though KI methods show some performance gains over vanilla LMs, the inner-workings of these methods are not well-understood. For instance, it is unclear how and what kind of knowledge is effectively integrated into these models and if such integration may lead to catastrophic forgetting of already learned knowledge. This paper revisits the KI process in these models with an information-theoretic view and shows that KI can be interpreted using a graph convolution operation. We propose a probe model called \textit{Graph Convolution Simulator} (GCS) for interpreting knowledge-enhanced LMs and exposing what kind of knowledge is integrated into these models. We conduct experiments to verify that our GCS can indeed be used to correctly interpret the KI process, and we use it to analyze two well-known knowledge-enhanced LMs: ERNIE and K-Adapter, and find that only a small amount of factual knowledge is integrated in them. We stratify knowledge in terms of various relation types and find that ERNIE and K-Adapter integrate different kinds of knowledge to different extent. Our analysis also shows that simply increasing the size of the KI corpus may not lead to better KI; fundamental advances may be needed.

In an international collaboration, University of Queensland palaeontologist Dr Anthony Romilio used AI pattern recognition to re-analyse footprints from the Dinosaur Stampede National Monument, south-west of Winton in Central Queensland. "Large dinosaur footprints were first discovered back in the 1970s at a track site called the Dinosaur Stampede National Monument, and for many years they were believed to be left by a predatory dinosaur, like Australovenator, with legs nearly two metres long," said Dr Romilio. "The mysterious tracks were thought to be left during the mid-Cretaceous Period, around 93 million years ago. "But working out what dino species made the footprints exactly -- especially from tens of millions of years ago -- can be a pretty difficult and confusing business. "Particularly since these big tracks are surrounded by thousands of tiny dinosaur footprints, leading many to think that this predatory beast could have sparked a stampede of smaller dinosaurs. "So, to crack the case, we decided to employ an AI program called Deep Convolutional Neural Networks." It was trained with 1,500 dinosaur footprints, all of which were theropod or ornithopod in origin -- the groups of dinosaurs relevant to the Dinosaur Stampede National Monument prints. The results were clear: the tracks had been made by a herbivorous ornithopod dinosaur. Dr Jens Lallensack, lead author from Liverpool John Moores University in the UK, said that the computer assistance was vital, as the team was originally at an impasse. "We were pretty stuck, so thank god for modern technology," Dr Lallensack said. "In our research team of three, one person was pro-meat-eater, one person was undecided, and one was pro-plant-eater.

It is our pleasure to invite you to submit a chapter for inclusion in the “Adversarial Multimedia Forensics” book to be Published by Springer – Advances in Information Security. The submitted chapter should have 15-20 pages of single-space single-column in latex and include sufficient details to be useful for Cybersecurity Applications experts and readers with […]