Performance Analysis
Robot Motion Prediction by Channel State Information
Zandi, Rojin, Salehinejad, Hojjat, Behzad, Kian, Motamedi, Elaheh, Siami, Milad
Autonomous robotic systems have gained a lot of attention, in recent years. However, accurate prediction of robot motion in indoor environments with limited visibility is challenging. While vision-based and light detection and ranging (LiDAR) sensors are commonly used for motion detection and localization of robotic arms, they are privacy-invasive and depend on a clear line-of-sight (LOS) for precise measurements. In cases where additional sensors are not available or LOS is not possible, these technologies may not be the best option. This paper proposes a novel method that employs channel state information (CSI) from WiFi signals affected by robotic arm motion. We developed a convolutional neural network (CNN) model to classify four different activities of a Franka Emika robotic arm. The implemented method seeks to accurately predict robot motion even in scenarios in which the robot is obscured by obstacles, without relying on any attached or internal sensors.
Scalable Membership Inference Attacks via Quantile Regression
Bertran, Martin, Tang, Shuai, Kearns, Michael, Morgenstern, Jamie, Roth, Aaron, Wu, Zhiwei Steven
The basic goal of privacy-preserving machine learning is to find models that are predictive on some underlying data distribution, without being disclosive of the particular data points on which they were trained. The simplest kind of attack that can be launched on a trained model--falsifying privacy guarantees--is a membership inference attack. A membership inference attack, informally, is a statistical test that is able to reliably determine whether a particular data point was included in the training set used to train the model or not. Almost all membership inference attacks are based on the observation that models tend to overfit their training sets in different ways. In particular, they tend to systematically predict higher confidence in the true labels of data points from their training set, compared to points drawn from the same distribution not in their training set. The confidence that a model places on the true label of a data-point is thus a natural test statistic to build a membership-inference hypothesis test around. A variety of recent methods [Shokri et al., 2017, Long et al., 2020, Sablayrolles et al., 2019, Song and Mittal, 2021, Carlini et al., 2022] are based around this idea, and aim to estimate the distribution of the test statistic (the confidence assigned to the true label of a datapoint) over the distribution of datapoints that were not used in training (and sometimes, Martin and Shuai are lead authors; all other authors are listed in alphabetical order.
The distribution of discourse relations within and across turns in spontaneous conversation
Cortez, S. Magalí López, Jacobs, Cassandra L.
Time pressure and topic negotiation may impose constraints on how people leverage discourse relations (DRs) in spontaneous conversational contexts. In this work, we adapt a system of DRs for written language to spontaneous dialogue using crowdsourced annotations from novice annotators. We then test whether discourse relations are used differently across several types of multi-utterance contexts. We compare the patterns of DR annotation within and across speakers and within and across turns. Ultimately, we find that different discourse contexts produce distinct distributions of discourse relations, with single-turn annotations creating the most uncertainty for annotators. Additionally, we find that the discourse relation annotations are of sufficient quality to predict from embeddings of discourse units.
S2vNTM: Semi-supervised vMF Neural Topic Modeling
Xu, Weijie, Desai, Jay, Sengamedu, Srinivasan, Jiang, Xiaoyu, Iannacci, Francis
Language model based methods are powerful techniques for text classification. However, the models have several shortcomings. In this paper, we propose Semi-Supervised vMF Neural Topic Modeling (S2vNTM) to overcome these difficulties. S2vNTM takes a few seed keywords as input for topics. S2vNTM leverages the pattern of keywords to identify potential topics, as well as optimize the quality of topics' keywords sets. Across a variety of datasets, S2vNTM outperforms existing semi-supervised topic modeling methods in classification accuracy with limited keywords provided. S2vNTM is at least twice as fast as baselines. Language Model (LM) pre-training Vaswani et al. (2017); Devlin et al. (2018) has proven to be useful in learning universal language representations. Recent language models such as Yang et al. (2019); Sun et al. (2019); Chen et al. (2022); Ding et al. (2021) have achieved amazing results in text classification. Most of these methods need enough high-quality labels to train. To make LM based methods work well when limited labels are available, few shot learning methods such as Bianchi et al. (2021); Meng et al. (2020a;b); Mekala and Shang (2020); Yu et al. (2021); Wang et al. (2021b) have been proposed. However, these methods rely on large pre-trained texts and can be biased to apply to a different environment. Topic modeling methods generate topics based on the pattern of words.
Undecimated Wavelet Transform for Word Embedded Semantic Marginal Autoencoder in Security improvement and Denoising different Languages
By combining the undecimated wavelet transform within a Word Embedded Semantic Marginal Autoencoder (WESMA), this research study provides a novel strategy for improving security measures and denoising multiple languages. The incorporation of these strategies is intended to address the issues of robustness, privacy, and multilingualism in data processing applications. The undecimated wavelet transform is used as a feature extraction tool to identify prominent language patterns and structural qualities in the input data. The proposed system may successfully capture significant information while preserving the temporal and geographical links within the data by employing this transform. This improves security measures by increasing the system's ability to detect abnormalities, discover hidden patterns, and distinguish between legitimate content and dangerous threats. The Word Embedded Semantic Marginal Autoencoder also functions as an intelligent framework for dimensionality and noise reduction. The autoencoder effectively learns the underlying semantics of the data and reduces noise components by exploiting word embeddings and semantic context. As a result, data quality and accuracy are increased in following processing stages. The suggested methodology is tested using a diversified dataset that includes several languages and security scenarios. The experimental results show that the proposed approach is effective in attaining security enhancement and denoising capabilities across multiple languages. The system is strong in dealing with linguistic variances, producing consistent outcomes regardless of the language used. Furthermore, incorporating the undecimated wavelet transform considerably improves the system's ability to efficiently address complex security concerns
Assisting Clinical Decisions for Scarcely Available Treatment via Disentangled Latent Representation
Xue, Bing, Said, Ahmed Sameh, Xu, Ziqi, Liu, Hanyang, Shah, Neel, Yang, Hanqing, Payne, Philip, Lu, Chenyang
Extracorporeal membrane oxygenation (ECMO) is an essential life-supporting modality for COVID-19 patients who are refractory to conventional therapies. However, the proper treatment decision has been the subject of significant debate and it remains controversial about who benefits from this scarcely available and technically complex treatment option. To support clinical decisions, it is a critical need to predict the treatment need and the potential treatment and no-treatment responses. Targeting this clinical challenge, we propose Treatment Variational AutoEncoder (TVAE), a novel approach for individualized treatment analysis. TVAE is specifically designed to address the modeling challenges like ECMO with strong treatment selection bias and scarce treatment cases. TVAE conceptualizes the treatment decision as a multi-scale problem. We model a patient's potential treatment assignment and the factual and counterfactual outcomes as part of their intrinsic characteristics that can be represented by a deep latent variable model. The factual and counterfactual prediction errors are alleviated via a reconstruction regularization scheme together with semi-supervision, and the selection bias and the scarcity of treatment cases are mitigated by the disentangled and distribution-matched latent space and the label-balancing generative strategy. We evaluate TVAE on two real-world COVID-19 datasets: an international dataset collected from 1651 hospitals across 63 countries, and a institutional dataset collected from 15 hospitals. The results show that TVAE outperforms state-of-the-art treatment effect models in predicting both the propensity scores and factual outcomes on heterogeneous COVID-19 datasets. Additional experiments also show TVAE outperforms the best existing models in individual treatment effect estimation on the synthesized IHDP benchmark dataset.
PseudoCell: Hard Negative Mining as Pseudo Labeling for Deep Learning-Based Centroblast Cell Detection
Seesawad, Narongrid, Ittichaiwong, Piyalitt, Sudhawiyangkul, Thapanun, Sawangjai, Phattarapong, Thuwajit, Peti, Boonsakan, Paisarn, Sripodok, Supasan, Veerakanjana, Kanyakorn, Luenam, Phoomraphee, Charngkaew, Komgrid, Pongpaibul, Ananya, Angkathunyakul, Napat, Hnoohom, Narit, Yuenyong, Sumeth, Thuwajit, Chanitra, Wilaiprasitporn, Theerawit
Patch classification models based on deep learning have been utilized in whole-slide images (WSI) of H&E-stained tissue samples to assist pathologists in grading follicular lymphoma patients. However, these approaches still require pathologists to manually identify centroblast cells and provide refined labels for optimal performance. To address this, we propose PseudoCell, an object detection framework to automate centroblast detection in WSI (source code is available at https://github.com/IoBT-VISTEC/PseudoCell.git). This framework incorporates centroblast labels from pathologists and combines them with pseudo-negative labels obtained from undersampled false-positive predictions using the cell's morphological features. By employing PseudoCell, pathologists' workload can be reduced as it accurately narrows down the areas requiring their attention during examining tissue. Depending on the confidence threshold, PseudoCell can eliminate 58.18-99.35% of non-centroblasts tissue areas on WSI. This study presents a practical centroblast prescreening method that does not require pathologists' refined labels for improvement. Detailed guidance on the practical implementation of PseudoCell is provided in the discussion section.
The Role of Subgroup Separability in Group-Fair Medical Image Classification
Jones, Charles, Roschewitz, Mélanie, Glocker, Ben
We investigate performance disparities in deep classifiers. We find that the ability of classifiers to separate individuals into subgroups varies substantially across medical imaging modalities and protected characteristics; crucially, we show that this property is predictive of algorithmic bias. Through theoretical analysis and extensive empirical evaluation, we find a relationship between subgroup separability, subgroup disparities, and performance degradation when models are trained on data with systematic bias such as underdiagnosis. Our findings shed new light on the question of how models become biased, providing important insights for the development of fair medical imaging AI.
Density-based Feasibility Learning with Normalizing Flows for Introspective Robotic Assembly
Feng, Jianxiang, Atad, Matan, Rodríguez, Ismael, Durner, Maximilian, Günnemann, Stephan, Triebel, Rudolph
Machine Learning (ML) models in Robotic Assembly Sequence Planning (RASP) need to be introspective on the predicted solutions, i.e. whether they are feasible or not, to circumvent potential efficiency degradation. Previous works need both feasible and infeasible examples during training. However, the infeasible ones are hard to collect sufficiently when re-training is required for swift adaptation to new product variants. In this work, we propose a density-based feasibility learning method that requires only feasible examples. Concretely, we formulate the feasibility learning problem as Out-of-Distribution (OOD) detection with Normalizing Flows (NF), which are powerful generative models for estimating complex probability distributions. Empirically, the proposed method is demonstrated on robotic assembly use cases and outperforms other single-class baselines in detecting infeasible assemblies. We further investigate the internal working mechanism of our method and show that a large memory saving can be obtained based on an advanced variant of NF.
Evaluating the Evaluators: Are Current Few-Shot Learning Benchmarks Fit for Purpose?
Shimabucoro, Luísa, Hospedales, Timothy, Gouk, Henry
The Few Shot-Learning (FSL) paradigm, which focuses on enabling models to generalise well with little data Numerous benchmarks for Few-Shot Learning through the use of transferred prior knowledge, has gained have been proposed in the last decade. However relevance in an attempt to overcome these challenges. A all of these benchmarks focus on performance significant amount of attention has been given to FSL and averaged over many tasks, and the question of related meta-learning research in the last decade (Wang how to reliably evaluate and tune models trained et al., 2020; Hospedales et al., 2021), with a large number of for individual tasks in this regime has not been methods and benchmarks proposed in application domains addressed. This paper presents the first investigation ranging from visual recognition systems for robots to identifying into task-level evaluation--a fundamental therapeutic properties of molecules (Xie et al., 2018; step when deploying a model. We measure the accuracy Stanley et al., 2021). of performance estimators in the few-shot setting, consider strategies for model selection, Even though many learning algorithms have been developed and examine the reasons for the failure of evaluators in this area and great efforts have been directed towards usually thought of as being robust. We improving model performance in FSL scenarios, the best conclude that cross-validation with a low number practices for how to evaluate models and design benchmarks of folds is the best choice for directly estimating for this paradigm remain relatively unexplored. In typical the performance of a model, whereas using bootstrapping academic benchmark setups, performance estimation often or cross validation with a large number relies on the existence of test ("query") sets that are several of folds is better for model selection purposes.