Performance Analysis
Identifying Transients in the Dark Energy Survey using Convolutional Neural Networks
Ayyar, Venkitesh, Knop, Robert Jr., Awbrey, Autumn, Andersen, Alexis, Nugent, Peter
The ability to discover new transient candidates via image differencing without direct human intervention is an important task in observational astronomy. For these kind of image classification problems, machine Learning techniques such as Convolutional Neural Networks (CNNs) have shown remarkable success. In this work, we present the results of an automated transient candidate identification on images with CNNs for an extant dataset from the Dark Energy Survey Supernova program (DES-SN), whose main focus was on using Type Ia supernovae for cosmology. By performing an architecture search of CNNs, we identify networks that efficiently select non-artifacts (e.g. The CNNs also help us identify a subset of mislabeled images. Performing a relabeling of the images in this subset, the resulting classification with CNNs is significantly better than previous results, lowering the false positive rate by 27% at a fixed missed detection rate of 0.05. INTRODUCTION A major aspect of observational astronomy is the "survey" which involves the wholesale mapping of various regions of the sky to create catalogs which are subsequently mined for scientifically important astronomical objects. We refer to a transient candidate as the detection on a single image of a new or varying source with respect to a previously taken reference image, regardless of its astrophysical nature since at this stage its classification is unknown and will remain so until further data is taken (spectroscopy and/or additional photometry). Some examples of such transient candidates are solar system objects, supernovae, active galactic nuclei, variable stars, and neutron star mergers, etc. Since some of these events are quite rare and will fade rapidly, it is often important to trigger follow-up observations immediately to glean their underlying nature and discover new physics. Hence, identifying transient candidates in images quickly and efficiently is very important so as not to waste precious, and expensive, follow-up resources. For many years this process was conducted by manual inspection of images by humans.
Overview of Machine Learning
In layman's terms, machine learning is to allow computers to learn automatically from data to obtain certain knowledge. As a discipline, machine learning usually refers to a type of problem and the method to solve this type of problem, that is, how to find the law from the observation data, and use the learned law to predict the unknown or unobservable data. In the early engineering field, machine learning is often called pattern recognition, but pattern recognition is more biased towards specific application tasks, such as optical character recognition, speech recognition, and face recognition. The characteristic of these tasks is that for us humans, these tasks are easy to complete, but we do not know how we do it, so it is difficult to manually design a computer program to complete these tasks. A feasible method is to design an algorithm that allows the computer to learn the rules from the labeled samples and use it to complete various recognition tasks. With the increasing application of machine learning technology, the concept of machine learning is now gradually replacing pattern recognition, becoming the general term for this type of problem and its solutions. Taking handwritten digit recognition as an example, we need to allow the computer to automatically recognize handwritten digits. Handwritten digit recognition is a classic machine learning task, which is simple for humans, but very difficult for computers. It is difficult for us to summarize the handwriting characteristics of each digit, or the rules for distinguishing different digits, so designing a set of recognition algorithms is an almost impossible task. In real life, many problems are similar to those of handwritten number recognition, such as object recognition and speech recognition. For this kind of problem, we don't know how to design a computer program to solve it. Even if it can be realized by some heuristic rules, the process is extremely complicated. Therefore, people began to try another way of thinking, that is, let the computer see a large number of samples, and learn some experience from them, and then use these experiences to identify new samples. To recognize handwritten digits, first manually annotate a large number of handwritten digital images (that is, each image is manually marked with what number it is), these images are used as training data, and then a set of models are automatically generated through the learning algorithm, and rely on it. This method of learning through data is called the method of machine learning. First, we use a life example to introduce some basic concepts in machine learning: samples, features, labels, models, learning algorithms, etc. Suppose we want to buy mangoes in the market, but we have no previous experience in selecting mangoes, how can we obtain this knowledge through learning? First, we randomly select some mangoes from the market and list the characteristics of each mango.
Learn How Amazon SageMaker Clarify Helps Detect Bias
Bias detection in data and model outcomes is a fundamental requirement for building responsible artificial intelligence (AI) and machine learning (ML) models. Unfortunately, detecting bias isn't an easy task for the vast majority of practitioners due to the large number of ways in which it can be measured and different factors that can contribute to a biased outcome. For instance, an imbalanced sampling of the training data may result in a model that is less accurate for certain subsets of the data. Bias may also be introduced by the ML algorithm itself--even with a well-balanced training dataset, the outcomes might favor certain subsets of the data as compared to the others. To detect bias, you must have a thorough understanding of different types of bias and the corresponding bias metrics. For example, at the time of this writing, Amazon SageMaker Clarify offers 21 different metrics to choose from.
Ensembling Neural Networks for Improved Prediction and Privacy in Early Diagnosis of Sepsis
Schamoni, Shigehiko, Hagmann, Michael, Riezler, Stefan
Ensembling neural networks is a long-standing technique for improving the generalization error of neural networks by combining networks with orthogonal properties via a committee decision. We show that this technique is an ideal fit for machine learning on medical data: First, ensembles are amenable to parallel and asynchronous learning, thus enabling efficient training of patient-specific component neural networks. Second, building on the idea of minimizing generalization error by selecting uncorrelated patient-specific networks, we show that one can build an ensemble of a few selected patient-specific models that outperforms a single model trained on much larger pooled datasets. Third, the non-iterative ensemble combination step is an optimal low-dimensional entry point to apply output perturbation to guarantee the privacy of the patient-specific networks. We exemplify our framework of differentially private ensembles on the task of early prediction of sepsis, using real-life intensive care unit data labeled by clinical experts.
A preprocessing perspective for quantum machine learning classification advantage using NISQ algorithms
Mancilla, Javier, Pere, Christophe
Machine Learning (ML) is a predominant tool nowadays to solve several challenges in different industries, such as credit scoring Provenzano et al. [2020], fraud analysis Tiwari et al. [2021], product recommendation Rohde et al. [2018], and demand forecasting Masini et al. [2020], among other extensively explored use cases. Under this premise, the research of the quantum computing properties applied to ML has expanded rapidly in recent years since a proven advantage could be a highly useful cross-industry. The recent progress of these explorations in Quantum Machine Learning (QML) Mishra et al. [2021] put a spotlight on quantum technology, introducing a challenge to determine if QML will provide an advantage over classical machine learning or not. The actual devices are noisy, meaning that the depth or consecutive gate operations are limited [Ristรจ et al., 2013, Burnett et al., 2019, Wang et al., 2021]. Qubits will lose their entanglement and so, the information. These devices make up the NISQ era Preskill [2018] and limit the use of quantum algorithms or hybrid algorithms to be useful Callison and Chancellor [2022].
Predicting the Stability of Hierarchical Triple Systems with Convolutional Neural Networks
Lalande, Florian, Trani, Alessandro Alberto
Understanding the long-term evolution of hierarchical triple systems is challenging due to its inherent chaotic nature, and it requires computationally expensive simulations. Here we propose a convolutional neural network model to predict the stability of hierarchical triples by looking at their evolution during the first $5 \times 10^5$ inner binary orbits. We employ the regularized few-body code TSUNAMI to simulate $5\times 10^6$ hierarchical triples, from which we generate a large training and test dataset. We develop twelve different network configurations that use different combinations of the triples' orbital elements and compare their performances. Our best model uses 6 time-series, namely, the semimajor axes ratio, the inner and outer eccentricities, the mutual inclination and the arguments of pericenter. This model achieves an area under the curve of over $95\%$ and informs of the relevant parameters to study triple systems stability. All trained models are made publicly available, allowing to predict the stability of hierarchical triple systems $200$ times faster than pure $N$-body methods.
Towards Multidimensional Textural Perception and Classification Through Whisker
Routray, Prasanna Kumar, Kanade, Aditya Sanjiv, Pounds, Pauline, Muniyandi, Manivannan
Texture-based studies and designs have been in focus recently. Whisker-based multidimensional surface texture data is missing in the literature. This data is critical for robotics and machine perception algorithms in the classification and regression of textural surfaces. In this study, we present a novel sensor design to acquire multidimensional texture information. The surface texture's roughness and hardness were measured experimentally using sweeping and dabbing. Three machine learning models (SVM, RF, and MLP) showed excellent classification accuracy for the roughness and hardness of surface textures. We show that the combination of pressure and accelerometer data, collected from a standard machined specimen using the whisker sensor, improves classification accuracy. Further, we experimentally validate that the sensor can classify texture with roughness depths as low as $2.5\mu m$ at an accuracy of $90\%$ or more and segregate materials based on their roughness and hardness. We present a novel metric to consider while designing a whisker sensor to guarantee the quality of texture data acquisition beforehand. The machine learning model performance was validated against the data collected from the laser sensor from the same set of surface textures. As part of our work, we are releasing two-dimensional texture data: roughness and hardness to the research community.
Negation detection in Dutch clinical texts: an evaluation of rule-based and machine learning methods
van Es, Bram, Reteig, Leon C., Tan, Sander C., Schraagen, Marijn, Hemker, Myrthe M., Arends, Sebastiaan R. S., Rios, Miguel A. R., Haitjema, Saskia
As structured data are often insufficient, labels need to be extracted from free text in electronic health records when developing models for clinical information retrieval and decision support systems. One of the most important contextual properties in clinical text is negation, which indicates the absence of findings. We aimed to improve large scale extraction of labels by comparing three methods for negation detection in Dutch clinical notes. We used the Erasmus Medical Center Dutch Clinical Corpus to compare a rule-based method based on ContextD, a biLSTM model using MedCAT and (finetuned) RoBERTa-based models. We found that both the biLSTM and RoBERTa models consistently outperform the rule-based model in terms of F1 score, precision and recall. In addition, we systematically categorized the classification errors for each model, which can be used to further improve model performance in particular applications. Combining the three models naively was not beneficial in terms of performance. We conclude that the biLSTM and RoBERTa-based models in particular are highly accurate accurate in detecting clinical negations, but that ultimately all three approaches can be viable depending on the use case at hand.
TypoSwype: An Imaging Approach to Detect Typo-Squatting
Lee, Joon Sern, David, Yam Gui Peng
Typo-squatting domains are a common cyber-attack technique. It involves utilising domain names, that exploit possible typographical errors of commonly visited domains, to carry out malicious activities such as phishing, malware installation, etc. Current approaches typically revolve around string comparison algorithms like the Demaru-Levenschtein Distance (DLD) algorithm. Such techniques do not take into account keyboard distance, which researchers find to have a strong correlation with typical typographical errors and are trying to take account of. In this paper, we present the TypoSwype framework which converts strings to images that take into account keyboard location innately. We also show how modern state of the art image recognition techniques involving Convolutional Neural Networks, trained via either Triplet Loss or NT-Xent Loss, can be applied to learn a mapping to a lower dimensional space where distances correspond to image, and equivalently, textual similarity. Finally, we also demonstrate our method's ability to improve typo-squatting detection over the widely used DLD algorithm, while maintaining the classification accuracy as to which domain the input domain was attempting to typo-squat.
Part-level Action Parsing via a Pose-guided Coarse-to-Fine Framework
Chen, Xiaodong, Liu, Xinchen, Liu, Wu, Liu, Kun, Wu, Dong, Zhang, Yongdong, Mei, Tao
Action recognition from videos, i.e., classifying a video into one of the pre-defined action types, has been a popular topic in the communities of artificial intelligence, multimedia, and signal processing. However, existing methods usually consider an input video as a whole and learn models, e.g., Convolutional Neural Networks (CNNs), with coarse video-level class labels. These methods can only output an action class for the video, but cannot provide fine-grained and explainable cues to answer why the video shows a specific action. Therefore, researchers start to focus on a new task, Part-level Action Parsing (PAP), which aims to not only predict the video-level action but also recognize the frame-level fine-grained actions or interactions of body parts for each person in the video. To this end, we propose a coarse-to-fine framework for this challenging task. In particular, our framework first predicts the video-level class of the input video, then localizes the body parts and predicts the part-level action. Moreover, to balance the accuracy and computation in part-level action parsing, we propose to recognize the part-level actions by segment-level features. Furthermore, to overcome the ambiguity of body parts, we propose a pose-guided positional embedding method to accurately localize body parts. Through comprehensive experiments on a large-scale dataset, i.e., Kinetics-TPS, our framework achieves state-of-the-art performance and outperforms existing methods over a 31.10% ROC score.