Performance Analysis
Semi-Supervised Learning for Cancer Detection of Lymph Node Metastases
Jaiswal, Amit Kumar, Panshin, Ivan, Shulkin, Dimitrij, Aneja, Nagender, Abramov, Samuel
Pathologists find tedious to examine the status of the sentinel lymph node on a large number of pathological scans. The examination process of such lymph node which encompasses metastasized cancer cells is histopathologically organized. However, the task of finding metastatic tissues is gradual which is often challenging. In this work, we present our deep convolutional neural network based model validated on PatchCamelyon (PCam) benchmark dataset for fundamental machine learning research in histopathology diagnosis. We find that our proposed model trained with a semi-supervised learning approach by using pseudo labels on PCam-level significantly leads to better performances to strong CNN baseline on the AUC metric.
Nested Cavity Classifier: performance and remedy
Mustafa, Waleed A., Yousef, Waleed A.
Many articles and books considered the assessment of classifiers using simulated and real-world datasets (e.g., (Raudys and Pikelis, 1980; Efron and Tibshirani, 1997; Hastie et al., 2001)); but none of them considered a systematic assessment of NCC. However, Inselberg and Avidan (2000) compared NCC with other classifiers only on few real high-dimensional datasets; that study mentioned the superiority of NCC over other classifiers. NCC, as described below, builds decision regions geometrically using convex hulls. This partitioning mechanism has a drawback on the performance of the NCC (as explained in Section 3). NCC classifies any testing observation--regardless to its class, whether "class 1" or "class 2"--as class, say, "class 2" as long as it does not lie inside the range of the training data set; i.e., within the minimum and maximum values of each dimension. Since this is not always true, the present article proposes combining NCC with LDA to classify observations outside the range of the training set.
MercurialMonkey/Harvard-University-Capstone-Project-Data-Science
I have submitted my own project using a dataset of my choosing. My project has been reviewed both by my peers and the professor. I chose to work with Credit Card Fraud Detection, It is important that credit card companies are able to recognize fraudulent credit card transactions so that customers are not charged for items that they did not purchase. The datasets contains transactions made by credit cards in September 2013 by european cardholders. Due to imbalancing nature of the data, many observations could be predicted as False Negative, in this case Legal Transactions instead of Fraudolent Transaction.
Comparing Classifiers: Decision Trees, K-NN & Naive Bayes
A myriad of options exist for classification. That said, three popular classification methods-- Decision Trees, k-NN & Naive Bayes--can be tweaked for practically every situation. Naive Bayes and K-NN, are both examples of supervised learning (where the data comes already labeled). Decision trees are easy to use for small amounts of classes. If you're trying to decide between the three, your best option is to take all three for a test drive on your data, and see which produces the best results.
Proof-of-concept system uses smart speakers to catch signs of cardiac arrest
In an effort to tackle in-home cardiac arrest, University of Washington researchers have devised a novel contactless system that uses smartphones or voice-based personal assistants to identify telltale breathing patterns that accompany an attack. The proof-of-concept strategy, described in an NPJ Digital Medicine paper published this morning, involved a supervised machine learning model called a support-vector machine that was trained for use in the bedroom, a controlled environment in which the majority of in-home cardiac arrests occur. "Sometimes reported as'gasping' breaths, agonal respirations may hold potential as an audible diagnostic biomarker, particularly in unwitnessed cardiac arrests that occur in a private residence, the location of [two-thirds] of all [out-of-hospital cardiac arrests]," the researchers wrote. "The widespread adoption of smartphones and smart speakers (projected to be in 75% of US households by 2020) presents a unique opportunity to identify this audible biomarker and connect unwitnessed cardiac arrest victims to emergency medical services (EMS) or others who can administer cardiopulmonary resuscitation." Cross-validation analysis of the trained classifier yielded an overall sensitivity and specificity of 97.24% and 99.51%.
Approximating Interactive Human Evaluation with Self-Play for Open-Domain Dialog Systems
Ghandeharioun, Asma, Shen, Judy Hanwen, Jaques, Natasha, Ferguson, Craig, Jones, Noah, Lapedriza, Agata, Picard, Rosalind
Building an open-domain conversational agent is a challenging problem. Current evaluation methods, mostly post-hoc judgments of single-turn evaluation, do not capture conversation quality in a realistic interactive context. In this paper, we investigate interactive human evaluation and provide evidence for its necessity; we then introduce a novel, model-agnostic, and dataset-agnostic method to approximate it. In particular, we propose a self-play scenario where the dialog system talks to itself and we calculate a combination of proxies such as sentiment and semantic coherence on the conversation trajectory. We show that this metric is capable of capturing the human-rated quality of a dialog model better than any automated metric known to-date, achieving a significant Pearson correlation (r>.7, p<.05). To investigate the strengths of this novel metric and interactive evaluation in comparison to state-of-the-art metrics and one-turn evaluation, we perform extended experiments with a set of models, including several that make novel improvements to recent hierarchical dialog generation architectures through sentiment and semantic knowledge distillation on the utterance level. Finally, we open-source the interactive evaluation platform we built and the dataset we collected to allow researchers to efficiently deploy and evaluate generative dialog models.
Joint Detection of Malicious Domains and Infected Clients
Prasse, Paul, Knaebel, Rene, Machlica, Lukas, Pevny, Tomas, Scheffer, Tobias
Detection of malware-infected computers and detection of malicious web domains based on their encrypted HTTPS traffic are challenging problems, because only addresses, timestamps, and data volumes are observable. The detection problems are coupled, because infected clients tend to interact with malicious domains. Traffic data can be collected at a large scale, and antivirus tools can be used to identify infected clients in retrospect. Domains, by contrast, have to be labeled individually after forensic analysis. We explore transfer learning based on sluice networks; this allows the detection models to bootstrap each other. In a large-scale experimental study, we find that the model outperforms known reference models and detects previously unknown malware, previously unknown malware families, and previously unknown malicious domains.
A Novel Deep Transfer Learning Method for Detection of Myocardial Infarction
Myocardial infarction (MI), also known as a cardiac attack, is one of the common cardiac disorders occurs when one or more coronary arteries are blocked. Hence, early detection of MI is critical for the reduction of the rising of the death rate. The cardiologists use the electrocardiogram (ECG) as a diagnostic tool to monitor and reveal the MI signals. However, all the MI signals are not constant and noisy, so it is tough to detect or observe these signals manually. Several computer-aided diagnosis systems (CADs) have been suggested to solve these difficulties. In this paper, we have proposed an effective CAD system to detect MI signals using the two-dimensional convolution neural network (CNN). In this study, we have employed two ways of the transfer learning technique to retrain the pre-trained VGG-Net and obtained two new networks VGG-MI1 and VGG-MI2. Moreover, the heartbeat data augmentation techniques are employed to increase the classification performance. We have utilized two-second ECG signals from the PTB database, which has been widely employed in MI detection studies. In case of using VGG-MI1, we achieved an accuracy, sensitivity, and specificity of 99.02%, 98.76%, and 99.17% respectively and we achieved an accuracy of 99.22%, a sensitivity of 99.15%, and a specificity of 99.49% when using VGG-MI2. Results showed that the proposed algorithm is more efficient than the state-of-the-art methods in terms of accuracy sensitivity, and specificity. Finally, the proposed algorithm can assist the specialists to detect the MI signals more precisely.
Unsupervised Ensemble Classification with Dependent Data
Traganitis, Panagiotis A., Giannakis, Georgios B.
Ensemble learning, the machine learning paradigm where multiple algorithms are combined, has exhibited promising perfomance in a variety of tasks. The present work focuses on unsupervised ensemble classification. The term unsupervised refers to the ensemble combiner who has no knowledge of the ground-truth labels that each classifier has been trained on. While most prior works on unsupervised ensemble classification are designed for independent and identically distributed (i.i.d.) data, the present work introduces an unsupervised scheme for learning from ensembles of classifiers in the presence of data dependencies. Two types of data dependencies are considered: sequential data and networked data whose dependencies are captured by a graph. Moment matching and Expectation Maximization algorithms are developed for the aforementioned cases, and their performance is evaluated on synthetic and real datasets.
On Tree-based Methods for Similarity Learning
Clémençon, Stéphan, Vogel, Robin
In many situations, the choice of an adequate similarity measure or metric on the feature space dramatically determines the performance of machine learning methods. Building automatically such measures is the specific purpose of metric/similarity learning. In Vogel et al. (2018), similarity learning is formulated as a pairwise bipartite ranking problem: ideally, the larger the probability that two observations in the feature space belong to the same class (or share the same label), the higher the similarity measure between them. From this perspective, the ROC curve is an appropriate performance criterion and it is the goal of this article to extend recursive tree-based ROC optimization techniques in order to propose efficient similarity learning algorithms. The validity of such iterative partitioning procedures in the pairwise setting is established by means of results pertaining to the theory of U-processes and from a practical angle, it is discussed at length how to implement them by means of splitting rules specifically tailored to the similarity learning task. Beyond these theoretical/methodological contributions, numerical experiments are displayed and provide strong empirical evidence of the performance of the algorithmic approaches we propose.