Performance Analysis
Discovering Suspicious Patterns Using a Graph Based Approach
Velampalli, Sirisha (C.R.Rao Advanced Institute of Mathematics, Statistics and Computer Science) | Mookiah, Lenin (Tennessee Technological University) | Eberle, William (Tennessee Technological University)
Recently, there has been much attention on tools and techniques for visualizing and acquiring new knowledge and insights. In the VAST 2018 competition, one of the challenges is to discover the fraudulent group of employees at Kasios, a furniture manufacturing company. In this work, we use a graph-based approach that analyzes the data for suspicious employee activities at Kasios. Graph based approaches enable one to handle rich contextual data and provide a deeper understanding of data due to the ability to discover patterns in databases that are not easily found using traditional query or statistical tools. We focus on graph based knowledge discovery in structural data to mine for interesting patterns and anomalies. Our approach first reports the normative patterns in the data, and then discovers any anomalous patterns associated with the previously discovered patterns. For visualizing the suspicious patterns, we also use the enterprise graph database Neo4j. Neo4j Browser provides a way to visualize graph structures.
SMART: Semantic Malware Attribute Relevance Tagging
Ducau, Felipe N., Rudd, Ethan M., Heppner, Tad M., Long, Alex, Berlin, Konstantin
With the rapid proliferation and increased sophistication of malicious software (malware), detection methods no longer rely only on manually generated signatures but have also incorporated more general approaches like Machine Learning (ML) detection. Although powerful for conviction of malicious artifacts, these methods do not produce any further information about the type of malware that has been detected. In this work, we address the information gap between ML and signature-based detection methods by introducing an ML-based tagging model that generates human interpretable semantic descriptions of malicious software (e.g. file-infector, coin-miner), and argue that for less prevalent malware campaigns these provide potentially more useful and flexible information than malware family names. For this, we first introduce a method for deriving high-level descriptions of malware files from an ensemble of vendor family names. Then we formalize the problem of malware description as a tagging problem and propose a joint embedding deep neural network architecture that can learn to characterize portable executable (PE) files based on static analysis, thus not requiring a dynamic trace to identify behaviors at deployment time. We empirically demonstrate that when evaluated against tags extracted from an ensemble of anti-virus detection names, the proposed tagging model correctly identifies more than 93.7% of eleven possible tag descriptions for a given sample, at a deployable false positive rate (FPR) of 1% per tag. Furthermore, we show that when evaluating this model against ground truth tags derived from the results of dynamic analysis, it correctly predicts 93.5% of the labels for a given sample. These results suggest that an ML tagging model can be effectively deployed alongside a detection model for malware description.
Detecting Slow HTTP POST DoS Attacks Using Netflow Features
Calvert, Chad (Florida Atlantic University) | Kemp, Clifford (Florida Atlantic University) | Khoshgoftaar, Taghi (Florida Atlantic University) | Najafabadi, Maryam (Florida Atlantic University)
Network security is a constant challenge, with new attacks and vulnerabilities being frequently introduced. Application layer Denial of Service (DoS) attacks are a rising attack variant, which inflicts network stress and service interruptions. The implementation of detection and mitigation techniques for such attacks have been a priority for some time, but more sophisticated attack permutations are constantly being introduced, often making prior prevention techniques ineffective. In this work, we focus specifically on the detection of Slow HTTP POST DoS attacks. We execute several Slow HTTP POST attack configurations within a live network environment to represent a real-world attack scenario, with varying levels of severity. For our methodology, we utilize features of network flow (Netflow) traffic to detect these attack configurations. Netflow has proven to be a more scalable solution compared to full packet capture when performing data collection, allowing for near real-time network monitoring. Eight machine learners were implemented to determine which learner would achieve optimal performance metrics when detecting Slow HTTP POST attacks. As our data is very large, we also evaluate the use of data sampling techniques to increase attack detection performance. Overall, our results show a high detection rate when detecting Slow HTTP POST attacks, achieving relatively low false alarm rates.
Detecting the Onset of a Network Layer DoS Attack with a Graph-Based Approach
Paudel, Ramesh (Tennessee Technological University) | Harlan, Peter (Western Kentucky University) | Eberle, William (Tennessee Technological University)
A denial-of-service (DoS) attack is a malicious act with the goal of interrupting the access to a computer network. The result of DoS attack can cause the computers on the network to squander their resources to serve illegitimate requests that result in a disruption of the networkโs services to legitimate users. With a sophisticated DoS attack, it becomes difficult to distinguish malicious requests from legitimate requests. Since a network layer DoS attack can cause interruptions to a network while causing collateral damage, it is vital to understand the measures to mitigate against such attacks. Generally, approaches that implement distribution charts based on statistical analysis or honeypots have been applied to detect a DoS attack. However, this is usually too late, as the damage is already done. We hypothesize in this work that a graph-based approach can provide the capability to identify a DoS attack at its inception. A graph-based approach will also allow us to not only focus on anomalies within an entity (like a computer) but also allow us to analyze the anomalies that exist in an entityโs relationship with other entities, thus providing a rich source of contextual analysis. We demonstrate our proposed approach using a publicly-available dataset.
An Empirical Evaluation of the Effect of Adversarial Labels on Classifier Accuracy Estimation
Clifford, Alexandra (MIT Lincoln Laboratory) | Corey, Cassian (MIT Lincoln Laboratory) | Holodnak, John T. (MIT Lincoln Laboratory)
This paper examines the effect of providing adversarial labels to several algorithms that use noisy labels from multiple experts to estimate classifier accuracy, referred to hereafter as "estimators." We propose four adversary labeling strategies and use experiments on synthetic data to gauge their impact on the estimators. Our results show that even a single adversary can considerably impact the effectiveness of an estimator. In addition, we find that estimators that weight the input of all experts equally tend to be much more affected by the inclusion of adversaries than those that can separately model each expert and that the impact of adversaries is lessened when the experts have higher accuracy.
A Comparison of Three Recommender Strategies for Facilitating Person-Centered Care in Nursing Homes
Martindale, Nathan (Tennessee Technological University) | Gannod, Gerald C. (Tennessee Technological University) | Abbott, Katherine M. (Miami University) | Haitsma, Kimberly Van (Pennsylvania State University)
The Preferences for Everyday Living Inventory (PELI) is a 72-question instrument used for helping nursing homes assess person-centered care. In particular, the approach allows residents to express their preferences for both care and activities in order to provide direct care workers with insights on how to best provide a high-quality living experience. Among the challenges of using the PELI is its length: 72 questions give rise to issues of survey fatigue while also creating a workflow bottleneck for those providing care. In this paper we explore and evaluate the use of three different recommender strategies that we have applied to the PELI. In particular, we present the use of both rule-based and neighborhood-based collaborative filtering in order to make recommendations on which preference questions to present to a resident. We illustrate the approaches by providing a domain-specific example, and then compare the approaches across a number of performance and quality metrics.
Classification of Spontaneous Speech of Individuals with Dementia Based on Automatic Prosody Analysis Using Support Vector Machines (SVM)
Ossewaarde, Roelant (Rijksuniversiteit Groningen) | Jonkers, Roel (Rijksuniversiteit Groningen) | Jalvingh, Fedor (Rijksuniversiteit Groningen) | Bastiaanse, Roelien (Rijksuniversiteit Groningen)
Analysis of spontaneous speech is an important tool for clinical linguists to diagnose various dementia types that affect the language processing areas. Prosody is affected by some dementia types, most notably Parkinson's disease (PD, degradation of voice quality, unstable pitch), Alzheimer's disease (AD, monotonic pitch), and the non-fluent type of Primary Progressive Aphasia (PPA-NF, hesitant, non-fluent speech). Prosodic features can be computed efficiently by software. In this study, we evaluate the performance of a SVM classifier that is trained on prosodic features only. The limitation to only prosody yields baseline results that can be used in a later stage to evaluate the added effect of variables of (morpho) syntax. The goal is to distinguish different dementia types based on the recorded speech. Results show that the classifier can distinguish some dementia types (PPA-NF, AD), but not others (PD, PPA-SD).
Vehicle Shape and Color Classification Using Convolutional Neural Network
Nafzi, Mohamed, Brauckmann, Michael, Glasmachers, Tobias
This paper presents a module of vehicle reidentification based on make/model and color classification. It could be used by the Automated Vehicular Surveillance (AVS) or by the fast analysis of video data. Many of problems, that are related to this topic, had to be addressed. In order to facilitate and accelerate the progress in this subject, we will present our way to collect and to label a large scale data set. We used deeper neural networks in our training. They showed a good classification accuracy. We show the results of make/model and color classification on controlled and video data set. We demonstrate with the help of a developed application the re-identification of vehicles on video images based on make/model and color classification. This work was partially funded under the grant.
Evaluation of Machine Learning Algorithms for Intrusion Detection System
To gauge the accuracy of machine learning models we use various parameters. The metrics used here will be Average Accuracy, False Positive Rates and False Negative Rates. K-Means is excluded from this metric as it is an unsupervised algorithm. Average Accuracy is defined as the ratio of the correctly classified data points to the total number of data points. False Positives are those cases which were supposed to be returned as threats but aren't. False negatives are just the opposite.
Approximating the Ideal Observer and Hotelling Observer for binary signal detection tasks by use of supervised learning methods
Zhou, Weimin, Li, Hua, Anastasio, Mark A.
It is widely accepted that optimization of medical imaging system performance should be guided by task-based measures of image quality (IQ). Task-based measures of IQ quantify the ability of an observer to perform a specific task such as detection or estimation of a signal (e.g., a tumor). For binary signal detection tasks, the Bayesian Ideal Observer (IO) sets an upper limit of observer performance and has been advocated for use in optimizing medical imaging systems and data-acquisition designs. Except in special cases, determination of the IO test statistic is analytically intractable. Markov-chain Monte Carlo (MCMC) techniques can be employed to approximate IO detection performance, but their reported applications have been limited to relatively simple object models. In cases where the IO test statistic is difficult to compute, the Hotelling Observer (HO) can be employed. To compute the HO test statistic, potentially large covariance matrices must be accurately estimated and subsequently inverted, which can present computational challenges. This work investigates supervised learning-based methodologies for approximating the IO and HO test statistics. Convolutional neural networks (CNNs) and single-layer neural networks (SLNNs) are employed to approximate the IO and HO test statistics, respectively. Numerical simulations were conducted for both signal-known-exactly (SKE) and signal-known-statistically (SKS) signal detection tasks. The performances of the supervised learning methods are assessed via receiver operating characteristic (ROC) analysis and the results are compared to those produced by use of traditional numerical methods or analytical calculations when feasible. The potential advantages of the proposed supervised learning approaches for approximating the IO and HO test statistics are discussed.