Statistical Learning
Distributed Cartesian Power Graph Segmentation for Graphon Estimation
Wei, Shitong, Madrid-Padilla, Oscar Hernan, Sharpnack, James
We study an extention of total variation denoising over images to over Cartesian power graphs and its applications to estimating non-parametric network models. The power graph fused lasso (PGFL) segments a matrix by exploiting a known graphical structure, $G$, over the rows and columns. Our main results shows that for any connected graph, under subGaussian noise, the PGFL achieves the same mean-square error rate as 2D total variation denoising for signals of bounded variation. We study the use of the PGFL for denoising an observed network $H$, where we learn the graph $G$ as the $K$-nearest neighborhood graph of an estimated metric over the vertices. We provide theoretical and empirical results for estimating graphons, a non-parametric exchangeable network model, and compare to the state of the art graphon estimation methods.
A Comparative Study of Classification Techniques in Data Mining Algorithms
Classification is used to find out in which group each data instance is related within a given dataset. It is used for classifying data into different classes according to some constrains. Several major kinds of classification algorithms including C4.5, ID3, k-nearest neighbor classifier, Naive Bayes, SVM, and ANN are used for classification. Generally a classification technique follows three approaches Statistical, Machine Learning and Neural Network for classification. While considering these approaches this paper provides an inclusive survey of different classification algorithms and their features and limitations.
How To Build a Basic Website Based on Real-Time Predictions
The first model (using a logistic regression classifier) gave me an accuracy response of more than 80 percent and an AUC of 84 percent. The results were also meaningful - they showed that the probability of an account belonging to a company increases with the number of Tweets, likes, or followings. On the other side, if you've linked to an Instagram or LinkedIn account in your bio, you are more likely to be a human.
GIS and Innovations in Machine Learning GIS Lounge
Machine learning or artificial techniques has been rapidly transforming many areas related to GIS and spatial applications. One example is using web GIS with machine learning algorithms to predict or forecast the success of given potential hotel sites. This has been created into an application called Hotel Location Selection and Analyzing Toolset (HoLSAT). Techniques such as pursuit regression, artificial neural network, and support vector regression allow the tool to determine beneficial hotel locations based on a variety of criteria.[1] Determining where landslides might occur is also another potential application for machine learning techniques such as decision trees (DT) and adaptive neuro-fuzzy inference methods.
How to Transform Data to Better Fit The Normal Distribution
A large portion of the field of statistics is concerned with methods that assume a Gaussian distribution: the familiar bell curve. If your data has a Gaussian distribution, the parametric methods are powerful and well understood. This gives some incentive to use them if possible. Even if your data does not have a Gaussian distribution. It is possible that your data does not look Gaussian or fails a normality test, but can be transformed to make it fit a Gaussian distribution.
A Beginner's Guide to EDA with Linear Regression -- Part 3
Mother Race -- but what we are seeing at X-Axis here is a bunch of variables. When you look closer you would notice that each variable seems to be representing each unique value of Mother Race variable. Linear Regression function'lm' in R automatically transforms a categorical variable into something called'dummy' variables. It will create a column for each categorical value (e.g. Japanese) and have a value of 0 or 1 based on whether a given row matches a given column (e.g.
Forming IDEAS Interactive Data Exploration & Analysis System
Bridges, Robert A., Vincent, Maria A., Huffer, Kelly M. T., Goodall, John R., Jamieson, Jessie D., Burch, Zachary
Modern cyber security operations collect an enormous amount of logging and alerting data. While analysts have the ability to query and compute simple statistics and plots from their data, current analytical tools are too simple to admit deep understanding. To detect advanced and novel attacks, analysts turn to manual investigations. While commonplace, current investigations are time-consuming, intuition-based, and proving insufficient. Our hypothesis is that arming the analyst with easy-to-use data science tools will increase their work efficiency, provide them with the ability to resolve hypotheses with scientific inquiry of their data, and support their decisions with evidence over intuition. To this end, we present our work to build IDEAS (Interactive Data Exploration and Analysis System). We present three real-world use-cases that drive the system design from the algorithmic capabilities to the user interface. Finally, a modular and scalable software architecture is discussed along with plans for our pilot deployment with a security operation command.
Searching for a Single Community in a Graph
Ray, Avik, Sanghavi, Sujay, Shakkottai, Sanjay
In standard graph clustering/community detection, one is interested in partitioning the graph into more densely connected subsets of nodes. In contrast, the "search" problem of this paper aims to only find the nodes in a "single" such community, the target, out of the many communities that may exist. To do so , we are given suitable side information about the target; for example, a very small number of nodes from the target are labeled as such. We consider a general yet simple notion of side information: all nodes are assumed to have random weights, with nodes in the target having higher weights on average. Given these weights and the graph, we develop a variant of the method of moments that identifies nodes in the target more reliably, and with lower computation, than generic community detection methods that do not use side information and partition the entire graph. Our empirical results show significant gains in runtime, and also gains in accuracy over other graph clustering algorithms.
Measuring Item Similarity in Introductory Programming: Python and Robot Programming Case Studies
Pelánek, Radek, Effenberger, Tomáš, Vaněk, Matěj, Sassmann, Vojtěch, Gmiterko, Dominik
A personalized learning system needs a large pool of items for learners to solve. When working with a large pool of items, it is useful to measure the similarity of items. We outline a general approach to measuring the similarity of items and discuss specific measures for items used in introductory programming. Evaluation of quality of similarity measures is difficult. To this end, we propose an evaluation approach utilizing three levels of abstraction. We illustrate our approach to measuring similarity and provide evaluation using items from three diverse programming environments.