Did you know using XGBoost algorithm is one of the popular winning recipe of data science competitions? So, what makes it more powerful than a traditional Random Forest or Neural Network? In the last few years, predictive modeling has become much faster and accurate. I remember spending long hours on feature engineering for improving model by few decimals. A lot of that difficult work, can now be done by using better algorithms.

I used a regularized (LASSO) cox regression to estimate relapse times of patients and used Frank Harrell's bootstrapping method to obtain an optimism-corrected performance estimate of my model. Would be such an optimism corrected b be a better predictor for unseen cases?

Practicing the scientific approach to the data exploration one should know at what extent certain method can be applied. Neural Nets are futile for the stock market's predictions. Monte-Carlo algorithms couldn't offer much help either, and poorly implemented Random Forest algorithm can literally ruin your vacation in South-East Asia, especially if it was implemented by NSA. In this article we will briefly introduce machine learning methods classification and see how they are relevant to the different lines of business. From the cradle to the grave, we are making decisions - from our first decision to attract mother's attention to one of our last decisions when asking the doctor for pain treatment.

LASSICAL unsupervised learning such as Principal Components Analysis (PCA) and Independent Component Analysis (ICA) is useful to design artificial sensory systems and to understand the organization of natural sensory systems. On the artificial side, examples include representations/transforms for image coding [6]-[9] and image categorization [10], [11]. On the natural side, examples include the analysis of visual cortex [12]-[16]. PCA and ICA obtain basis of the space according to different optimization criteria. These basis functions can be interpreted as linear sensors: the projection of data onto these basis represents the response of the set of sensors. PCA defines a sensor hierarchy: for example, an image sensory system made out of principal directions with highest eigenvalues minimizes the image reconstruction error [6], [7]. In ICA, the basis is intended to provide responses as independent as possible, which is equivalent to design a sensory system that maximizes the transmitted information (infomax) [17], [18].

We propose a nonconvex estimator for joint multivariate regression and precision matrix estimation in the high dimensional regime, under sparsity constraints. A gradient descent algorithm with hard thresholding is developed to solve the nonconvex estimator, and it attains a linear rate of convergence to the true regression coefficients and precision matrix simultaneously, up to the statistical error. Compared with existing methods along this line of research, which have little theoretical guarantee, the proposed algorithm not only is computationally much more efficient with provable convergence guarantee, but also attains the optimal finite sample statistical rate up to a logarithmic factor. Thorough experiments on both synthetic and real datasets back up our theory.

We present a novel k-way high-dimensional graphical model called the Generalized Root Model (GRM) that explicitly models dependencies between variable sets of size k > 2---where k = 2 is the standard pairwise graphical model. This model is based on taking the k-th root of the original sufficient statistics of any univariate exponential family with positive sufficient statistics, including the Poisson and exponential distributions. As in the recent work with square root graphical (SQR) models [Inouye et al. 2016]---which was restricted to pairwise dependencies---we give the conditions of the parameters that are needed for normalization using the radial conditionals similar to the pairwise case [Inouye et al. 2016]. In particular, we show that the Poisson GRM has no restrictions on the parameters and the exponential GRM only has a restriction akin to negative definiteness. We develop a simple but general learning algorithm based on L1-regularized node-wise regressions. We also present a general way of numerically approximating the log partition function and associated derivatives of the GRM univariate node conditionals---in contrast to [Inouye et al. 2016], which only provided algorithm for estimating the exponential SQR. To illustrate GRM, we model word counts with a Poisson GRM and show the associated k-sized variable sets. We finish by discussing methods for reducing the parameter space in various situations.

In this article we present an approach that enables joint wind speed and wind power forecasts for a wind park. We combine a multivariate seasonal time varying threshold autoregressive moving average (TVARMA) model with a power threshold generalized autoregressive conditional heteroscedastic (power-TGARCH) model. The modeling framework incorporates diurnal and annual periodicity modeling by periodic B-splines, conditional heteroscedasticity and a complex autoregressive structure with nonlinear impacts. In contrast to usually time-consuming estimation approaches as likelihood estimation, we apply a high-dimensional shrinkage technique. We utilize an iteratively re-weighted least absolute shrinkage and selection operator (lasso) technique. It allows for conditional heteroscedasticity, provides fast computing times and guarantees a parsimonious and regularized specification, even though the parameter space may be vast. We are able to show that our approach provides accurate forecasts of wind power at a turbine-specific level for forecasting horizons of up to 48 hours (short-to medium-term forecasts).

We conduct an axiomatic study of the problem of estimating the strength of a known causal relationship between a pair of variables. We propose that an estimate of causal strength should be based on the conditional distribution of the effect given the cause (and not on the driving distribution of the cause), and study dependence measures on conditional distributions. Shannon capacity, appropriately regularized, emerges as a natural measure under these axioms. We examine the problem of calculating Shannon capacity from the observed samples and propose a novel fixed-$k$ nearest neighbor estimator, and demonstrate its consistency. Finally, we demonstrate an application to single-cell flow-cytometry, where the proposed estimators significantly reduce sample complexity.

Think about the data that you are trying to cluster with. How many dimensions are you using? Are the variables highly related? DO the variables have different standard deviations? For instance, if your data is log-normal then a lot of the cases will be in the low end of the distribution with a few at the high end.

The following publication was presented at the 2014 IEEE International Workshop on Cyber Crime and received the Best Paper Award on 5/18/2014. The original IEEE LaTeX formatted PDF publication can also be downloaded from here: IWCC Combined Clustering. To be successful, cybercriminals must figure out how to scale their scams. They duplicate content on new websites, often staying one step ahead of defenders that shut down past schemes. For some scams, such as phishing and counterfeitgoods shops, the duplicated content remains nearly identical. In others, such as advanced-fee fraud and online Ponzi schemes, the criminal must alter content so that it appears different in order to evade detection by victims and law enforcement. Nevertheless, similarities often remain, in terms of the website structure or content, since making truly unique copies does not scale well. In this paper, we present a novel combined clustering method that links together replicated scam websites, even when the criminal has taken steps to hide connections. We evaluate its performance against two collected datasets of scam websites: fake-escrow services and high-yield investment programs (HYIPs). We find that our method more accurately groups similar websites together than does existing general-purpose consensus clustering methods.