We study the robustness of active learning (AL) algorithms against prior misspecification: whether an algorithm achieves similar performance using a perturbed prior as compared to using the true prior. In both the average and worst cases of the maximum coverage setting, we prove that all $\alpha$-approximate algorithms are robust (i.e., near $\alpha$-approximate) if the utility is Lipschitz continuous in the prior. We further show that robustness may not be achieved if the utility is non-Lipschitz. This suggests we should use a Lipschitz utility for AL if robustness is required. For the minimum cost setting, we can also obtain a robustness result for approximate AL algorithms. Our results imply that many commonly used AL algorithms are robust against perturbed priors. We then propose the use of a mixture prior to alleviate the problem of prior misspecification. We analyze the robustness of the uniform mixture prior and show experimentally that it performs reasonably well in practice.

Modal regression estimates the local modes of the distribution of $Y$ given $X=x$, instead of the mean, as in the usual regression sense, and can hence reveal important structure missed by usual regression methods. We study a simple nonparametric method for modal regression, based on a kernel density estimate (KDE) of the joint distribution of $Y$ and $X$. We derive asymptotic error bounds for this method, and propose techniques for constructing confidence sets and prediction sets. The latter is used to select the smoothing bandwidth of the underlying KDE. The idea behind modal regression is connected to many others, such as mixture regression and density ridge estimation, and we discuss these ties as well.

Prerequisites: No prerequisites, knowledge of some undergraduate level mathematics would help but is not mandatory. Working knowledge of Python would be helpful if you want to run the source code that is provided. Taught by a Stanford-educated, ex-Googler and an IIT, IIM - educated ex-Flipkart lead analyst. This team has decades of practical experience in quant trading, analytics and e-commerce. The course is shy but confident: It is authoritative, drawn from decades of practical experience -but shies away from needlessly complicating stuff.

Both R & Python should be measured based on their effectiveness in advanced analytics & data science. Initially, as a new comer in data science field we spend good amount of time to understand the pros and cons of these two. I too carried out this study solely for "self" to decide which tool should i pick to get in depth of data science. Eventually, i have started realizing that both (R & Python) has its space of mastery along with their broad support to data science. Now, when you start getting into space of predictive modeling, machine learning and mathematical modeling, Python can give a easy hand.

What are some recent advances in non-convex optimization research? Non-convex optimization is now ubiquitous in machine learning. While previously, the focus was on convex relaxation methods, now the emphasis is on being able to solve non-convex problems directly. It is not possible to find the global optimum of every non-convex problem due to NP-hardness barrier. An alternate approach is: when can it be solved efficiently (preferably in low order polynomial time).

Thanks to Rong for the very nice blog post describing critical points of nonconvex functions and how to avoid them. I'd like to follow up on his post to highlight a fact that is not widely appreciated in nonlinear optimization. Though we often teach the contrary in our intro courses, it is in fact super hard to converge to a saddle point. If you move ever so slightly you fall off the saddle). Even simple algorithms like gradient descent with constant step sizes can't converge to saddle points unless you try really hard.

Brain decoding is a popular multivariate approach for hypothesis testing in neuroimaging. It is well known that the brain maps derived from weights of linear classifiers are hard to interpret because of high correlations between predictors, low signal to noise ratios, and the high dimensionality of neuroimaging data. Therefore, improving the interpretability of brain decoding approaches is of primary interest in many neuroimaging studies. Despite extensive studies of this type, at present, there is no formal definition for interpretability of multivariate brain maps. As a consequence, there is no quantitative measure for evaluating the interpretability of different brain decoding methods. In this paper, first, we present a theoretical definition of interpretability in brain decoding; we show that the interpretability of multivariate brain maps can be decomposed into their reproducibility and representativeness. Second, as an application of the proposed theoretical definition, we formalize a heuristic method for approximating the interpretability of multivariate brain maps in a binary magnetoencephalography (MEG) decoding scenario. Third, we propose to combine the approximated interpretability and the performance of the brain decoding model into a new multi-objective criterion for model selection. Our results for the MEG data show that optimizing the hyper-parameters of the regularized linear classifier based on the proposed criterion results in more informative multivariate brain maps. More importantly, the presented definition provides the theoretical background for quantitative evaluation of interpretability, and hence, facilitates the development of more effective brain decoding algorithms in the future.

Asymptotic factorizations for the small-ball probability (SmBP) of a Hilbert valued random element $X$ are rigorously established and discussed. In particular, given the first $d$ principal components (PCs) and as the radius $\varepsilon$ of the ball tends to zero, the SmBP is asymptotically proportional to (a) the joint density of the first $d$ PCs, (b) the volume of the $d$-dimensional ball with radius $\varepsilon$, and (c) a correction factor weighting the use of a truncated version of the process expansion. Moreover, under suitable assumptions on the spectrum of the covariance operator of $X$ and as $d$ diverges to infinity when $\varepsilon$ vanishes, some simplifications occur. In particular, the SmBP factorizes asymptotically as the product of the joint density of the first $d$ PCs and a pure volume parameter. All the provided factorizations allow to define a surrogate intensity of the SmBP that, in some cases, leads to a genuine intensity. To operationalize the stated results, a non-parametric estimator for the surrogate intensity is introduced and it is proved that the use of estimated PCs, instead of the true ones, does not affect the rate of convergence. Finally, as an illustration, simulations in controlled frameworks are provided.

In statistical genetics an important task involves building predictive models for the genotype-phenotype relationships and thus attribute a proportion of the total phenotypic variance to the variation in genotypes. Numerous models have been proposed to incorporate additive genetic effects into models for prediction or association. However, there is a scarcity of models that can adequately account for gene by gene or other forms of genetical interactions. In addition, there is an increased interest in using marker annotations in genome-wide prediction and association. In this paper, we discuss an hybrid modeling methodology which combines the parametric mixed modeling approach and the non-parametric rule ensembles. This approach gives us a flexible class of models that can be used to capture additive, locally epistatic genetic effects, gene x background interactions and allows us to incorporate one or more annotations into the genomic selection or association models. We use benchmark data sets covering a range of organisms and traits in addition to simulated data sets to illustrate the strengths of this approach. The improvement of model accuracies and association results suggest that a part of the "missing heritability" in complex traits can be captured by modeling local epistasis.

Jaguar said when it unveiled its new F-Type SVR that the coupe could hit 200 mph, but Fast and Furious actress Michelle Rodriguez proved them wrong. In her hands, the F-Type SVR topped out at 201 mph, which was both the fastest the coupe has driven and the highest top speed she has achieved on her own. Previously, she says in the video, her personal top speed record was about 140 mph. Rodriguez and a professional driver riding shotgun took the orange coupe up past the magic 200-mph mark on a closed Nevada highway. While Rodriguez has a lengthy Hollywood resume, she is probably best known to car enthusiasts for her role as Leticia "Letty" Ortiz in the Fast and the Furious franchise, and she'll reprise the role in the Furious 8 installment that's being filmed right now.