Mathematical & Statistical Methods
Adaptive Estimation in Structured Factor Models with Applications to Overlapping Clustering
Bing, Xin, Bunea, Florentina, Ning, Yang, Wegkamp, Marten
This work introduces a novel estimation method, called LOVE, of the entries and structure of a loading matrix A in a sparse latent factor model X = AZ + E, for an observable random vector X in Rp, with correlated unobservable factors Z \in RK, with K unknown, and independent noise E. Each row of A is scaled and sparse. In order to identify the loading matrix A, we require the existence of pure variables, which are components of X that are associated, via A, with one and only one latent factor. Despite the fact that the number of factors K, the number of the pure variables, and their location are all unknown, we only require a mild condition on the covariance matrix of Z, and a minimum of only two pure variables per latent factor to show that A is uniquely defined, up to signed permutations. Our proofs for model identifiability are constructive, and lead to our novel estimation method of the number of factors and of the set of pure variables, from a sample of size n of observations on X. This is the first step of our LOVE algorithm, which is optimization-free, and has low computational complexity of order p2. The second step of LOVE is an easily implementable linear program that estimates A. We prove that the resulting estimator is minimax rate optimal up to logarithmic factors in p. The model structure is motivated by the problem of overlapping variable clustering, ubiquitous in data science. We define the population level clusters as groups of those components of X that are associated, via the sparse matrix A, with the same unobservable latent factor, and multi-factor association is allowed. Clusters are respectively anchored by the pure variables, and form overlapping sub-groups of the p-dimensional random vector X. The Latent model approach to OVErlapping clustering is reflected in the name of our algorithm, LOVE.
Computational Linear Algebra for Coders Review - Machine Learning Mastery
Numerical linear algebra is concerned with the practical implications of implementing and executing matrix operations in computers with real data. It is an area that requires some previous experience of linear algebra and is focused on both the performance and precision of the operations. In this post, you will discover the fast.ai Computational Linear Algebra for Coders Review Photo by Ruocaled, some rights reserved. The course "Computational Linear Algebra for Coders" is a free online course provided by fast.ai.
Linear Algebra for Deep Learning - Machine Learning Mastery
Linear algebra is a field of applied mathematics that is a prerequisite to reading and understanding the formal description of deep learning methods, such as in papers and textbooks. Generally, an understanding of linear algebra (or parts thereof) is presented as a prerequisite for machine learning. Although important, this area of mathematics is seldom covered by computer science or software engineering degree programs. In this post, you will discover the crash course in linear algebra for deep learning presented in the de facto textbook on deep learning. Linear Algebra for Deep Learning Photo by Quinn Dombrowski, some rights reserved.
Introduction to Linear Algebra by Gilbert Strang for Machine Learning - Machine Learning Mastery
Concepts in the book are laid out clearly, often with diagrams, but the book moves quickly. The book expects you to keep up or you will fall behind. That being said, each section has an overview of the concepts to be covered and ends with worked examples and quiz questions, the answers to which are available on the book's website. Take my free 7-day email crash course now (with sample code). Click to sign-up and also get a free PDF Ebook version of the course.
Angle-Based Joint and Individual Variation Explained
Feng, Qing, Jiang, Meilei, Hannig, Jan, Marron, J. S.
Integrative analysis of disparate data blocks measured on a common set of experimental subjects is a major challenge in modern data analysis. This data structure naturally motivates the simultaneous exploration of the joint and individual variation within each data block resulting in new insights. For instance, there is a strong desire to integrate the multiple genomic data sets in The Cancer Genome Atlas to characterize the common and also the unique aspects of cancer genetics and cell biology for each source. In this paper we introduce Angle-Based Joint and Individual Variation Explained capturing both joint and individual variation within each data block. This is a major improvement over earlier approaches to this challenge in terms of a new conceptual understanding, much better adaption to data heterogeneity and a fast linear algebra computation. Important mathematical contributions are the use of score subspaces as the principal descriptors of variation structure and the use of perturbation theory as the guide for variation segmentation. This leads to an exploratory data analysis method which is insensitive to the heterogeneity among data blocks and does not require separate normalization. An application to cancer data reveals different behaviors of each type of signal in characterizing tumor subtypes. An application to a mortality data set reveals interesting historical lessons. Software and data are available at GitHub
A Simple Introduction to Complex Stochastic Processes - Part 2
In my first article on this topic (see here) I introduced some of the complex stochastic processes used by Wall Street data scientists, using a simple approach that can be understood by people with no statistics background other than a first course such as stats 101. I defined and illustrated the continuous Brownian motion (the mother of all these stochastic processes) using approximations by discrete random walks, simply re-scaling the X-axis and the Y-axis appropriately, and making time increments (the X-axis) smaller and smaller, so that the limiting process is a time-continuous one. This was done without using any complicated mathematics such as measure theory or filtrations. Here I am going one step further, introducing the integral and derivative of such processes, using rudimentary mathematics. All the articles that I've found on this subject are full of complicated equations and formulas.
Number Theory: Nice Generalization of the Waring Conjecture
The Waring conjecture - actually a problem associated with a number of conjectures, many now being solved - is one of the most fascinating mathematical problems. This article covers new aspects of this problem, with a generalization and new conjectures, some with a tentative solution, and a new framework to tackle the problem. Yet it is written in simple English and accessible to the layman. I also review a number of famous related mathematical conjectures, including one with a $1 million award still waiting for a solution, as well as Goldbach's conjecture, yet unproved as of today. Many curious properties of the Floor function are also listed, and the emphasis is on machine learning and efficient computer-intensive algorithms to try to find surprising results, which then need to be formally proved or disproved.