We develop the Sparse VAE, a deep generative model for unsupervised representation learning on high-dimensional data. Given a dataset of observations, the Sparse VAE learns a set of latent factors that captures its distribution. The model is sparse in the sense that each feature of the dataset (i.e., each dimension) depends on a small subset of the latent factors. As examples, in ratings data each movie is only described by a few genres; in text data each word is only applicable to a few topics; in genomics, each gene is active in only a few biological processes. We first show that the Sparse VAE is identifiable: given data drawn from the model, there exists a uniquely optimal set of factors. (In contrast, most VAE-based models are not identifiable.) The key assumption behind Sparse-VAE identifiability is the existence of "anchor features", where for each factor there exists a feature that depends only on that factor. Importantly, the anchor features do not need to be known in advance. We then show how to fit the Sparse VAE with variational EM. Finally, we empirically study the Sparse VAE with both simulated and real data. We find that it recovers meaningful latent factors and has smaller heldout reconstruction error than related methods.

Machine learning training methods depend plentifully and intricately on hyperparameters, motivating automated strategies for their optimisation. Many existing algorithms restart training for each new hyperparameter choice, at considerable computational cost. Some hypergradient-based one-pass methods exist, but these either cannot be applied to arbitrary optimiser hyperparameters (such as learning rates and momenta) or take several times longer to train than their base models. We extend these existing methods to develop an approximate hypergradient-based hyperparameter optimiser which is applicable to any continuous hyperparameter appearing in a differentiable model weight update, yet requires only one training episode, with no restarts. We also provide a motivating argument for convergence to the true hypergradient, and perform tractable gradient-based optimisation of independent learning rates for each model parameter. Our method performs competitively from varied random hyperparameter initialisations on several UCI datasets and Fashion-MNIST (using a one-layer MLP), Penn Treebank (using an LSTM) and CIFAR-10 (using a ResNet-18), in time only 2-3x greater than vanilla training.

This is a course on Machine Learning, Deep Learning (Tensorflow PyTorch) and Bayesian Learning (yes all 3 topics in one place!!!). We start off by analysing data using pandas, and implementing some algorithms from scratch using Numpy. These algorithms include linear regression, Classification and Regression Trees (CART), Random Forest and Gradient Boosted Trees. We start off using TensorFlow for our Deep Learning lessons. This will include Feed Forward Networks, Convolutional Neural Nets (CNNs) and Recurrent Neural Nets (RNNs).

It's been a hot, hot year in the world of data, machine learning and AI. Just when you thought it couldn't grow any more explosively, the data/AI landscape just did: rapid pace of company creation, exciting new product and project launches, a deluge of VC financings, unicorn creation, IPOs, etc. It has also been a year of multiple threads and stories intertwining. One story has been the maturation of the ecosystem, with market leaders reaching large scale and ramping up their ambitions for global market domination, in particular through increasingly broad product offerings. Some of those companies, such as Snowflake, have been thriving in public markets (see our MAD Public Company Index), and a number of others (Databricks, Dataiku, Datarobot, etc.) have raised very large (or in the case of Databricks, gigantic) rounds at multi-billion valuations and are knocking on the IPO door (see our Emerging MAD company Index – both indexes will be updated soon). But at the other end of the spectrum, this year has also seen the rapid emergence of a whole new generation of data and ML startups. Whether they were founded a few years or a few months ago, many experienced a growth spurt in the last year or so. As we will discuss, part of it is due to a rabid VC funding environment and part of it, more fundamentally, is due to inflection points in the market. In the last year, there's been less headline-grabbing discussion of futuristic applications of AI (self-driving vehicle, etc.), and a bit less AI hype as a result. Regardless, data and ML/AI-driven application companies have continued to thrive, particularly those focused on enterprise use cases. Meanwhile, a lot of the action has been happening behind the scenes on the data and ML infrastructure side, with entire new categories (data observability, reverse ETL, metrics stores, etc.) appearing and/or drastically accelerating. To keep track of this evolution, this is our eighth annual landscape and "state of the union" of the data and AI ecosystem – co-authored this year with my FirstMark colleague John Wu. (For anyone interested, here are the prior versions: 2012, 2014, 2016, 2017, 2018, 2019 (Part I and Part II) and 2020.) For those who have remarked over the years how insanely busy the chart is, you'll love our new acronym – Machine learning, Artificial intelligence and Data (MAD) – this is now officially the MAD landscape! We've learned over the years that those posts are read by a broad group of people, so we have tried to provide a little bit for everyone – a macro view that will hopefully be interesting and approachable to most; and then a slightly more granular overview of trends in data infrastructure and ML/AI for people with deeper familiarity with the industry. This (long!) post is organized as follows: Let's start with the high level view of the market. As the number of companies in the space keeps increasing every year, the inevitable questions are: why is this happening?

Computers have become an inseparable part of our life, as we all know. Today, technology has progressed to the point where a computer can execute jobs with a high percentage of success. Artificial Intelligence (AI) has made all of this feasible. AI is a branch of science that allows robots to do activities that would otherwise need human Intelligence. AI is a broad subject in computer science that is created and implemented using machine learning and deep learning techniques.

Machine learning (ML) has become an important component of many applications we use today. And adding machine learning capabilities to applications is becoming increasingly easy. Many ML libraries and online services don't even require a thorough knowledge of machine learning. However, even easy-to-use machine learning systems come with their own challenges. Among them is the threat of adversarial attacks, which has become one of the important concerns of ML applications.

The discriminator connects to two loss functions. During discriminator training, the discriminator ignores the generator loss and just uses the discriminator loss. We use the generator loss during generator training. The discriminator's training data comes from two sources: In the given figure above, the two "Sample" boxes represent these two data sources feeding into the discriminator. During discriminator training the generator does not train. Its weights remains fixed while it produces examples for the discriminator to train on.

Artificial intelligence in healthcare is the use of machine-learning algorithms and software to analyze, process and present complex medical and health care data. It has been widely used to support clinical decisions, improve workflows and predict health outcomes. Thus, wide application of AI in the healthcare sector is likely to propel the growth of the market. The growth of the artificial intelligence in healthcare market is attributed to the rising application of artificial intelligence in healthcare, growing investment in AI healthcare start-ups, and increasing cross-industry partnerships and collaborations. However, dearth of skilled AI workforce and imprecise regulatory guidelines for medical software is the major factor hindering the market growth.

Artificial intelligence (AI) machine learning is rapidly transforming how physicians, clinicians, pathologists, and health care providers diagnose patient conditions. A recent NYU Langone Health study published in Nature Communications shows how AI applied to ultrasound images can identify breast cancer with radiologist-level accuracy, reduce requested biopsies by 27.8 percent, and significantly decrease false positive rates of breast cancer by 37 percent. "In this work, we present an AI system that achieves radiologist-level accuracy in identifying breast cancer in ultrasound images," wrote Krzysztof Geras, PhD., the study senior investigator and assistant professor at NYU Grossman School of Medicine, in collaboration with co-investigator and radiologist Linda Moy, MD. a professor at NYU Grossman School of Medicine, and their research colleagues. Both Geras and Moy are members of the Perlmutter Cancer Center. Breast cancer is a leading cause of death among women worldwide.

A combination of high-resolution satellite imaging and'deep learning' has identified more than 1.8 billion trees across the West African Sahara, Sahel and sub-humid zone – significantly more trees than were previously thought to exist in the region. The collaboration between NASA and several geoscience departments across the world used 11,128 satellite images from four satellites to count individual trees across 1.3 million square kilometres. The deep-learning approach has, for the first time, allowed researchers to identify individual trees across the dryland expanse. Because of the absence of closed canopies, many parts of the Sahara and the Sahel have previously been mapped with zero per cent tree cover. 'You need high-resolution satellite images to be able to detect individual trees and not just to make estimations based on identified areas of canopy cover,' says Martin Brandt from the University of Copenhagen.