For statistical learning, categorical variables in a table are usually considered as discrete entities and encoded separately to feature vectors, e.g., with one-hot encoding. "Dirty" non-curated data gives rise to categorical variables with a very high cardinality but redundancy: several categories reflect the same entity. In databases, this issue is typically solved with a deduplication step. We show that a simple approach that exposes the redundancy to the learning algorithm brings significant gains. We study a generalization of one-hot encoding, similarity encoding, that builds feature vectors from similarities across categories. We perform a thorough empirical validation on non-curated tables, a problem seldom studied in machine learning. Results on seven real-world datasets show that similarity encoding brings significant gains in prediction in comparison with known encoding methods for categories or strings, notably one-hot encoding and bag of character n-grams. We draw practical recommendations for encoding dirty categories: 3-gram similarity appears to be a good choice to capture morphological resemblance. For very high-cardinality, dimensionality reduction significantly reduces the computational cost with little loss in performance: random projections or choosing a subset of prototype categories still outperforms classic encoding approaches.

This course gives an overview of modern data-driven techniques for natural language processing. The course moves from shallow bag-of-words models to richer structural representations of how words interact to create meaning. At each level, we will discuss the salient linguistic phemonena and most successful computational models. Along the way we will cover machine learning techniques which are especially relevant to natural language processing. Readings will be drawn mainly from my notes.

Editor's note: A version of this commentary by Brigham Young University professor Darin Gates was published by BYU's Wheatley Institution on May 22. Artificial intelligence, or AI, generates the most pressing ethical questions of any technology today, in part because of the nearly ubiquitous influence it will have in so many areas of our lives. AI will have an immense ethical impact both in terms of the amount of good it can bring about, and in terms of the potential harms it can unleash. Interest in the ethical dimensions of AI has increased dramatically in recent years -- from the nearly daily reporting of ethical-related AI issues in various news outlets, to major companies such as Amazon, Google, Facebook, DeepMind, Microsoft and IBM coming together to create the Partnership on AI to Benefit People and Society. Elon Musk, the late Stephen Hawking and 8,000 others have signed an open letter with concerns about the future of AI. There has thus been an increasing recognition for the need to focus on the ethical aspects of AI. Here are some of the most significant ethical issues facing the continued implementation of artificial intelligence.

The Global Deep Learning Courses for NLP Market report forecast 2017-2021 is a professional and detailed study on the present state also focuses on the major drivers, Global Deep Learning Courses for NLP business strategists and effective growth for the key players. Global Deep Learning Courses for NLP Industry also provides granular analysis of the Global Deep Learning Courses for NLP market dynamics, share, segmentation, revenue forecasts and facilitate better decision-making. With a purpose of enlightening new entrants about the possibilities in this Global Deep Learning Courses for NLP Market, this report provides a competitive scenario of the Global Deep Learning Courses for NLP market with growth trends, structure, driving factors, scope, opportunities, challenges, vendor landscape analysis and so on, is discussed in the report. Analysis and Key Opportunities of Global Deep Learning Courses for NLP Market Report: Extensive analysis of the Market, by component, helps in understanding the components of the Market that are currently used along with the variants that would gain prominence in the future. Global Deep Learning Courses for NLP market report analyses the market potential for each geographical region based on the growth rate, macroeconomic parameters, consumer buying patterns, and market demand and supply scenarios.

Here, the only thing required to pursue this course is basic programming knowledge, a proficiency in Python and a general understanding of ML. The course was launched by Andrew Ng, a former chief scientist at Baidu in 2017. It aims to spread the benefit of recent advances in ML far beyond big tech companies. The course cost $49 a month and is offered via Coursera. It features five tracks that include neural networks, backpropagation, convolutional nets and recurrent nets. It also teaches other core aspects of deep learning. The students also get to participate in applied deep learning projects to address real-world problems in language understand, healthcare and music generation.

Valid causal inference in observational studies often requires controlling for confounders. However, in practice measurements of confounders may be noisy, and can lead to biased estimates of causal effects. We show that we can reduce the bias caused by measurement noise using a large number of noisy measurements of the underlying confounders. We propose the use of matrix factorization to infer the confounders from noisy covariates, a flexible and principled framework that adapts to missing values, accommodates a wide variety of data types, and can augment many causal inference methods. We bound the error for the induced average treatment effect estimator and show it is consistent in a linear regression setting, using Exponential Family Matrix Completion preprocessing. We demonstrate the effectiveness of the proposed procedure in numerical experiments with both synthetic data and real clinical data.

Program synthesis is the process of automatically translating a specification into computer code. Traditional synthesis settings require a formal, precise specification. Motivated by computer education applications where a student learns to code simple turtle-style drawing programs, we study a novel synthesis setting where only a noisy user-intention drawing is specified. This allows students to sketch their intended output, optionally together with their own incomplete program, to automatically produce a completed program. We formulate this synthesis problem as search in the space of programs, with the score of a state being the Hausdorff distance between the program output and the user drawing. We compare several search algorithms on a corpus consisting of real user drawings and the corresponding programs, and demonstrate that our algorithms can synthesize programs optimally satisfying the specification.

The UN Sustainable Development Goals allude to the importance of infrastructure quality in three of its seventeen goals. However, monitoring infrastructure quality in developing regions remains prohibitively expensive and impedes efforts to measure progress toward these goals. To this end, we investigate the use of widely available remote sensing data for the prediction of infrastructure quality in Africa. We train a convolutional neural network to predict ground truth labels from the Afrobarometer Round 6 survey using Landsat 8 and Sentinel 1 satellite imagery. Our best models predict infrastructure quality with AUROC scores of 0.881 on Electricity, 0.862 on Sewerage, 0.739 on Piped Water, and 0.786 on Roads using Landsat 8. These performances are significantly better than models that leverage OpenStreetMap or nighttime light intensity on the same tasks. We also demonstrate that our trained model can accurately make predictions in an unseen country after fine-tuning on a small sample of images. Furthermore, the model can be deployed in regions with limited samples to predict infrastructure outcomes with higher performance than nearest neighbor spatial interpolation.

Real images often lie on a union of disjoint manifolds rather than one globally connected manifold, and this can cause several difficulties for the training of common Generative Adversarial Networks (GANs). In this work, we first show that single generator GANs are unable to correctly model a distribution supported on a disconnected manifold, and investigate how sample quality, mode collapse and local convergence are affected by this. Next, we show how using a collection of generators can address this problem, providing new insights into the success of such multi-generator GANs. Finally, we explain the serious issues caused by considering a fixed prior over the collection of generators and propose a novel approach for learning the prior and inferring the necessary number of generators without any supervision. Our proposed modifications can be applied on top of any other GAN model to enable learning of distributions supported on disconnected manifolds. We conduct several experiments to illustrate the aforementioned shortcoming of GANs, its consequences in practice, and the effectiveness of our proposed modifications in alleviating these issues.

Current deep learning based text classification methods are limited by their ability to achieve fast learning and generalization when the data is scarce. We address this problem by integrating a meta-learning procedure that uses the knowledge learned across many tasks as an inductive bias towards better natural language understanding. Based on the Model-Agnostic Meta-Learning framework (MAML), we introduce the Attentive Task-Agnostic Meta-Learning (ATAML) algorithm for text classification. The essential difference between MAML and ATAML is in the separation of task-agnostic representation learning and task-specific attentive adaptation. The proposed ATAML is designed to encourage task-agnostic representation learning by way of task-agnostic parameterization and facilitate task-specific adaptation via attention mechanisms. We provide evidence to show that the attention mechanism in ATAML has a synergistic effect on learning performance. In comparisons with models trained from random initialization, pretrained models and meta trained MAML, our proposed ATAML method generalizes better on single-label and multi-label classification tasks in miniRCV1 and miniReuters-21578 datasets.