Predicting molecular properties remains a challenging task with numerous potential applications, notably in drug discovery. Recently, the development of deep learning, combined with rising amounts of data, has provided powerful tools to build predictive models. Since molecules can be encoded as graphs, Graph Neural Networks (GNNs) have emerged as a popular choice of architecture to tackle this task. Training GNNs to predict molecular properties however faces the challenge of collecting annotated data which is a costly and time consuming process. On the other hand, it is easy to access large databases of molecules without annotations.

BigML's upcoming release on Wednesday, December 15, 2021, will be presenting a new set of Image Processing resources to the BigML platform. In this post, we show you how to build a simple image classifier on the BigML Dashboard. Image classification is a supervised learning technique for images. Image classification models are trained to identify various classes of images and have a tremendous amount of applications as touched on in our prior posts. As such, BigML introduces image data support with the latest Image Processing release.

Tactics analyze the current proof state (goal and assumptions) and apply non-trivial proof transformations. Formalized proofs take advantage of different levels of automation which are in increasing order of generality: specialized rules, theory-based strategies and general purpose strategies. Thanks to progress in proof automation, developers can delegate more and more complicated proof obligations to general purpose strategies. Those are implemented by automated theorem provers (ATPs) such as E prover [32]. Communication between an ITP and ATPs is made possible by a "hammer" system [4,14]. It acts as an interface by performing premise selection, translation and proof reconstruction. Yet, ATPs are not flawless and more precise user-guidance, achieved by applying a particular sequence of specialized rules, is almost always necessary to develop a mathematical theory.

Whether you are an algorithm developer in a growing startup company, a data scientist in a university research lab, or a kaggle hobbyist, there may come a point in time when the training resources that you have onsite no longer meet your training demands. In this post we target development teams that are (finally) ready to move their machine learning (ML) workloads to the cloud. We will discuss some of the important decisions that need to made during this big transition. Naturally, any attempt to encompass all of the steps of such an endeavor is doomed to fail. Machine learning projects come in many shapes and forms and as their complexity increases so does the undertaking of making such a significant change as migrating to the cloud. In this post we will highlight what we believe to be some of the most important considerations that are common to most typical deep learning projects.

While neural language models often perform surprisingly well on natural language understanding (NLU) tasks, their strengths and limitations remain poorly understood. Controlled synthetic tasks are thus an increasingly important resource for diagnosing model behavior. In this work we focus on story understanding, a core competency for NLU systems. However, the main synthetic resource for story understanding, the bAbI benchmark, lacks such a systematic mechanism for controllable task generation. We develop Dyna-bAbI, a dynamic framework providing fine-grained control over task generation in bAbI. We demonstrate our ideas by constructing three new tasks requiring compositional generalization, an important evaluation setting absent from the original benchmark. We tested both special-purpose models developed for bAbI as well as state-of-the-art pre-trained methods, and found that while both approaches solve the original tasks (>99% accuracy), neither approach succeeded in the compositional generalization setting, indicating the limitations of the original training data. We explored ways to augment the original data, and found that though diversifying training data was far more useful than simply increasing dataset size, it was still insufficient for driving robust compositional generalization (with <70% accuracy for complex compositions). Our results underscore the importance of highly controllable task generators for creating robust NLU systems through a virtuous cycle of model and data development.

The exponential number of research and publications have introduced many terms and concepts in the domain of machine learning, yet many have degenerated to merely buzzwords without many people fully understanding their differences. The most common, and perhaps THE type that we refer to when talking about machine learning is supervised learning. In simple words, supervised learning provides a set of input-output pairs such that we can learn an intermediate system that maps inputs to correct outputs. A naive example of supervised learning is determining the class (i.e., dogs/cats, etc) of an image based on a dataset of images and their corresponding classes, which we will refer to as their labels. With the given input-label pair, the current popular approach will be to directly train a deep neural network (i.e., a convolutional neural network) to output a label prediction from the given image, compute a differentiable loss between the prediction and the actual correct answers, and backpropagate through the network to update weights to optimise the predictions.

This article is part of Demystifying AI, a series of posts that (try to) disambiguate the jargon and myths surrounding AI. Machine learning models can perform wonderful things--if they have enough training data. Unfortunately, for many applications, access to quality data remains a barrier. One solution to this problem is "data augmentation," a technique that generates new training examples from existing ones. Data augmentation is a low-cost and effective method to improve the performance and accuracy of machine learning models in data-constrained environments.

Whether you are an algorithm developer in a growing startup company, a data scientist in a university research lab, or a kaggle hobbyist, there may come a point in time when the training resources that you have onsite no longer meet your training demands. In this post we target development teams that are (finally) ready to move their machine learning (ML) workloads to the cloud. We will discuss some of the important decisions that need to made during this big transition. Naturally, any attempt to encompass all of the steps of such an endeavor is doomed to fail. Machine learning projects come in many shapes and forms and as their complexity increases so does the undertaking of making such a significant change as migrating to the cloud. In this post we will highlight what we believe to be some of the most important considerations that are common to most typical deep learning projects.

This article is part of our reviews of AI research papers, a series of posts that explore the latest findings in artificial intelligence. Contrastive learning (CL) is a machine learning technique that has gained popularity in the past few years because it reduces the need for annotated data, one of the main pain points of developing ML models. But due to its peculiarities, contrastive learning presents security challenges that are different from those found in supervised machine learning. Machine learning and security researchers are worried about the effect of adversarial attacks on ML models trained through contrastive learning. Accepted at NeurIPS 2021, the paper introduces a new technique that helps protect contrastive learning models against adversarial attacks while also preserving their accuracy. Supervised learning, the traditional way of training ML models, requires large sets of labeled data.

Self-supervised learning algorithms, including BERT and SimCLR, have enabled significant strides in fields like natural language processing, computer vision, and speech processing. However, these algorithms are domain-specific, meaning that new self-supervised learning algorithms must be developed for each new setting, including myriad healthcare, scientific, and multimodal domains. To catalyze progress toward domain-agnostic methods, we introduce DABS: a Domain-Agnostic Benchmark for Self-supervised learning. To perform well on DABS, an algorithm is evaluated on seven diverse domains: natural images, multichannel sensor data, English text, speech recordings, multilingual text, chest x-rays, and images with text descriptions. Each domain contains an unlabeled dataset for pretraining; the model is then is scored based on its downstream performance on a set of labeled tasks in the domain. We also present e-Mix and ShED: two baseline domain-agnostic algorithms; their relatively modest performance demonstrates that significant progress is needed before self-supervised learning is an out-of-the-box solution for arbitrary domains. Code for benchmark datasets and baseline algorithms is available at https://github.com/alextamkin/dabs.