Representation learning is a fundamental but challenging problem, especially when the distribution of data is unknown. We propose a new representation learning method, termed Structure Transfer Machine (STM), which enables feature learning process to converge at the representation expectation in a probabilistic way. We theoretically show that such an expected value of the representation (mean) is achievable if the manifold structure can be transferred from the data space to the feature space. The resulting structure regularization term, named manifold loss, is incorporated into the loss function of the typical deep learning pipeline. The STM architecture is constructed to enforce the learned deep representation to satisfy the intrinsic manifold structure from the data, which results in robust features that suit various application scenarios, such as digit recognition, image classification and object tracking. Compared to state-of-the-art CNN architectures, we achieve the better results on several commonly used benchmarks\footnote{The source code is available. https://github.com/stmstmstm/stm }.

This paper introduces a very challenging dataset of historic German documents and evaluates Fully Convolutional Neural Network (FCNN) based methods to locate handwritten annotations of any kind in these documents. The handwritten annotations can appear in form of underlines and text by using various writing instruments, e.g., the use of pencils makes the data more challenging. We train and evaluate various end-to-end semantic segmentation approaches and report the results. The task is to classify the pixels of documents into two classes: background and handwritten annotation. The best model achieves a mean Intersection over Union (IoU) score of 95.6% on the test documents of the presented dataset. We also present a comparison of different strategies used for data augmentation and training on our presented dataset. For evaluation, we use the Layout Analysis Evaluator for the ICDAR 2017 Competition on Layout Analysis for Challenging Medieval Manuscripts.

A major goal of unsupervised learning is to discover data representations that are useful for subsequent tasks, without access to supervised labels during training. Typically, this goal is approached by minimizing a surrogate objective, such as the negative log likelihood of a generative model, with the hope that representations useful for subsequent tasks will arise as a side effect. In this work, we propose instead to directly target a later desired task by meta-learning an unsupervised learning rule, which leads to representations useful for that task. Here, our desired task (meta-objective) is the performance of the representation on semi-supervised classification, and we meta-learn an algorithm -- an unsupervised weight update rule -- that produces representations that perform well under this meta-objective. Additionally, we constrain our unsupervised update rule to a be a biologically-motivated, neuron-local function, which enables it to generalize to novel neural network architectures. We show that the meta-learned update rule produces useful features and sometimes outperforms existing unsupervised learning techniques. We show that the meta-learned unsupervised update rule generalizes to train networks with different widths, depths, and nonlinearities. It also generalizes to train on data with randomly permuted input dimensions and even generalizes from image datasets to a text task.

We present a neurosymbolic approach to the lifelong learning of algorithmic tasks that mix perception and procedural reasoning. Reusing highlevel concepts across domains and learning complex procedures are two key challenges in lifelong learning. We show that a combination of gradientbased learning and symbolic program synthesis can be a more effective response to these challenges than purely neural methods. Concretely, our approach, called HOUDINI, represents neural networks as strongly typed, end-to-end differentiable functional programs that use symbolic higher-order combinators to compose a library of neural functions. Our learning algorithm consists of: (1) a program synthesizer that performs a type-directed search over programs in this language, and decides on the library functions that should be reused and the architectures that should be used to combine them; and (2) a neural module that trains synthesized programs using stochastic gradient descent. We evaluate our approach on three algorithmic tasks. Our experiments show that our type-directed search technique is able to significantly prune the search space of programs, and that the overall approach transfers high-level concepts more effectively than monolithic neural networks as well as traditional transfer learning.

A generative model is trained to learn the underlying distribution of the data. The idea behind having such models is not to memorize the entire data, but to learn those specific semantic and structural properties which help the model create new samples. These samples need not belong to the training set, yet can convincingly become a part of it. The other popular models such as Restricted Boltzmann Machines (RBMs) [9], Variational Auto-encoders (VAEs) [11] make use of latent variables as a hidden representation of the data samples. These models specify an explicit parameterized log-likelihood functions representing the data. The parameters are learned from the data. Estimating the maximum likelihood of the parameters requires integrating over the entire space of latent variables, which is intractable. Hence approximation techniques are used which may not always yield the best results. On the other hand, Generative Adversarial Networks, GANs, are one of the few implicit probabilistic models which define a stochastic procedure that directly generates data from a latent variable that belongs to a lower dimensional space.

AgShift, a technology startup building the first ever autonomous food inspection system, has raised $2 million seed funding from India's Exfinity Ventures and other companies. The purpose of the fundraising was to bolster product development and reach more customers, to help the startup pursue its mission of reducing global food waste. "Current food inspection processes are paper-based and tedious, needing continuous personal training. Inconsistent & subjective inspections result in a loss of $15.6 billion a year for the organizations responsible -- not counting the millions of dollars in recovery costs, claim management and loss of brand reputation incurred by the companies involved," said AgShift founder and CEO Miku Jha. "At AgShift, we are re-imagining food inspection at various layers -- starting from digitizing product specifications, using a mobile-first approach for operational efficiencies to leveraging Deep Learning to make inspections autonomous. Our goal is to standardize food inspection across the entire supply chain and reduce food wastage resulting from inconsistencies in food quality interpretation."

Deep learning techniques may pave the way for a more accurate outcome prediction in colorectal cancer patients as compared to evaluations currently performed by an experienced human observer. Researchers at the University of Helsinki are reporting in Scientific Reports that they have created a deep learning algorithm that appears to help clinicians better predict patient outcomes based on colorectal cancer tissue samples. "In our study we hypothesized whether a deep learning–based algorithm can be trained to extract prognostic features from cancer tissue images without any expert-defined supervision. It appeared exciting that almost no domain expertise is needed to build accurate classifiers," said study investigator Johan Lundin, MD, PhD, who is the Research Director of FIMM-Institute for Molecular Medicine Finland, at the University of Helsinki. The researchers combined convolutional and recurrent architectures to train a deep network to predict colorectal cancer outcome based simply on images of tumor tissue samples.

Voice technology and natural language recognition is at the forefront of AI development that is transforming our everyday life, and will undoubtedly be part of our future. Deep learning is at the core of identifying and processing voice for automation and new innovations. Join this Data Science Central webinar as conversational AI pioneer, Voicebox, shares their journey from an on-premise system that was manually intensive and costly to maintain, to an agile cloud platform that has allowed them to build, schedule, and run multiple production data pipelines that feed into their deep learning models with ease.

In a 2 hour talk that filled the Keynote Hall and spillover rooms at the San Jose McEnery Convention Center and had thousands of people in line for hours before, Nvidia's CEO Jen-Hsun Huang, in characteristic jeans, leather jacket, and humble humor, described the world of graphics processing units (GPUs) with brilliant images and memorable one-liners: Most of Nvidia's revenue comes from GPUs for gaming, super-capable ray-tracing professional graphics, and extraordinarily powerful super computers for data centers. Most of their current research and development is involved with AI-ready chips that enable clients to develop machine and deep learning models and apps. Nvidia is banking on these new development chips to be the chips of the future. In AI-focused healthcare, this covers CLARA, a deep learning engine that uses present-day black and white sonogram, PET and MRI 2D scans and enhances the data to 3D and then color rendering. In the example on the right, a black and white ultrasound sonogram on the left is enhanced into the fully rendered baby picture on the right.

TensorFlow is an open source software library for high performance numerical computation. Its flexible architecture allows easy deployment of computation across a variety of platforms (CPUs, GPUs, TPUs), and from desktops to clusters of servers to mobile and edge devices. Originally developed by researchers and engineers from the Google Brain team within Google's AI organization, it comes with strong support for machine learning and deep learning and the flexible numerical computation core is used across many other scientific domains.