Every company is now a data company, capable of using machine learning in the cloud to deploy intelligent apps at scale, thanks to three machine learning trends: data flywheels, the algorithm economy, and cloud-hosted intelligence. That was the takeaway from the inaugural Machine Learning / Artificial Intelligence Summit, hosted by Madrona Venture Group* last month in Seattle, where more than 100 experts, researchers, and journalists converged to discuss the future of artificial intelligence, trends in machine learning, and how to build smarter applications. With hosted machine learning models, companies can now quickly analyze large, complex data, and deliver faster, more accurate insights without the high cost of deploying and maintaining machine learning systems. "Every successful new application built today will be an intelligent application," Soma Somasegar said, venture partner at Madrona Venture Group. "Intelligent building blocks and learning services will be the brains behind apps."

As we all know, the solution to a non-convex optimization algorithm (like stochastic gradient descent) depends on the initial values of the parameters. This post is about choosing initialization parameters for deep networks and how it affects the convergence. We will also discuss the related topic of vanishing gradients. First, let's go back to the time of sigmoidal activation functions and initialization of parameters using IID Gaussian or uniform distributions with fairly arbitrarily set variances. Building deep networks was difficult because of exploding or vanishing activations and gradients.

Artificial neural networks are most commonly trained with the back-propagation algorithm, where the gradient for learning is provided by back-propagating the error, layer by layer, from the output layer to the hidden layers. A recently discovered method called feedback-alignment shows that the weights used for propagating the error backward don't have to be symmetric with the weights used for propagation the activation forward. In fact, random feedback weights work evenly well, because the network learns how to make the feedback useful. In this work, the feedback alignment principle is used for training hidden layers more independently from the rest of the network, and from a zero initial condition. The error is propagated through fixed random feedback connections directly from the output layer to each hidden layer. This simple method is able to achieve zero training error even in convolutional networks and very deep networks, completely without error back-propagation. The method is a step towards biologically plausible machine learning because the error signal is almost local, and no symmetric or reciprocal weights are required. Experiments show that the test performance on MNIST and CIFAR is almost as good as those obtained with back-propagation for fully connected networks. If combined with dropout, the method achieves 1.45% error on the permutation invariant MNIST task.

In this paper, a Wide Learning architecture is proposed that attempts to automate the feature engineering portion of the machine learning (ML) pipeline. Feature engineering is widely considered as the most time consuming and expert knowledge demanding portion of any ML task. The proposed feature recommendation approach is tested on 3 healthcare datasets: a) PhysioNet Challenge 2016 dataset of phonocardiogram (PCG) signals, b) MIMIC II blood pressure classification dataset of photoplethysmogram (PPG) signals and c) an emotion classification dataset of PPG signals. While the proposed method beats the state of the art techniques for 2nd and 3rd dataset, it reaches 94.38% of the accuracy level of the winner of PhysioNet Challenge 2016. In all cases, the effort to reach a satisfactory performance was drastically less (a few days) than manual feature engineering.

Artificial intelligence (or AI) has permeated most facets of our lives. Algorithms suggest our social media mates. But could the arrival of the robots be applied to education? Jozef Misik, managing director of Knowble, a language tech start-up whose products are built on AI, believes so: "Most educational technology products will have an AI or deep learning component in future," he says. Already, AI is able to address common learning challenges.

Most deep learning techniques are extensions or adaptions of ANN's, called deep nets. Different configurations of deep nets are suitable for different machine learning tasks: Restricted Boltzman Machines (RBM's) (Smolensky 1986; Hinton & Salakhutdinov, 2006) and Autoencoders (Vincent, Larochelle, Bengio, & Manzagol,2008) are the main deep learning techniques for finding patterns in unlabeled data. This includes tasks such as feature extraction, pattern recognition and other unsupervised learning settings.

According to the American Cancer Society, cancer kills more than 8 million people each year. Early detection can boost survival rates. Researchers and clinicians are feverishly exploring avenues to provide early and accurate diagnoses, as well as more targeted treatments. Blood screenings are used to detect many types of cancers, including liver, ovarian, colon and lung cancers. Current blood screening methods typically rely on affixing biochemical labels to cells or biomolecules.

London artificial intelligence lab DeepMind is setting up a sizeable new team in the US in a bid to increase collaboration with parent company Google. DeepMind, bought by Google in 2014 for £400 million, is planning to hire "a couple of dozen" people at Google's headquarters in Mountain View, according to a DeepMind spokesperson. "We're proud to already have close partnerships with many teams at Google, but we're yet to develop an algorithm that gets rid of time zone differences," the spokesperson told Business Insider. "So we're hiring a small DeepMind Applied team in Mountain View to bridge the gap between Google and our team in London, helping us collaborate even more closely to bring our research breakthroughs to Google users around the world." The expansion represents a significant milestone in DeepMind's journey and comes after Yann LeCun, the head of AI research at Facebook, suggested that DeepMind was too far away from the Google "mothership" to have a significant impact.

At KDnuggets, we try to keep our finger on the pulse of main events and developments in industry, academia, and technology. We also do our best to look forward to key trends on the horizon. We recently asked some of the leading experts in Big Data, Data Science, Artificial Intelligence, and Machine Learning for their opinion on the most important developments of 2016 and key trends they 2017. Common themes include the triumphs of deep neural networks, reinforcement learning's successes, AlphaGo as exemplar of the power of both of these phenomena in unison, the application of machine learning to the Internet of Things, self-driving vehicles, and automation, among others. We generally asked participants to keep their responses to within 100 words or so, but were amenable to longer answers if the situation warranted.

At the 2016 Deep Learning Summit in Boston, Andrew Tulloch, Research Engineer at Facebook, talked about some of the tools and tricks Facebook use for scaling both the training and deployment of some of their deep learning models at Facebook. He also covered some useful libraries that they'd open-sourced for production-oriented deep learning applications. Tulloch's session can be watched in full below.