Deep Learning
Optimization Methods for Supervised Machine Learning: From Linear Models to Deep Learning
Curtis, Frank E., Scheinberg, Katya
The goal of this tutorial is to introduce key models, algorithms, and open questions related to the use of optimization methods for solving problems arising in machine learning. It is written with an INFORMS audience in mind, specifically those readers who are familiar with the basics of optimization algorithms, but less familiar with machine learning. We begin by deriving a formulation of a supervised learning problem and show how it leads to various optimization problems, depending on the context and underlying assumptions. We then discuss some of the distinctive features of these optimization problems, focusing on the examples of logistic regression and the training of deep neural networks. The latter half of the tutorial focuses on optimization algorithms, first for convex logistic regression, for which we discuss the use of first-order methods, the stochastic gradient method, variance reducing stochastic methods, and second-order methods. Finally, we discuss how these approaches can be employed to the training of deep neural networks, emphasizing the difficulties that arise from the complex, nonconvex structure of these models.
Neural Sequence Model Training via $\alpha$-divergence Minimization
Koyamada, Sotetsu, Kikuchi, Yuta, Kanemura, Atsunori, Maeda, Shin-ichi, Ishii, Shin
We propose a new neural sequence model training method in which the objective function is defined by $\alpha$-divergence. We demonstrate that the objective function generalizes the maximum-likelihood (ML)-based and reinforcement learning (RL)-based objective functions as special cases (i.e., ML corresponds to $\alpha \to 0$ and RL to $\alpha \to1$). We also show that the gradient of the objective function can be considered a mixture of ML- and RL-based objective gradients. The experimental results of a machine translation task show that minimizing the objective function with $\alpha > 0$ outperforms $\alpha \to 0$, which corresponds to ML-based methods.
Artificial Intelligence On Wall Street: AI Labs Has Edge On Robo-Advice With Neural Networks
There's a lot of noise about how Robo-Advice is going to disrupt the wealth management sector. Indeed, the UK's FCA received over 40 applications last year to use automated wealth solutions of one sort or another. Many Robo-Advisors tout some level of machine intelligence to foster results, but really offer a generic, pre-set portfolio of three to six ETFs to clients. San Francisco-based AI Labs began building its Vise platform n January 2015. Today it uses a mix of machine learning and deep learning.
5 Actual Big Data Uses That Make Our Life Better
The Big Data Revolution (with the help of Machine Learning, Deep Learning and Artificial Intelligence) can transform our world in a ยซ Big Brother Nightmare ยปโฆ It could beโฆ but, The Big Data can at the opposite improve our lives for a better tomorrow. So, let's check these 5 actual cases where the Big Data ecosystem improve humanity lives for real. Data analysis, Deep Learning and robotics improve patient follow-up, long-term care and help prevent relapses. Big Data is used to predict which patients are likely to follow their doctor's advice, and who may not. Mobile applications are developed to check whether a patient is taking medication, such as an inhaler with a GPS chip for asthmatics.
Interpreting Deep Neural Networks using Cognitive Psychology DeepMind
We tried this experiment with our deep networks (Matching Networks and an Inception baseline model) and found that - like humans - our networks have a strong bias towards object shape rather than colour or texture. In other words, they have a'shape bias'. This suggests that Matching Networks and the Inception classifier use an inductive bias for shape to eliminate incorrect hypotheses, giving us a clear insight into how these networks solve the one-shot word learning problem. We observed that the shape bias emerges gradually over the course of early training in our networks. This is reminiscent of the emergence of shape bias in humans: young children show smaller shape bias than older children, and adults show the largest bias (2).
Fake news: you ain't seen nothing yet
EARLIER this year Franรงoise Hardy, a French musician, appeared in a YouTube video (see link). She is asked, by a presenter off-screen, why President Donald Trump sent his press secretary, Sean Spicer, to lie about the size of the inauguration crowd. Then she says Mr Spicer "gave alternative facts to that". It's all a little odd, not least because Franรงoise Hardy (pictured), who is now 73, looks only 20, and the voice coming out of her mouth belongs to Kellyanne Conway, an adviser to Mr Trump. The video, called "Alternative Face v1.1", is the work of Mario Klingemann, a German artist.
Learning through human feedback DeepMind
We believe that Artificial Intelligence will be one of the most important and widely beneficial scientific advances ever made, helping humanity tackle some of its greatest challenges, from climate change to delivering advanced healthcare. But for AI to deliver on this promise, we know that the technology must be built in a responsible manner and that we must consider all potential challenges and risks. That is why DeepMind co-founded initiatives like the Partnership on AI to Benefit People and Society and why we have a team dedicated to technical AI Safety. Research in this field needs to be open and collaborative to ensure that best practices are adopted as widely as possible, which is why we are also collaborating with OpenAI on research in technical AI Safety. One of the central questions in this field is how we allow humans to tell a system what we want it to do and - importantly - what we don't want it to do.
Maluuba Supports and Encourages Diversity in Deep Learning Research
Artificial intelligence algorithms, services and products are a reflection of their human designers. Accordingly, it's important that these teams are comprised of diverse individuals, each bringing unique perspectives and experiences to the table. Following our recent support of McGill's AI Lab for Social Good, we are proud to be a sponsor of the Women in Deep Learning event at the Universitรฉ de Montrรฉal. Taking place alongside the Deep Learning Summer School, this full day program seeks to create a discussion about creating work environments that meet the needs of women in research and science. "A diverse working environment is the most creative and productive one for longer terms. AI is shaping our future, and AI/STEM is still struggling to achieve even 25% of women in its global workforce. Creating awareness about increasing diversity including breaking the gender stereotypes is the need of the hour," said Subarna Tripathi, PhD candidate, University of California San Diego.
The most popular deep learning libraries - code(love)
Roger is an entrepreneur who has co-founded a social network entitled ThoughtBasin that looks to connect students looking to make a difference with organizations looking for difference makers. This experience has given him some setbacks, but also some priceless insights. He is deferring admission from the law school of University of Toronto to pursue his dream of creating impact through entrepreneurship, and he is constantly looking to learn and create, and to do more. He contributes to social entrepreneurship projects with his fellow Global Shapers, coordinates a volunteer tutoring site, and on his off time he unwinds by reading, writing, and dancing---sometimes, all at the same time. Follow him on Twitter at https://twitter.com/Rogerh1991.
Understanding Recurrent Neural Networks: The Prefered Neural Network for Time-Series Data
Artificial intelligence has been in the background for decades, kicking up dust in the distance, but never quite arriving. Well that era is over. In 2017, AI has broken through the dust cloud and arrived in a big way. And what do recurrent neural networks have to do with it? Thanks to an ingenious form of short-term memory that is unheard of in conventional neural networks, today's recurrent neural networks (RNNs) have been proving themselves as powerful predictive engines.