Optimization algorithms for training deep models not only affects the convergence rate and stability of the training process, but are also highly related to the generalization performance of the models. While adaptive algorithms, such as Adam and RMSprop, have shown better optimization performance than stochastic gradient descent (SGD) in many scenarios, they often lead to worse generalization performance than SGD, when used for training deep neural networks (DNNs). In this work, we identify two problems of Adam that may degrade the generalization performance. As a solution, we propose the normalized direction-preserving Adam (ND-Adam) algorithm, which combines the best of both worlds, i.e., the good optimization performance of Adam, and the good generalization performance of SGD. In addition, we further improve the generalization performance in classification tasks, by using batch-normalized softmax. This study suggests the need for more precise control over the training process of DNNs.

DeepMind Alberta will turbo-charge the research ecosystem, mirroring the partnerships DeepMind has nurtured with the top academic institutions in the UK,

At a Calgary branch of ATB Financial, one of the bank's latest recruits educates customers about financial literacy, plays music, challenges them to an impromptu dance-off and, naturally, takes a selfie. A four-part look at how robots are changing the way we work. Pepper, the new hire, doesn't have the most sophisticated skill set at this point -- for one thing, she can't make financial transactions -- but she's made a big leap forward in becoming the first customer service robot in Canadian banking. At some point, Pepper, developed by SoftBank Robotics Corp., could be programmed to do more complicated tasks. But for now, Edmonton-based ATB Financial is more interested in gauging how people react and figure out which customer situations the robot best fits in.

At a Calgary branch of ATB Financial, one of the bank's latest recruits educates customers about financial literacy, plays music, challenges them to an impromptu dance-off and, naturally, takes a selfie. A four-part look at how robots are changing the way we work. Pepper, the new hire, doesn't have the most sophisticated skill set at this point -- for one thing, she can't make financial transactions -- but she's made a big leap forward in becoming the first customer service robot in Canadian banking. At some point, Pepper, developed by SoftBank Robotics Corp., could be programmed to do more complicated tasks. But for now, Edmonton-based ATB Financial is more interested in gauging how people react and figure out which customer situations the robot best fits in.

At a Calgary branch of ATB Financial, one of the bank's latest recruits educates customers about financial literacy, plays music, challenges them to an impromptu dance-off and, naturally, takes a selfie. Pepper, the new hire, doesn't have the most sophisticated skill set at this point -- for one thing, she can't make financial transactions -- but she's made a big leap forward in becoming the robot in customer service in Canadian banking. At some point, Pepper, developed by SoftBank Robotics Corp., could be programmed to do more complicated tasks. But for now, Edmonton-based ATB Financial is more interested in gauging how people react and figure out which customer situations the robot best fits in. "This isn't … the world's going to be run by robots or something like that," chief executive Dave Mowat said.

The purpose of language is to reveal the contents of our minds, says Julie Sedivy. We are social animals and language is what springs us from our isolated selves and unites us with others. Sedivy has taught linguistics and psychology at Brown University and the University of Calgary. She specializes in psycholinguistics, the psychology of language, notably the psychological pressures that give birth to language and comprehension. More recently Sedivy has been writing about language in her own life. She was born in Czechoslovakia, spent time as a kid in Austria and Italy, and came of age in Canada. She speaks Czech, French, and English, and gets by in Spanish, Italian, and German.

Matrix factorization is a popular approach to solving matrix estimation problems based on partial observations. Existing matrix factorization is based on least squares and aims to yield a low-rank matrix to interpret the conditional sample means given the observations. However, in many real applications with skewed and extreme data, least squares cannot explain their central tendency or tail distributions, yielding undesired estimates. In this paper, we propose \emph{expectile matrix factorization} by introducing asymmetric least squares, a key concept in expectile regression analysis, into the matrix factorization framework. We propose an efficient algorithm to solve the new problem based on alternating minimization and quadratic programming. We prove that our algorithm converges to a global optimum and exactly recovers the true underlying low-rank matrices when noise is zero. For synthetic data with skewed noise and a real-world dataset containing web service response times, the proposed scheme achieves lower recovery errors than the existing matrix factorization method based on least squares in a wide range of settings.

Matrix factorization is a popular approach to solving matrix estimation problems based on partial observations. Existing matrix factorization is based on least squares and aims to yield a low-rank matrix to interpret the conditional sample means given the observations. However, in many real applications with skewed and extreme data, least squares cannot explain their central tendency or tail distributions, yielding undesired estimates. In this paper, we propose expectile matrix factorization by introducing asymmetric least squares, a key concept in expectile regression analysis, into the matrix factorization framework. We propose an efficient algorithm to solve the new problem based on alternating minimization and quadratic programming. We prove that our algorithm converges to a global optimum and exactly recovers the true underlying low-rank matrices when noise is zero. For synthetic data with skewed noise and a real-world dataset containing web service response times, the proposed scheme achieves lower recovery errors than the existing matrix factorization method based on least squares in a wide range of settings.

Ian H. Witten is a professor of computer science at the University of Waikato in New Zealand. He directs the New Zealand Digital Library research project. His research interests include information retrieval, machine learning, text compression, and programming by demonstration. He received an MA in Mathematics from Cambridge University, England; an MSc in Computer Science from the University of Calgary, Canada; and a PhD in Electrical Engineering from Essex University, England. He is a fellow of the ACM and of the Royal Society of New Zealand.

Huawei boss: Smartphones will be totally different by 2020. Here's what's going to change Rise of the Machines: is Artificial Intelligence Manipulating Our Minds? Stay up-to-date on the topics you care about. We'll send you an email alert whenever a news article matches your alert term. It's free, and you can add new alerts at any time.