Abstract--In this letter, we introduce a distributed Nesterov method, termed as ABN, that does not require doubly-stochastic weight matrices. Instead, the implementation is based on a simultaneous application of both row-and column-stochastic weights that makes this method applicable to arbitrary (stronglyconnected) graphs.Since constructing column-stochastic weights needs additional information (the number of outgoing neighbors at each agent), not available in certain communication protocols, we derive a variation, termed as FROZEN, that only requires row-stochastic weights but at the expense of additional iterations for eigenvector learning. We numerically study these algorithms for various objective functions and network parameters and show that the proposed distributed Nesterov methods achieve acceleration compared to the current state-of-the-art methods for distributed optimization. I. INTRODUCTION Distributed optimization has recently seen a surge of interest particularly with the emergence of modern signal processing and machine learning applications. A well-studied problem in this domain is finite sum minimization that also has some relevance to empirical risk formulations, i.e., min R is a smooth and convex function available at an agent i. 's depend on data that may be private to each agent and communicating large data is impractical, developing distributed solutions of the above problem have attracted a strong interest.

This paper focuses on a new task, i.e., transplanting a category-and-task-specific neural network to a generic, modular network without strong supervision. We design a functionally interpretable structure for the generic network. Like building LEGO blocks, we teach the generic network a new category by directly transplanting the module corresponding to the category from a pre-trained network with a few or even without sample annotations. Our method incrementally adds new categories to the generic network but does not affect representations of existing categories. In this way, our method breaks the typical bottleneck of learning a net for massive tasks and categories, i.e., the requirement of collecting samples for all tasks and categories at the same time before the learning begins. Thus, we use a new distillation algorithm, namely back-distillation, to overcome specific challenges of network transplanting. Our method without training samples even outperformed the baseline with 100 training samples.

We present a scalable, black box, perception-in-the-loop technique to find adversarial examples for deep neural network classifiers. Black box means that our procedure only has input-output access to the classifier, and not to the internal structure, parameters, or intermediate confidence values. Perception-in-the-loop means that the notion of proximity between inputs can be directly queried from human participants rather than an arbitrarily chosen metric. Our technique is based on covariance matrix adaptation evolution strategy (CMA-ES), a black box optimization approach. CMA-ES explores the search space iteratively in a black box manner, by generating populations of candidates according to a distribution, choosing the best candidates according to a cost function, and updating the posterior distribution to favor the best candidates. We run CMA-ES using human participants to provide the fitness function, using the insight that the choice of best candidates in CMA-ES can be naturally modeled as a perception task: pick the top $k$ inputs perceptually closest to a fixed input. We empirically demonstrate that finding adversarial examples is feasible using small populations and few iterations. We compare the performance of CMA-ES on the MNIST benchmark with other black-box approaches using $L_p$ norms as a cost function, and show that it performs favorably both in terms of success in finding adversarial examples and in minimizing the distance between the original and the adversarial input. In experiments on the MNIST, CIFAR10, and GTSRB benchmarks, we demonstrate that CMA-ES can find perceptually similar adversarial inputs with a small number of iterations and small population sizes when using perception-in-the-loop. Finally, we show that networks trained specifically to be robust against $L_\infty$ norm can still be susceptible to perceptually similar adversarial examples.

Loss functions with a large number of saddle points are one of the main obstacles to training many modern machine learning models. Gradient descent (GD) is a fundamental algorithm for machine learning and converges to a saddle point for certain initial data. We call the region formed by these initial values the "attraction region." For quadratic functions, GD converges to a saddle point if the initial data is in a subspace of up to n-1 dimensions. In this paper, we prove that a small modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher, et al., arXiv:1806.06317] contributes to avoiding saddle points without sacrificing the convergence rate of GD. In particular, we show that the dimension of the LSGD's attraction region is at most floor((n-1)/2) for a class of quadratic functions which is significantly smaller than GD's (n-1)-dimensional attraction region.

We present a novel cross-view classification algorithm where the gallery and probe data come from different views. A popular approach to tackle this problem is the multi-view subspace learning (MvSL) that aims to learn a latent subspace shared by multi-view data. Despite promising results obtained on some applications, the performance of existing methods deteriorates dramatically when the multi-view data is sampled from nonlinear manifolds or suffers from heavy outliers. To circumvent this drawback, motivated by the Divide-and-Conquer strategy, we propose Multi-view Hybrid Embedding (MvHE), a unique method of dividing the problem of cross-view classification into three subproblems and building one model for each subproblem. Specifically, the first model is designed to remove view discrepancy, whereas the second and third models attempt to discover the intrinsic nonlinear structure and to increase discriminability in intra-view and inter-view samples respectively. The kernel extension is conducted to further boost the representation power of MvHE. Extensive experiments are conducted on four benchmark datasets. Our methods demonstrate overwhelming advantages against the state-of-the-art MvSL based cross-view classification approaches in terms of classification accuracy and robustness.

Indian Institute of Technology Hyderabad is launching a full-fledged BTech Program in Artificial Intelligence (AI) from the coming Academic Year (2019-2020). It has become the first Indian Educational Institution to offer such a full-fledged BTech program in AI and likely the third institute globally - after Carnegie Mellon University and Massachusetts Institute of Technology (MIT), both of which are in the US. The Course will have an intake of around 20 students who can take the program through the JEE-Advanced. The mission of the Department of Artificial Intelligence, IIT Hyderabad, is to produce students with a sound understanding of the fundamentals of theory and practice of Artificial Intelligence and Machine Learning. It also aims to enable students to become leaders in the industry and academia nationally and internationally and meet the pressing demands of the nation in the areas of AI and Machine Learning.

In the world of science and technology, it is being said that we are at the beginning of the Fourth Industrial Revolution. The first that lasted from 1760 to 1840 brought in the age of mechanized production. It was the result of new materials like iron and steel,which combined with new energy resources of coal and steam, led to'mass production', and a factory system with division of labour. The second industrial revolution from 1870 to the early part of 20th century was the result of electricity, and the internal combustion engine. Both powered industrial machines and made transport possible.

Hal Abelson, a renowned computer scientist at the Massachusetts Institute of Technology, was working the classroom, coffee cup in hand, pacing back and forth. The subject was artificial intelligence, and his students last week were mainly senior policymakers from countries in the 36-nation Organization for Economic Cooperation and Development. Mr. Abelson began with a brisk history of machine learning, starting in the 1950s. Next came a description of how the technology works, a hands-on project using computer-vision models and then case studies. The goal was to give the policymakers from countries like France, Japan and Sweden a sense of the technology's strengths and weaknesses, emphasizing the crucial role of human choices.

If you've ever led a team at work before, you know how hard it can be to keep people motivated. But one Japanese startup is using technology to make that easier than ever. The Tokyo-based company Attuned offers what it calls "predictive HR analytics" to help companies understand what makes each of their employees tick. And companies in Japan are paying thousands of dollars for the chance to get a better read on their workers. It's a simple process: When a company signs on with Attuned, its employees take a 55-question online test in which they're presented with pairs of statements, such as "Planning my day in advance gives me a sense of security," and "I prefer to be able to decide which task to focus on at any given time."

Education is one of the most powerful predictors of future success that human society has at its disposal. How we gather, process, and disseminate knowledge to each successive generation impacts not just individual success, but a host of other related factors such as economic growth, political empowerment, and technological innovation. It is no secret that access to more effective education for individual students is a key factor in the overall betterment of society – and to women's role in society. I've long been a proponent of better education for women – from my early career days working for CARE, to becoming the Chief Strategy Officer of Top Scholar, contributing to the book "Innovating Women" and to founding a non-profit to help the disadvantaged attain better education. Recently, I've been looking around globally for innovative solutions that can leapfrog women's education forward.