An approach to distributed machine learning is to train models on local datasets and aggregate these models into a single, stronger model. A popular instance of this form of parallelization is federated learning, where the nodes periodically send their local models to a coordinator that aggregates them and redistributes the aggregation back to continue training with it. The most frequently used form of aggregation is averaging the model parameters, e.g., the weights of a neural network. However, due to the non-convexity of the loss surface of neural networks, averaging can lead to detrimental effects and it remains an open question under which conditions averaging is beneficial. In this paper, we study this problem from the perspective of information theory: We measure the mutual information between representation and inputs as well as representation and labels in local models and compare it to the respective information contained in the representation of the averaged model. Our empirical results confirm previous observations about the practical usefulness of averaging for neural networks, even if local dataset distributions vary strongly. Furthermore, we obtain more insights about the impact of the aggregation frequency on the information flow and thus on the success of distributed learning. These insights will be helpful both in improving the current synchronization process and in further understanding the effects of model aggregation.

Correctly pricing products or services in an online marketplace presents a challenging problem and one of the critical factors for the success of the business. When users are looking to buy an item they typically search for it. Query relevance models are used at this stage to retrieve and rank the items on the search page from most relevant to least relevant. The presented items are naturally "competing" against each other for user purchases. We provide a practical two-stage model to price this set of retrieved items for which distributions of their values are learned. The initial output of the pricing strategy is a price vector for the top displayed items in one search event. We later aggregate these results over searches to provide the supplier with the optimal price for each item. We applied our solution to large-scale search data obtained from Airbnb Experiences marketplace. Offline evaluation results show that our strategy improves upon baseline pricing strategies on key metrics by at least +20% in terms of booking regret and +55% in terms of revenue potential.

We study distributed stochastic gradient (D-SG) method and its accelerated variant (D-ASG) for solving decentralized strongly convex stochastic optimization problems where the objective function is distributed over several computational units, lying on a fixed but arbitrary connected communication graph, subject to local communication constraints where noisy estimates of the gradients are available. We develop a framework which allows to choose the stepsize and the momentum parameters of these algorithms in a way to optimize performance by systematically trading off the bias, variance, robustness to gradient noise and dependence to network effects. When gradients do not contain noise, we also prove that distributed accelerated methods can \emph{achieve acceleration}, requiring $\mathcal{O}(\kappa \log(1/\varepsilon))$ gradient evaluations and $\mathcal{O}(\kappa \log(1/\varepsilon))$ communications to converge to the same fixed point with the non-accelerated variant where $\kappa$ is the condition number and $\varepsilon$ is the target accuracy. To our knowledge, this is the first acceleration result where the iteration complexity scales with the square root of the condition number in the context of \emph{primal} distributed inexact first-order methods. For quadratic functions, we also provide finer performance bounds that are tight with respect to bias and variance terms. Finally, we study a multistage version of D-ASG with parameters carefully varied over stages to ensure exact $\mathcal{O}(-k/\sqrt{\kappa})$ linear decay in the bias term as well as optimal $\mathcal{O}(\sigma^2/k)$ in the variance term. We illustrate through numerical experiments that our approach results in practical algorithms that are robust to gradient noise and that can outperform existing methods.

Due to the increasing use of machine learning in practice it becomes more and more important to be able to explain the pred iction and behavior of machine learning models. An instance of expl anations are counterfactual explanations which provide an intuitive an d useful explanations of machine learning models. In this survey we review model-specific methods for efficientl y computing counterfactual explanations of many different machine learning models and propose methods for models that have not been considered in l iterature so far.

Recent research has found that many families of machine learning models are vulnerable to adversarial examples: inputs that are specifically designed to cause the target model to produce erroneous outputs. In this survey, we focus on machine learning models in the visual domain, where methods for generating and detecting such examples have been most extensively studied. We explore a variety of adversarial attack methods that apply to image-space content, real world adversarial attacks, adversarial defenses, and the transferability property of adversarial examples. We also discuss strengths and weaknesses of various methods of adversarial attack and defense. Our aim is to provide an extensive coverage of the field, furnishing the reader with an intuitive understanding of the mechanics of adversarial attack and defense mechanisms and enlarging the community of researchers studying this fundamental set of problems.

Artificial Intelligence: Solving problems, growing the economy and improving our quality of life outlines the importance of action for Australia to capture the benefits of artificial intelligence (AI), estimated to be worth AU$22.17 Published by the Australian Government in November 2019, and codeveloped by CSIRO's Data61 and the Department of Industry, Innovation and Science, the report identifies strategies to help develop a national AI capability to boost the productivity of Australian industry, create jobs and economic growth, and improve the quality of life for current and future generations. The roadmap identifies three high potential areas of AI specialisation for Australia based on the opportunity to solve significant problems at home, export the solutions to the world and build on Australia's existing strengths. This report is intended to help guide future investment in AI and machine learning, and accompanies Artificial Intelligence: Australia's Ethics Framework, a discussion paper prepared by CSIRO's Data61 and published by the Australian Government in April 2019.

After breakfast one morning in August, the mathematician Terence Tao opened an email from three physicists he didn't know. The trio explained that they'd stumbled across a simple formula that, if true, established an unexpected relationship between some of the most basic and important objects in linear algebra. The formula "looked too good to be true," said Tao, who is a professor at the University of California, Los Angeles, a Fields medalist, and one of the world's leading mathematicians. "Something this short and simple -- it should have been in textbooks already," he said. "So my first thought was, no, this can't be true."

NEW YORK--China is eroding America's military superiority and conventional deterrence through the integration of artificial intelligence (AI) systems in its military strategies, operations, and capabilities, an independent U.S. federal commission warned, adding that the United States needs to step up investment in the technology and apply it to national security missions. China's communist regime has established research and development institutes to advance its military applications of AI. Those institutes are equivalent to the Defense Advanced Research Projects Agency (DARPA)--a U.S. agency under the Department of Defense responsible for the development of emerging technologies for military use. Military applications of AI technologies are being developed by Chinese researchers in the areas of "swarming, decision support, and information operations," while the country's defense industry is pursuing the development of "increasingly autonomous weapons systems," an interim report released by The National Security Commission on Artificial Intelligence said on Nov. 4. The Chinese Communist Party (CCP) declared it would be the world leader in AI by 2030, part of its broader strategy to challenge America's military and economic position in Asia, as Beijing also pursues a process of "intelligentization" as a new imperative of its military modernization.

Hospitals of the future will need to become more'tunable' than ever to the needs of patients and staff. BuildingIQ is already playing a role in this ongoing change by lending the machine learning and artificial intelligence (AI) capabilities of its 5i Platform and services to hospitals in Sydney, Australia. There are a number of reasons why BuildingIQ is the right fit for Hospitals and we explore them in detail in this Market Brief.

E-commerce is experiencing continuous evolution and has revolutionized retail industry significantly. This evolution is a vital requirement to meet the changing needs of people and make online shopping easier for them. The industry has seen steady growth in the last couple of years and doesn't look like it is stopping anytime soon. Speaking of its growth in recent years, a study, revealed that global e-commerce sales worth a whopping $3.453 trillion were made in 2019, and projected to even grow to $4.135 trillion in 2020. In 2021, the industry is expected to grow even further to hit the $4.878 trillion mark. At the start of the e-commerce wave, it was fairly limited in its capabilities due to limiting technology.