Africa
AI & IoT Insider Labs: Helping transform smallholder farming
This blog post was authored by Peter Cooper, Senior Product Manager, Microsoft IoT. From smart factories and smart cities to virtual personal assistants and self-driving cars, artificial intelligence (AI) and the Internet of Things (IoT) are transforming how people around the world live, work, and play. But fundamentally changing the ways people, devices, and data interact is not simple or easy work. Microsoft's AI & IoT Insider Labs was created to help all types of organizations accelerate their digital transformation. Member organizations around the world get access to support both technology development and product commercialization, for everything from hardware design to manufacturing to building applications and turning data into insights using machine learning.
The online conference that might change video games for good
Language is a tool, and just like any tool, it has equal capacity to inflict both good and bad on the world. Language is a beautiful, human thing; the connective tissue that transfers culture, knowledge and critical information across borders and generations. It's that second function -- the divisive one -- that inspired developer Rami Ismail and voice actor Sarah Elmaleh to produce a conference for game creators that removes language as a barrier to entry. Gamedev.world is billed as the first truly global online games conference, with plans to host 48 hours of expert panels and live Q&A sessions on Twitch, YouTube and Mixer, translated in real-time into English, Japanese, Spanish, Portuguese, Russian, Arabic and Simplified Chinese. It's all scheduled to take place later this year. "If games can be played by anyone, and made by anyone, we want to make sure everyone feels like they truly belong here," Ismail told Engadget.
Communication Complexity of Estimating Correlations
Hadar, Uri, Liu, Jingbo, Polyanskiy, Yury, Shayevitz, Ofer
We characterize the communication complexity of the following distributed estimation problem. Alice and Bob observe infinitely many iid copies of $\rho$-correlated unit-variance (Gaussian or $\pm1$ binary) random variables, with unknown $\rho\in[-1,1]$. By interactively exchanging $k$ bits, Bob wants to produce an estimate $\hat\rho$ of $\rho$. We show that the best possible performance (optimized over interaction protocol $\Pi$ and estimator $\hat \rho$) satisfies $\inf_{\Pi,\hat\rho}\sup_\rho \mathbb{E} [|\rho-\hat\rho|^2] = \Theta(\tfrac{1}{k})$. Furthermore, we show that the best possible unbiased estimator achieves performance of $1+o(1)\over {2k\ln 2}$. Curiously, thus, restricting communication to $k$ bits results in (order-wise) similar minimax estimation error as restricting to $k$ samples. Our results also imply an $\Omega(n)$ lower bound on the information complexity of the Gap-Hamming problem, for which we show a direct information-theoretic proof. Notably, the protocol achieving (almost) optimal performance is one-way (non-interactive). For one-way protocols we also prove the $\Omega(\tfrac{1}{k})$ bound even when $\rho$ is restricted to any small open sub-interval of $[-1,1]$ (i.e. a local minimax lower bound). %We do not know if this local behavior remains true in the interactive setting. Our proof techniques rely on symmetric strong data-processing inequalities, various tensorization techniques from information-theoretic interactive common-randomness extraction, and (for the local lower bound) on the Otto-Villani estimate for the Wasserstein-continuity of trajectories of the Ornstein-Uhlenbeck semigroup.
Is AI the Next Frontier for National Competitive Advantage?
Artificial intelligence (AI) presents limitless opportunity, but not without potential pitfalls and risks. This paradox has become increasingly evident for government leaders. They want to give domestic companies an edge over the competition, but are also expected to protect their citizens and use AI for social good. They want to support innovation, while still maintaining some level of control over how new technologies impact society at large. With a huge payoff on the line -- by our own estimates, AI has the potential to increase worldwide GDP by 14 percent by 2030, an infusion of US$15.7 trillion into the global economy -- it should come as no surprise that governments are eager to claim their share.
Google Gives Wikimedia Millions--Plus Machine Learning Tools
Google is pouring an additional $3.1 million into Wikipedia, bringing its total contribution to the free encyclopedia over the past decade to more than $7.5 million, the company announced at the World Economic Forum Tuesday. A little over a third of those funds will go toward sustaining current efforts at the Wikimedia Foundation, the nonprofit that runs Wikipedia, and the remaining $2 million will focus on long-term viability through the organization's endowment. Google will also begin allowing Wikipedia editors to use several of its machine learning tools for free, the tech giant said. What's more, Wikimedia and Google will soon broaden Project Tiger, a joint initiative they launched in 2017 to increase the number of Wikipedia articles written in underrepresented languages in India, and to include 10 new languages in a handful of countries and regions. It will now be called GLOW, Growing Local Language Content on Wikipedia.
Navy to test 'ghost fleet' attack drone boats in war scenarios
File photo - An unmanned 11-meter rigid-hull inflatable boat from Naval Surface Warfare Center Carderock operates autonomously during an Office of Naval Research-sponsored demonstration of swarmboat technology on the James River in Newport News, Va.(U.S. Navy photo by John F. Williams/Released) The U.S. Navy will launch a swarm of interconnected small attack drone boats on mock-combat missions to refine command and control technology and prepare its "Ghost Fleet" of autonomous, yet networked surface craft for war. Developed by the Office of Naval Research and Naval Sea Systems Command, "Ghost Fleet" represents a Navy strategy to surveil, counter, overwhelm and attack enemies in a coordinated fashion - all while keeping sailors on host ships at safer distances. The small boats, many of them called Unmanned Surface Vessels, are designed to conduct ISR missions, find and destroy mines and launch a range of attacks including electronic warfare and even mounted guns. The concept is to use advanced computer algorithms bringing new levels of autonomy to surface warfare, enabling ships to coordinate information exchange, operate in tandem without colliding and launch combined assaults. "Ghost Fleet is really helping us in the Command and Control and coms arena. The demonstration will allow us to learn lessons about integrated payloads with USVs," Capt.
Heavy-Tailed Universality Predicts Trends in Test Accuracies for Very Large Pre-Trained Deep Neural Networks
Martin, Charles H., Mahoney, Michael W.
Given two or more Deep Neural Networks (DNNs) with the same or similar architectures, and trained on the same dataset, but trained with different solvers, parameters, hyper-parameters, regularization, etc., can we predict which DNN will have the best test accuracy, and can we do so without peeking at the test data? In this paper, we show how to use a new Theory of Heavy-Tailed Self-Regularization (HT-SR) to answer this. HT-SR suggests, among other things, that modern DNNs exhibit what we call Heavy-Tailed Mechanistic Universality (HT-MU), meaning that the correlations in the layer weight matrices can be fit to a power law with exponents that lie in common Universality classes from Heavy-Tailed Random Matrix Theory (HT-RMT). From this, we develop a Universal capacity control metric that is a weighted average of these PL exponents. Rather than considering small toy NNs, we examine over 50 different, large-scale pre-trained DNNs, ranging over 15 different architectures, trained on ImagetNet, each of which has been reported to have different test accuracies. We show that this new capacity metric correlates very well with the reported test accuracies of these DNNs, looking across each architecture (VGG16/.../VGG19, ResNet10/.../ResNet152, etc.). We also show how to approximate the metric by the more familiar Product Norm capacity measure, as the average of the log Frobenius norm of the layer weight matrices. Our approach requires no changes to the underlying DNN or its loss function, it does not require us to train a model (although it could be used to monitor training), and it does not even require access to the ImageNet data.
Traditional and Heavy-Tailed Self Regularization in Neural Network Models
Martin, Charles H., Mahoney, Michael W.
Random Matrix Theory (RMT) is applied to analyze the weight matrices of Deep Neural Networks (DNNs), including both production quality, pre-trained models such as AlexNet and Inception, and smaller models trained from scratch, such as LeNet5 and a miniature-AlexNet. Empirical and theoretical results clearly indicate that the empirical spectral density (ESD) of DNN layer matrices displays signatures of traditionally-regularized statistical models, even in the absence of exogenously specifying traditional forms of regularization, such as Dropout or Weight Norm constraints. Building on recent results in RMT, most notably its extension to Universality classes of Heavy-Tailed matrices, we develop a theory to identify \emph{5+1 Phases of Training}, corresponding to increasing amounts of \emph{Implicit Self-Regularization}. For smaller and/or older DNNs, this Implicit Self-Regularization is like traditional Tikhonov regularization, in that there is a `size scale' separating signal from noise. For state-of-the-art DNNs, however, we identify a novel form of \emph{Heavy-Tailed Self-Regularization}, similar to the self-organization seen in the statistical physics of disordered systems. This implicit Self-Regularization can depend strongly on the many knobs of the training process. By exploiting the generalization gap phenomena, we demonstrate that we can cause a small model to exhibit all 5+1 phases of training simply by changing the batch size.
Location reference identification from tweets during emergencies: A deep learning approach
Kumar, Abhinav, Singh, Jyoti Prakash
Twitter is recently being used during crises to communicate with officials and provide rescue and relief operation in real time. The geographical location information of the event, as well as users, are vitally important in such scenarios. The identification of geographic location is one of the challenging tasks as the location information fields, such as user location and place name of tweets are not reliable. The extraction of location information from tweet text is difficult as it contains a lot of nonstandard English, grammatical errors, spelling mistakes, nonstandard abbreviations, and so on. This research aims to extract location words used in the tweet using a Convolutional Neural Network (CNN) based model. We achieved the exact matching score of 0.929, Hamming loss of 0.002, and F Our model was able to extract even three-to four-word long location references which is also evident from the exact matching score of over 92%. The findings of this paper can help in early event localization, emergency situations, real-time road traffic management, localized advertisement, and in various location-based services. Keywords: Location references, Tweets, Geo-locations, Named entity recognition, Gazetteer, Convolutional Neural Network 1. Introduction Tweets are very responsive to real-world events, and are sometimes even more immediate than traditional news channels. Therefore, it is possible to keep track of the latest information by following tweets. Several examples were seen when the news was first reported on Twitter, such as an airplane crash over the Hudson River in New York in the year 2009 (Sakaki et al., 2013), the death of former British Prime Minister Margaret Thatcher in April 2013 Preprint submitted to Elsevier January 25, 2019 Sakaki et al., 2013; Singh et al., 2017; Yuan & Liu, 2018). In an American Red Cross survey, a question was asked to individuals that "whom they contacted in an emergency?" The estimation and detection of location information of events and users from tweets are a major concern in relation to the above-mentioned tasks. Twitter provides three location information fields for sharing a user's location: (1) User location; (2) Place name; and (3) Geo-coordinate. The user location field has 140 character spaces (previously it was limited to 30 characters) in which the user can write his/her home location information while creating their profile. This field is optional to the user and the user can write any arbitrary words or leave it blank.