File photo - An unmanned 11-meter rigid-hull inflatable boat from Naval Surface Warfare Center Carderock operates autonomously during an Office of Naval Research-sponsored demonstration of swarmboat technology on the James River in Newport News, Va.(U.S. Navy photo by John F. Williams/Released) The U.S. Navy will launch a swarm of interconnected small attack drone boats on mock-combat missions to refine command and control technology and prepare its "Ghost Fleet" of autonomous, yet networked surface craft for war. Developed by the Office of Naval Research and Naval Sea Systems Command, "Ghost Fleet" represents a Navy strategy to surveil, counter, overwhelm and attack enemies in a coordinated fashion - all while keeping sailors on host ships at safer distances. The small boats, many of them called Unmanned Surface Vessels, are designed to conduct ISR missions, find and destroy mines and launch a range of attacks including electronic warfare and even mounted guns. The concept is to use advanced computer algorithms bringing new levels of autonomy to surface warfare, enabling ships to coordinate information exchange, operate in tandem without colliding and launch combined assaults. "Ghost Fleet is really helping us in the Command and Control and coms arena. The demonstration will allow us to learn lessons about integrated payloads with USVs," Capt.

Given two or more Deep Neural Networks (DNNs) with the same or similar architectures, and trained on the same dataset, but trained with different solvers, parameters, hyper-parameters, regularization, etc., can we predict which DNN will have the best test accuracy, and can we do so without peeking at the test data? In this paper, we show how to use a new Theory of Heavy-Tailed Self-Regularization (HT-SR) to answer this. HT-SR suggests, among other things, that modern DNNs exhibit what we call Heavy-Tailed Mechanistic Universality (HT-MU), meaning that the correlations in the layer weight matrices can be fit to a power law with exponents that lie in common Universality classes from Heavy-Tailed Random Matrix Theory (HT-RMT). From this, we develop a Universal capacity control metric that is a weighted average of these PL exponents. Rather than considering small toy NNs, we examine over 50 different, large-scale pre-trained DNNs, ranging over 15 different architectures, trained on ImagetNet, each of which has been reported to have different test accuracies. We show that this new capacity metric correlates very well with the reported test accuracies of these DNNs, looking across each architecture (VGG16/.../VGG19, ResNet10/.../ResNet152, etc.). We also show how to approximate the metric by the more familiar Product Norm capacity measure, as the average of the log Frobenius norm of the layer weight matrices. Our approach requires no changes to the underlying DNN or its loss function, it does not require us to train a model (although it could be used to monitor training), and it does not even require access to the ImageNet data.

Random Matrix Theory (RMT) is applied to analyze the weight matrices of Deep Neural Networks (DNNs), including both production quality, pre-trained models such as AlexNet and Inception, and smaller models trained from scratch, such as LeNet5 and a miniature-AlexNet. Empirical and theoretical results clearly indicate that the empirical spectral density (ESD) of DNN layer matrices displays signatures of traditionally-regularized statistical models, even in the absence of exogenously specifying traditional forms of regularization, such as Dropout or Weight Norm constraints. Building on recent results in RMT, most notably its extension to Universality classes of Heavy-Tailed matrices, we develop a theory to identify \emph{5+1 Phases of Training}, corresponding to increasing amounts of \emph{Implicit Self-Regularization}. For smaller and/or older DNNs, this Implicit Self-Regularization is like traditional Tikhonov regularization, in that there is a `size scale' separating signal from noise. For state-of-the-art DNNs, however, we identify a novel form of \emph{Heavy-Tailed Self-Regularization}, similar to the self-organization seen in the statistical physics of disordered systems. This implicit Self-Regularization can depend strongly on the many knobs of the training process. By exploiting the generalization gap phenomena, we demonstrate that we can cause a small model to exhibit all 5+1 phases of training simply by changing the batch size.

Twitter is recently being used during crises to communicate with officials and provide rescue and relief operation in real time. The geographical location information of the event, as well as users, are vitally important in such scenarios. The identification of geographic location is one of the challenging tasks as the location information fields, such as user location and place name of tweets are not reliable. The extraction of location information from tweet text is difficult as it contains a lot of nonstandard English, grammatical errors, spelling mistakes, nonstandard abbreviations, and so on. This research aims to extract location words used in the tweet using a Convolutional Neural Network (CNN) based model. We achieved the exact matching score of 0.929, Hamming loss of 0.002, and F Our model was able to extract even three-to four-word long location references which is also evident from the exact matching score of over 92%. The findings of this paper can help in early event localization, emergency situations, real-time road traffic management, localized advertisement, and in various location-based services. Keywords: Location references, Tweets, Geo-locations, Named entity recognition, Gazetteer, Convolutional Neural Network 1. Introduction Tweets are very responsive to real-world events, and are sometimes even more immediate than traditional news channels. Therefore, it is possible to keep track of the latest information by following tweets. Several examples were seen when the news was first reported on Twitter, such as an airplane crash over the Hudson River in New York in the year 2009 (Sakaki et al., 2013), the death of former British Prime Minister Margaret Thatcher in April 2013 Preprint submitted to Elsevier January 25, 2019 Sakaki et al., 2013; Singh et al., 2017; Yuan & Liu, 2018). In an American Red Cross survey, a question was asked to individuals that "whom they contacted in an emergency?" The estimation and detection of location information of events and users from tweets are a major concern in relation to the above-mentioned tasks. Twitter provides three location information fields for sharing a user's location: (1) User location; (2) Place name; and (3) Geo-coordinate. The user location field has 140 character spaces (previously it was limited to 30 characters) in which the user can write his/her home location information while creating their profile. This field is optional to the user and the user can write any arbitrary words or leave it blank.

Recent advancements in graph neural networks (GNN) have led to state-of-the-art performance in various applications including chemo-informatics, question answering systems, and recommendation systems, to name a few. However, making these methods scalable to huge graphs such as web-mining remains a challenge. In particular, the existing methods for accelerating GNN are either not theoretically guaranteed in terms of approximation error or require at least linear time computation cost. In this paper, we propose a constant time approximation algorithm for the inference and training of GNN that theoretically guarantees arbitrary precision with arbitrary probability. The key advantage of the proposed algorithm is that the complexity is completely independent of the number of nodes, edges, and neighbors of the input. To the best of our knowledge, this is the first constant time approximation algorithm for GNN with theoretical guarantee. Through experiments using synthetic and real-world datasets, we evaluate our proposed approximation algorithm and show that the algorithm can successfully approximate GNN in constant time.

Lossy compression algorithms are typically designed and analyzed through the lens of Shannon's rate-distortion theory, where the goal is to achieve the lowest possible distortion (e.g., low MSE or high SSIM) at any given bit rate. However, in recent years, it has become increasingly accepted that "low distortion" is not a synonym for "high perceptual quality", and in fact optimization of one often comes at the expense of the other. In light of this understanding, it is natural to seek for a generalization of rate-distortion theory which takes perceptual quality into account. In this paper, we adopt the mathematical definition of perceptual quality recently proposed by Blau & Michaeli (2018), and use it to study the three-way tradeoff between rate, distortion, and perception. We show that restricting the perceptual quality to be high, generally leads to an elevation of the rate-distortion curve, thus necessitating a sacrifice in either rate or distortion. We prove several fundamental properties of this triple-tradeoff, calculate it in closed form for a Bernoulli source, and illustrate it visually on a toy MNIST example.

In unsupervised learning, there is no obvious straightforward loss function which can capture the major factors of variations and similarities. Since natural systems have smooth dynamics, an opportunity is lost if an unsupervised loss function remains static during the training process. The absence of concrete supervision suggests that smooth complex dynamics should be integrated as a substitute to the classical static loss functions to better make use of the gradual and uncertain knowledge acquired through self-supervision. In this paper, we propose Dynamic Autoencoder (DynAE), a new model for deep clustering that allows to solve a clustering-reconstruction trade-off by gradually and smoothly eliminating the reconstruction objective in favor of a construction one while preserving the space topology. Experimental evaluations on benchmark datasets show that our approach achieves state-of-the-art results compared to all the other autoencoder-based clustering methods.

This paper describes some applications of an incremental implementation of the principal component analysis (PCA). The algorithm updates the transformation coefficients matrix on-line for each new sample, without the need to keep all the samples in memory. The algorithm is formally equivalent to the usual batch version, in the sense that given a sample set the transformation coefficients at the end of the process are the same. The implications of applying the PCA in real time are discussed with the help of data analysis examples. In particular we focus on the problem of the continuity of the PCs during an on-line analysis.

Google is pouring an additional $3.1 million into Wikipedia, bringing its total contribution to the free encyclopedia over the past decade to more than $7.5 million, the company announced at the World Economic Forum Tuesday. A little over a third of those funds will go toward sustaining current efforts at the Wikimedia Foundation, the nonprofit that runs Wikipedia, and the remaining $2 million will focus on long-term viability through the organization's endowment. Google will also begin allowing Wikipedia editors to use several of its machine learning tools for free, the tech giant said. And Wikimedia and Google will soon broaden Project Tiger, a joint initiative they launched in 2017 to increase the number of Wikipedia articles written in underrepresented languages in India, to include 10 new languages in a handful of countries and regions. It will now be called GLOW, Growing Local Language Content on Wikipedia.

Beyond the lip service, I feel like we are in a trough of disillusionment on so many pivotal technologies. In particular, there are five emerging fields which have garnered tremendous excitement over the past couple of years, and yet this excitement is likely to die down in the next year as the reality sets in that the real impact of these technologies is still a good few years away. It is useful to understand these fading trends because while rising trends can be obvious, to take advantage of many of them in 2019, you needed to have acted a while ago. However, it is not too late to avoid wasting time and resources on these non-trends, at least for 2019, unless you are making significant long-term bets in which case, load up. It's useful to understand why we have hype and then disappointment before the real impact kicks in.