We propose to add independent pseudo quantization noise to model parameters during training to approximate the effect of a quantization operator. This method, DiffQ, is differentiable both with respect to the unquantized parameters, and the number of bits used. Given a single hyper-parameter expressing the desired balance between the quantized model size and accuracy, DiffQ can optimize the number of bits used per individual weight or groups of weights, in a single training. We experimentally verify that our method outperforms state-of-the-art quantization techniques on several benchmarks and architectures for image classification, language modeling, and audio source separation. For instance, on the Wikitext-103 language modeling benchmark, DiffQ compresses a 16 layers transformer model by a factor of 8, equivalent to 4 bits precision, while losing only 0.5 points of perplexity. Code is available at: https://github.com/facebookresearch/diffq

Nothing tells of the ubiquity of satellite imagery like Google Maps. A completely unpaid service provides anyone with internet access a entire planet's worth of satellite imagery. While Google Maps is free, other paid alternatives exist which take photos of the earth's surface on a more frequent basis for commercial use. World governments also utilize their satellites for many domestic uses. As the availability of satellite imagery outpaces the ability of humans to look through them manually, an automated means to classify them must be developed.

Discover the essential news about IT & CIO news.

Randomized algorithms have propelled advances in artificial intelligence and represent a foundational research area in advancing AI for Science. Future advancements in DOE Office of Science priority areas such as climate science, astrophysics, fusion, advanced materials, combustion, and quantum computing all require randomized algorithms for surmounting challenges of complexity, robustness, and scalability. This report summarizes the outcomes of that workshop, "Randomized Algorithms for Scientific Computing (RASC)," held virtually across four days in December 2020 and January 2021.

This paper studies the construction of p-values for nonparametric outlier detection, taking a multiple-testing perspective. The goal is to test whether new independent samples belong to the same distribution as a reference data set or are outliers. We propose a solution based on conformal inference, a broadly applicable framework which yields p-values that are marginally valid but mutually dependent for different test points. We prove these p-values are positively dependent and enable exact false discovery rate control, although in a relatively weak marginal sense. We then introduce a new method to compute p-values that are both valid conditionally on the training data and independent of each other for different test points; this paves the way to stronger type-I error guarantees. Our results depart from classical conformal inference as we leverage concentration inequalities rather than combinatorial arguments to establish our finite-sample guarantees. Furthermore, our techniques also yield a uniform confidence bound for the false positive rate of any outlier detection algorithm, as a function of the threshold applied to its raw statistics. Finally, the relevance of our results is demonstrated by numerical experiments on real and simulated data.

Universities worldwide are experiencing a surge in enrollments, therefore campus estate managers are seeking continuous data on attendance patterns to optimize the usage of classroom space. As a result, there is an increasing trend to measure classrooms attendance by employing various sensing technologies, among which pervasive WiFi infrastructure is seen as a low cost method. In a dense campus environment, the number of connected WiFi users does not well estimate room occupancy since connection counts are polluted by adjoining rooms, outdoor walkways, and network load balancing. In this paper, we develop machine learning based models to infer classroom occupancy from WiFi sensing infrastructure. Our contributions are three-fold: (1) We analyze metadata from a dense and dynamic wireless network comprising of thousands of access points (APs) to draw insights into coverage of APs, behavior of WiFi connected users, and challenges of estimating room occupancy; (2) We propose a method to automatically map APs to classrooms using unsupervised clustering algorithms; and (3) We model classroom occupancy using a combination of classification and regression methods of varying algorithms. We achieve 84.6% accuracy in mapping APs to classrooms while the accuracy of our estimation for room occupancy is comparable to beam counter sensors with a symmetric Mean Absolute Percentage Error (sMAPE) of 13.10%.

Locating each node in a wireless sensor network is essential for starting the monitoring job and sending information about the area. One method that has been used in hard and inaccessible environments is randomly scattering each node in the area. In order to reduce the cost of using GPS at each node, some nodes should be equipped with GPS (anchors), Then using the belief propagation algorithm, locate other nodes. The number of anchor nodes must be reduced since they are expensive. Furthermore, the location of these nodes affects the algorithm's performance. Using multi-objective optimization, an algorithm is introduced in this paper that minimizes the estimated location error and the number of anchor nodes. According to simulation results, This algorithm proposes a set of solutions with less energy consumption and less error than similar algorithms.

Abstract--In recent years, Machine Learning (ML) algorithms have become widespread in all fields of Remote Sensing (RS) and Earth Observation (EO). This has allowed a rapid development of new procedures to solve problems affecting these sectors. In this context, the authors of this work aim to present a novel method for filtering the speckle noise from Sentinel-1 data by applying Deep Learning (DL) algorithms, based on Convolutional Neural Networks (CNNs). The obtained results, if compared with the state of the art, show a clear improvement in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), by proving the effectiveness of the proposed architecture. Moreover, the generated open-source code and dataset have been made available for further developments and investigation by interested researchers.

Electrical batteries are increasingly crucial in a variety of applications, from integration of intermittent energy sources with demand, to unlocking carbon-free power for the transportation sector through electric vehicles (EVs), trains and ships, to a host of advanced electronics and robotic applications. A key challenge however is that batteries degrade quickly with operating conditions. It is currently difficult to estimate battery health without interrupting the operation of the battery or without going through a lengthy procedure of charge-discharge that requires specialized equipment. In work recently published by Nature Machine Intelligence, researchers from the Smart Systems Group at Heriot-Watt University in Edinburgh, UK working together with researchers from the CALCE group at the University of Maryland in the US developed a new method to estimate battery health irrespective of operating conditions and battery design or chemistry, by feeding artificial intelligence (AI) algorithms with the raw battery voltage and current operational data. Darius Roman, the Ph.D. student that designed the AI framework said: "To date, the progress of data-driven models for battery degradation relies on the development of algorithms that carry out inference faster. Whilst researchers often spend a considerable amount of time on model or algorithm development, very few people take the time to understand the engineering context in which the algorithms are applied. By contrast, our work is built from the ground up. We first understand battery degradation through collaborations with the CALCE group at the University of Maryland, where in-house degradation testing of batteries was carried out. We then concentrate on the data, where we engineer features that capture battery degradation, we select the most important features and only then we deploy the AI techniques to estimate battery health."

TORONTO--Sometimes a problem can become its own solution. For CEA-Leti scientists, it means that traits of resistive-RAM (ReRAM) devices that have been previously considered as "non-ideal" may be the answer to overcoming barriers to developing ReRAM-based edge-learning systems, as outlined in a recent Nature Electronics publication titled "In-situ learning using intrinsic memristor variability via Markov chain Monte Carlo sampling." It describes how RRAM, or memristor, technology can be used to create intelligent systems that learn locally at the edge, independent of the cloud. Thomas Dalgaty, a CEA-Leti scientist at France's Université Grenoble, explained how the team were able to navigate the intrinsic non-idealities of ReRAM technology--the learning algorithms used in current ReRAM-based edge approaches cannot be reconciled with device programming randomness, or variability, among others. In a telephone interview with EE Times, he said the solution was to implement a Markov Chain Monte Carlo (MCMC) sampling learning algorithm in a fabricated chip that acts as a Bayesian machine-learning model, which actively exploited memristor randomness. For the purposes of the research, Dalgaty said it's important to clearly define what is meant by an edge system.