Artificial Intelligence (AI) simply means intelligence in machines, in contrast to natural intelligence found in humans and other natural organisms. Artificial intelligence gained its name and became a formal field of research in 1956, and initial work led to new tools for solving mathematical problems. However, researchers discovered that creating an AI is incredibly difficult, and progress slowed in the 1970s. More recently, increases in computing power and availability of massive data sets have set the groundwork for advances in AI. In particular, scientists have made giant strides in one particular application of AI, machine learning.

With resolution 1,000 times greater than a light microscope, electron microscopes are exceptionally good at imaging materials and detailing their properties. But like all technologies, they have some limitations. To overcome these limitations, scientists have traditionally focused on upgrading hardware, which is costly. But researchers at the U.S. Department of Energy's (DOE) Argonne National Laboratory are showing that advanced software developments can push their performance further. Argonne researchers have recently uncovered a way to improve the resolution and sensitivity of an electron microscope by using an artificial intelligence (AI) framework in a unique way.

In this special issue we are reprinting our top stories of the past year. This article first appeared online in our "Wonder" issue in February, 2021. Suppose aliens existed, and imagine that some of them had been watching our planet for its entire four and a half billion years. What would they have seen? Over most of that vast timespan, Earth's appearance altered slowly and gradually. Continents drifted; ice cover waxed and waned; successive species emerged, evolved, with many of them becoming extinct. But in just a tiny sliver of Earth's history--the last hundred centuries--the patterns of vegetation altered much faster than before. This signaled the start of agriculture--and later urbanization.

Modern neural language models widely used in tasks across NLP risk memorizing sensitive information from their training data. As models continue to scale up in parameters, training data, and compute, understanding memorization in language models is both important from a learning-theoretical point of view, and is practically crucial in real world applications. An open question in previous studies of memorization in language models is how to filter out "common" memorization. In fact, most memorization criteria strongly correlate with the number of occurrences in the training set, capturing "common" memorization such as familiar phrases, public knowledge or templated texts. In this paper, we provide a principled perspective inspired by a taxonomy of human memory in Psychology. From this perspective, we formulate a notion of counterfactual memorization, which characterizes how a model's predictions change if a particular document is omitted during training. We identify and study counterfactually-memorized training examples in standard text datasets. We further estimate the influence of each training example on the validation set and on generated texts, and show that this can provide direct evidence of the source of memorization at test time.

Looking back at 2021, it can surely be labelled as the year of large language models, with all the tech giants releasing models to stay ahead in the innovation game. In December itself, we saw back-to-back releases – DeepMind's 280 billion parameter transformer language model, Gopher, Google's Generalist Language Model (GLaM), a trillion weight model that uses sparsity, LG AI Research's artificial intelligence language model "Exaone", with capabilities of tuning 300 billion different parameters or variables. With innovations in language models accelerating at such a massive pace, can we possibly see a 100T large language model in the very near future? The idea is surely not too far-fetched if we look at the growth that tech companies have made, bringing out improved versions of the models that exist today in a span of just a few years. After the release of the GPT-3 autoregressive language model with 175 billion machine learning parameters from Open AI in 2020 (its predecessor, GPT-2, was over 100 times smaller, at 1.5 billion parameters), major efforts have gone into bringing out more such models by tech mammoths.

The Ocean Aware project, led by Innovasea and funded through Canada’s Ocean Supercluster, is developing a fish passage observation platform to monitor fish without the use of traditional tags.This will provide an alternative to standard tracking technology, such as acoustic telemetry fish tracking, which are often not appropriate for tracking at-risk fish species protected by legislation.Rather, the observation platform uses a combination of sensors including acoustic devices, visual and active sonar, and optical cameras.This will enable more in-depth scientific research and better support regulatory monitoring of at-risk fish species in fish passages or marine energy sites.Analysis of this data will require a robust and accurate method to automatically detect fish, count fish, and classify them by species in real-time using both sonar and optical cameras.To meet this need, we developed and tested an automated real-time deep learning framework combining state of the art convolutional neural networks and Kalman filters.First, we showed that an adaptation of the widely used YOLO machine learning model can accurately detect and classify eight species of fish from a public high resolution DIDSON imaging sonar dataset captured from the Ocqueoc River in Michigan, USA.Although there has been extensive research in the literature identifying particular fish such as eel vs non-eel and seal vs fish, to our knowledge this is the first successful application of deep learning for classi...

This month marks a year since we launched our focus series on the UN sustainable development goals (SDGs). Since then, we've published AI work pertaining to eight of the goals. We've had the pleasure of hearing from many experts with interesting stories to tell about their research. Here, we compile some of our favourite interviews and articles from the across the series. Interview with Lily Xu – applying machine learning to the prevention of illegal wildlife poaching Lily Xu tells us about her work applying machine learning and game theory to wildlife conservation.

Flux inversion is the process by which sources and sinks of a gas are identified from observations of gas mole fraction. The inversion often involves running a Lagrangian particle dispersion model (LPDM) to generate sensitivities between observations and fluxes over a spatial domain of interest. The LPDM must be run backward in time for every gas measurement, and this can be computationally prohibitive. To address this problem, here we develop a novel spatio-temporal emulator for LPDM sensitivities that is built using a convolutional variational autoencoder (CVAE). With the encoder segment of the CVAE, we obtain approximate (variational) posterior distributions over latent variables in a low-dimensional space. We then use a spatio-temporal Gaussian process emulator on the low-dimensional space to emulate new variables at prediction locations and time points. Emulated variables are then passed through the decoder segment of the CVAE to yield emulated sensitivities. We show that our CVAE-based emulator outperforms the more traditional emulator built using empirical orthogonal functions and that it can be used with different LPDMs. We conclude that our emulation-based approach can be used to reliably reduce the computing time needed to generate LPDM outputs for use in high-resolution flux inversions.

The 2021 NeurIPS Machine Learning for Combinatorial Optimization (ML4CO) competition was designed with the goal of improving state-of-the-art combinatorial optimization solvers by replacing key heuristic components with machine learning models. The competition's main scientific question was the following: is machine learning a viable option for improving traditional combinatorial optimization solvers on specific problem distributions, when historical data is available? This was motivated by the fact that in many practical scenarios, the data changes only slightly between the repetitions of a combinatorial optimization problem, and this is an area where machine learning models are particularly powerful at. This paper summarizes the solution and lessons learned by the Huawei EI-OROAS team in the dual task of the competition. The submission of our team achieved the second place in the final ranking, with a very close distance to the first spot. In addition, our solution was ranked first consistently for several weekly leaderboard updates before the final evaluation. We provide insights gained from a large number of experiments, and argue that a simple Graph Convolutional Neural Network (GCNNs) can achieve state-of-the-art results if trained and tuned properly.

Lots of things are being called "smart" these days -- everything from light bulbs to cars. Increasingly, the smarts come from some form of artificial intelligence or machine learning. AI is no longer limited to big central data centers. By moving it to the edge, enterprises can reduce latency, improve performance, reduce bandwidth requirements, and enable devices to continue to operate even when there's no network connectivity. One of the main drivers for the use of AI at the edge is that the sheer amount of data produced in the field would cripple the internet if it all had to be processed by centralized cloud computing solutions and traditional data centers.