Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about and how we talk about it change over time. This inherent dynamic nature of language contrasts with the current static language modelling paradigm, which trains and evaluates models on utterances from overlapping time periods. Despite impressive recent progress, we demonstrate that Transformer-XL language models perform worse in the realistic setup of predicting future utterances from beyond their training period, and that model performance becomes increasingly worse with time. We find that, while increasing model size alone--a key driver behind recent progress--does not solve this problem, having models that continually update their knowledge with new information can indeed mitigate this performance degradation over time. Hence, given the compilation of ever-larger language modelling datasets, combined with the growing list of language-model-based NLP applications that require up-to-date factual knowledge about the world, we argue that now is the right time to rethink the static way in which we currently train and evaluate our language models, and develop adaptive language models that can remain up-to-date with respect to our ever-changing and non-stationary world. We publicly release our dynamic, streaming language modelling benchmarks for WMT and arXiv to facilitate language model evaluation that takes temporal dynamics into account.

Our world is open-ended, non-stationary, and constantly evolving; thus what we talk about and how we talk about it change over time. This inherent dynamic nature of language contrasts with the current static language modelling paradigm, which trains and evaluates models on utterances from overlapping time periods. Despite impressive recent progress, we demonstrate that Transformer-XL language models perform worse in the realistic setup of predicting future utterances from beyond their training period, and that model performance becomes increasingly worse with time. We find that, while increasing model size alone--a key driver behind recent progress--does not solve this problem, having models that continually update their knowledge with new information can indeed mitigate this performance degradation over time. Hence, given the compilation of ever-larger language modelling datasets, combined with the growing list of language-model-based NLP applications that require up-to-date factual knowledge about the world, we argue that now is the right time to rethink the static way in which we currently train and evaluate our language models, and develop adaptive language models that can remain up-to-date with respect to our ever-changing and non-stationary world.

Data are the new gold, and single-cell genomics is a good match for data-hungry machine-learning algorithms. Machine learning has become increasingly crucial in single-cell genomics. Recent progress in machine learning6, primarily image classification, has been revolutionized by convolutional neural networks. The trick is to focus on local patches of an image and then build up the whole image step by step -- similar to, and inspired by, the way that hierarchies of receptive fields have been discovered in the human brain. Such convolutional neural networks have become state-of-the-art tools for several prediction problems in genomics and bioinformatics, such as the prediction of transcription-factor binding sites, analysis of genetic variants, sequence analysis and protein conformation prediction7.

GeoAI, or geospatial artificial intelligence, has become a trending topic and the frontier for spatial analytics in Geography. Although much progress has been made in exploring the integration of AI and Geography, there is yet no clear definition of GeoAI, its scope of research, or a broad discussion of how it enables new ways of problem solving across social and environmental sciences. This paper provides a comprehensive overview of GeoAI research used in large-scale image analysis, and its methodological foundation, most recent progress in geospatial applications, and comparative advantages over traditional methods. We organize this review of GeoAI research according to different kinds of image or structured data, including satellite and drone images, street views, and geo-scientific data, as well as their applications in a variety of image analysis and machine vision tasks. While different applications tend to use diverse types of data and models, we summarized six major strengths of GeoAI research, including (1) enablement of large-scale analytics; (2) automation; (3) high accuracy; (4) sensitivity in detecting subtle changes; (5) tolerance of noise in data; and (6) rapid technological advancement. As GeoAI remains a rapidly evolving field, we also describe current knowledge gaps and discuss future research directions.

Sensors and algorithms are two fundamental elements to construct intelligent systems. The recent progress in machine learning (ML) has produced great advancements in intelligent systems, owing to the powerful data analysis capability of ML algorithms. However, the performance of most systems is still hindered by sensing techniques that typically rely on rigid and bulky sensor devices, which cannot conform to irregularly curved and dynamic surfaces for high‐quality data acquisition. Skin‐like stretchable sensing technology with unique characteristics, such as high conformability, low modulus, and light weight, has been recently developed to solve this issue. Here, the recent progress in the fusion of emerging stretchable electronics and ML technology, for bioelectrical signal recognition, tactile perception, and multimodal integration is summarized, and the challenges and future developments are further discussed.

If you had asked me a year or two ago when Artificial General Intelligence (AGI) would be invented, I'd have told you that we were a long way off. Most experts were saying that AGI was decades away, and some were saying it might not happen at all. The consensus is -- was? -- that all the recent progress in AI concerns so-called "narrow AI," meaning systems that can only perform one specific task. An AGI, or a "strong AI," which could perform any task as well as a human being, is a much harder problem. It is so hard that there isn't a clear roadmap for achieving it, and few researchers are openly working on the topic. GPT-3 is the first model to shake that status-quo seriously. GPT-3 is the latest language model from the OpenAI team.

AI has made major strides in the last decade, from beating the world champion of Go, to learning how to program, to telling fantastical short stories. However, a basic human trait continues to elude machines: common sense. Common sense is a big term with plenty of baggage, but it typically includes shared background knowledge (I know certain facts about the world, like "the sky is blue," and I know that you know them too), elements of logic, and the ability to infer what is plausible. It looms large as one of the hardest and most central problems in AI. Machines can seem glaringly unintelligent when they lack common sense.

The availability of computational resources enables the simulation of increasingly intricate models in many fields of science. Scientists learn about the world by observing, manipulating, measuring, and abstracting. To make sure that they truly understand their system, and to gain insight beyond what experimental data can provide, many also turn to building mathematical models. Some models are based directly on fundamental physical laws, but most rely on approximations. The computational costs vary widely--from exactly solvable models to those that require all the computer power you can get.

Having an intuition for how machine learning algorithms work -- even in the most general sense -- is becoming an important business skill. As Andrew Ng has written: "Almost all of AI's recent progress is through one type, in which some input data (A) is used to quickly generate some simple response (B)." But how does this work? As you might imagine, many exciting machine learning problems can't be reduced to a simple equation like y mx b. But at their essence, supervised machine learning algorithms are solving for complex versions of m, based on labeled values for x and y, so that they can predict future y's from future x's.

Having an intuition for how machine learning algorithms work -- even in the most general sense -- is becoming an important business skill. As Andrew Ng has written: "Almost all of AI's recent progress is through one type, in which some input data (A) is used to quickly generate some simple response (B)." But how does this work? As you might imagine, many exciting machine learning problems can't be reduced to a simple equation like y mx b. But at their essence, supervised machine learning algorithms are solving for complex versions of m, based on labeled values for x and y, so that they can predict future y's from future x's.