From the simple embedded processor in your washing machine to powerful processors in data center servers, most computing today takes place on general-purpose programmable processors or CPUs. CPUs are attractive because they are easy to program and because large code bases exist for them. The programmability of CPUs stems from their execution of sequences of simple instructions, such as ADD or BRANCH; however, the energy required to fetch and interpret an instruction is 10x to 4000x more than that required to perform a simple operation such as ADD. This high overhead was acceptable when processor performance and efficiency were scaling according to Moore's Law.32 One could simply wait and an existing application would run faster and more efficiently. Our economy has become dependent on these increases in computing performance and efficiency to enable new features and new applications. Today, Moore's Law has largely ended,12 and we must look to alternative architectures with lower overhead, such as domain-specific accelerators, to continue scaling of performance and efficiency. There are several ways to realize domain-specific accelerators as discussed in the sidebar on accelerator options. A domain-specific accelerator is a hardware computing engine that is specialized for a particular domain of applications. Accelerators have been designed for graphics,26 deep learning,16 simulation,2 bioinformatics,49 image processing,38 and many other tasks. Accelerators can offer orders of magnitude improvements in performance/cost and performance/W compared to general-purpose computers. For example, our bioinformatics accelerator, Darwin,49 is up to 15,000x faster than a CPU at reference-based, long-read assembly. The performance and efficiency of accelerators is due to a combination of specialized operations, parallelism, efficient memory systems, and reduction of overhead. Domain-specific accelerators7 are becoming more pervasive and more visible, because they are one of the few remaining ways to continue to improve performance and efficiency now that Moore's Law has ended.22 Most applications require modifications to achieve high speed up on domain-specific accelerators. These applications are highly tuned to balance the performance of conventional processors with their memory systems. When specialization reduces the cost of processing to near zero, they become memory limited.

Victoria Stodden (vcs@stodden.net) is a statistician and associate professor at the University of Illinois at Urbana-Champaign, IL, USA. This material is based upon work supported by National Science Foundation Award #1941443.

GPUs became the hardware of choice for deep learning largely by coincidence. The chips were initially designed to quickly render graphics in applications such as video games. Unlike CPUs, which have four to eight complex cores for doing a variety of computation, GPUs have hundreds of simple cores that can perform only specific operations--but the cores can tackle their operations at the same time rather than one after another, shrinking the time it takes to complete an intensive computation. It didn't take long for the AI research community to realize that this massive parallelization also makes GPUs great for deep learning. Like graphics-rendering, deep learning involves simple mathematical calculations performed hundreds of thousands of times.

Neural networks trained on large amounts of data have led to incredible technological leaps affecting nearly every part of our lives. These advances have come at a cost -- namely the interpretability and explainability of data models. Corresponding with the complexity of the operation, the criteria for "choosing" a given output for an input becomes rather mysterious, leading some to refer to neural networks as a "black box" method. Deep neural networks work so marvelously because they learn efficient representations of data, and they are intentionally constrained to capture complex, non-linear patterns in the data. The trade off of recognizing non-linear patterns is comparable to losing the sense of sight, only to gain a more subtle perception of sound.

GlobalData research has found the top artificial intelligence (AI) influencers based on their performance and engagement online. Using research from GlobalData's Influencer platform, Verdict has named ten of the most influential people in artificial intelligence on Twitter during May 2020. Ronald van Loon is a recognised thought leader in technologies including AI, big data, IoT, machine learning, deep learning, 5G, predictive analytics, cloud, edge and data science. He currently serves as principal analyst and CEO of the Intelligent World, an influencer network that connects experts, businesses, and influencers to new audiences, helping them collaborate, create and share diverse content. Loon is of the opinion that AI has progressed at a furious pace over the past few years, and though it has usurped large chunks of the big data, the technology is nowhere near human intelligence.

Data scarcity is a perennial problem when applying deep learning (DL) to medical imaging. In vision tasks related to natural images, DL practitioners often have access to astoundingly large annotated data sets on which they can train. However, due to privacy concerns and the expense of creating them, access to large annotated data sets is rare in medical imaging. The natural follow-up question is: How can practitioners in the field of medical imaging best use DL given limited data? In this article, I'll discuss one approach to stretch the use of available data, called self-supervised learning.

For the Hemingway one, please note that the first 3 sentences are me, not GPT-3. In my various tweets, ' ' delimits the prompt (because there's nothing else in the playground UI right now which keeps visual track). Anyway, the interesting thing here is that I spent a good hour trying to get GPT-3 to do this. I knew it could do it based on how it could do Tom Swifties and the Turing dialogue and the other things, but I just couldn't quite figure out how to get it to do so. At one point, I was trying the idea of writing a plot summary and asking it to then "Please write a plot summary of Harry Potter in the style of Ernest Hemingway", and it copied my plot summary verbatim and then added on the sarcastic comment: It still sounds like it came straight from Harry Potter and the Philosopher's Stone.

BERT and GPT-2/3 have shown the enormous power of using generative models as pre-training for classification tasks. However, for images, pre-training is usually done with supervised or self-supervised objectives. This paper investigates how far you can get when applying the principles from the world of NLP to the world of images. Abstract: Inspired by progress in unsupervised representation learning for natural language, we examine whether similar models can learn useful representations for images. We train a sequence Transformer to auto-regressively predict pixels, without incorporating knowledge of the 2D input structure.

Originally, the artificial intelligence (AI) development and research organization, OpenAI, was founded as a nonprofit with the ambitious mission of ensuring artificial general intelligence would benefit all humanity. Since then, much has happened, starting with original founder Elon Musk leaving OpenAI's board in 2018. In July of 2019, the narrative changed yet again after the company received a $1 billion investment from Microsoft. The company exists today as a "capped-profit" organization. Most notably during this time, the company developed a text-generating language system it chose to not release "due to our concerns about malicious applications of the technology," then subsequently released said system.

In this paper, we propose a new end-to-end methodology to optimize the energy performance and the comfort, air quality and hygiene of large buildings. A metamodel based on a Transformer network is introduced and trained using a dataset sampled with a simulation program. Then, a few physical parameters and the building management system settings of this metamodel are calibrated using the CMA-ES optimization algorithm and real data obtained from sensors. Finally, the optimal settings to minimize the energy loads while maintaining a target thermal comfort and air quality are obtained using a multi-objective optimization procedure. The numerical experiments illustrate how this metamodel ensures a significant gain in energy efficiency while being computationally much more appealing than models requiring a huge number of physical parameters to be estimated.