Deep learning is pervasive in our daily life, including self-driving cars, virtual assistants, social network services, healthcare services, face recognition, etc. However, deep neural networks demand substantial compute resources during training and inference. The machine learning community has mainly focused on model-level optimizations such as architectural compression of deep learning models, while the system community has focused on implementation-level optimization. In between, various arithmetic-level optimization techniques have been proposed in the arithmetic community. This article provides a survey on resource-efficient deep learning techniques in terms of model-, arithmetic-, and implementation-level techniques and identifies the research gaps for resource-efficient deep learning techniques across the three different level techniques. Our survey clarifies the influence from higher to lower-level techniques based on our resource-efficiency metric definition and discusses the future trend for resource-efficient deep learning research.

Autoregressive (AR) and Non-autoregressive (NAR) models have their own superiority on the performance and latency, combining them into one model may take advantage of both. Current combination frameworks focus more on the integration of multiple decoding paradigms with a unified generative model, e.g. Masked Language Model. However, the generalization can be harmful to the performance due to the gap between training objective and inference. In this paper, we aim to close the gap by preserving the original objective of AR and NAR under a unified framework. Specifically, we propose the Directional Transformer (Diformer) by jointly modelling AR and NAR into three generation directions (left-to-right, right-to-left and straight) with a newly introduced direction variable, which works by controlling the prediction of each token to have specific dependencies under that direction. The unification achieved by direction successfully preserves the original dependency assumption used in AR and NAR, retaining both generalization and performance. Experiments on 4 WMT benchmarks demonstrate that Diformer outperforms current united-modelling works with more than 1.5 BLEU points for both AR and NAR decoding, and is also competitive to the state-of-the-art independent AR and NAR models.

Both the industrialized and developing worlds are facing unprecedented demographic changes. Birth rates have reached a minimum in some of the world's largest countries, while literally billions of workers prepare to enter retirement. Researchers and policymakers have, over the last two decades, started to actively seek ways of dealing with the rising healthcare costs of aging populations. Across the board, AI has come to be considered the most advantageous solution. Not only does artificial intelligence automate basic tasks, removing the need for expensive human intervention in many cases, but it can be used to give a greater sense of privacy and discretion to patients.

Electrical utilities depend on short-term demand forecasting to proactively adjust production and distribution in anticipation of major variations. This systematic review analyzes 240 works published in scholarly journals between 2000 and 2019 that focus on applying Artificial Intelligence (AI), statistical, and hybrid models to short-term load forecasting (STLF). This work represents the most comprehensive review of works on this subject to date. A complete analysis of the literature is conducted to identify the most popular and accurate techniques as well as existing gaps. The findings show that although Artificial Neural Networks (ANN) continue to be the most commonly used standalone technique, researchers have been exceedingly opting for hybrid combinations of different techniques to leverage the combined advantages of individual methods. The review demonstrates that it is commonly possible with these hybrid combinations to achieve prediction accuracy exceeding 99%. The most successful duration for short-term forecasting has been identified as prediction for a duration of one day at an hourly interval. The review has identified a deficiency in access to datasets needed for training of the models. A significant gap has been identified in researching regions other than Asia, Europe, North America, and Australia.

We study a multivariate version of trend filtering, called Kronecker trend filtering or KTF, for the case in which the design points form a lattice in $d$ dimensions. KTF is a natural extension of univariate trend filtering (Steidl et al., 2006; Kim et al., 2009; Tibshirani, 2014), and is defined by minimizing a penalized least squares problem whose penalty term sums the absolute (higher-order) differences of the parameter to be estimated along each of the coordinate directions. The corresponding penalty operator can be written in terms of Kronecker products of univariate trend filtering penalty operators, hence the name Kronecker trend filtering. Equivalently, one can view KTF in terms of an $\ell_1$-penalized basis regression problem where the basis functions are tensor products of falling factorial functions, a piecewise polynomial (discrete spline) basis that underlies univariate trend filtering. This paper is a unification and extension of the results in Sadhanala et al. (2016, 2017). We develop a complete set of theoretical results that describe the behavior of $k^{\mathrm{th}}$ order Kronecker trend filtering in $d$ dimensions, for every $k \geq 0$ and $d \geq 1$. This reveals a number of interesting phenomena, including the dominance of KTF over linear smoothers in estimating heterogeneously smooth functions, and a phase transition at $d=2(k+1)$, a boundary past which (on the high dimension-to-smoothness side) linear smoothers fail to be consistent entirely. We also leverage recent results on discrete splines from Tibshirani (2020), in particular, discrete spline interpolation results that enable us to extend the KTF estimate to any off-lattice location in constant-time (independent of the size of the lattice $n$).

Nowadays Machine Learning (ML) techniques are extensively adopted in many socially sensitive systems, thus requiring to carefully study the fairness of the decisions taken by such systems. Many approaches have been proposed to address and to make sure there is no bias against individuals or specific groups which might originally come from biased training datasets or algorithm design. In this regard, we propose a fairness enforcing approach called EiFFFeL:Enforcing Fairness in Forests by Flipping Leaves which exploits tree-based or leaf-based post-processing strategies to relabel leaves of selected decision trees of a given forest. Experimental results show that our approach achieves a user defined group fairness degree without losing a significant amount of accuracy.

Amazon Alexa now is readily accessible in over 42 regions of the world and in a number of languages, making it more accessible than before. Alexa now can collaborate in much less prominent locations, such as the Cayman Islands and Cambodia, after initially being supported only in the United States, Canada, the United Kingdom, India, Japan, and Germany. However, it's not as simple as having to log into your Amazon account and order an Echo Dot or a full-fledged Amazon Echo smart speaker. We'll go over how to get Alexa if you live outside of the United States, which features you'll have access to, and some potential workarounds if you run into problems. If you really want Alexa, the very first thing you'll need is, well, an Alexa-enabled gadget.

The traditional data transformation procedure of extract, transform and load (ETL) is rapidly being turned on its head in a modern twist enabled by cloud technologies. The Cloud's lower costs, its flexibility and scalability, and the huge processing capability of cloud data warehouses, have driven a major change: the ability to load all data into the cloud, before transforming it. This trend means that ETL itself has been transformed--into extract, load and transform, or ELT. ELT offers several advantages, including retention of data granularity, reduced need for expensive software engineers and significantly reduced project turnaround times. Data is vital for organizations, who use it to understand their customers, identify new opportunities and support decision-makers with mission-critical and up-to-date information.

In this special issue we are reprinting our top stories of the past year. This article first appeared on Nautilus in our "Universality" issue in April, 2021. It all started with an odd pile of shells: a pile that, upon closer inspection, fell apart like a flower losing its petals, introducing a burned-out nature documentarian named Craig Foster--and, in time, the world--to the octopus hiding cleverly inside. Known simply as "her," she would become the star of My Octopus Teacher, the Oscar-nominated Netflix documentary and surprise pandemic hit that told the story of Foster's unlikely relationship with that eight-armed mollusk. Released in September 2020, it arrived at the perfect moment. Audiences exhausted by lockdowns and unrelenting 2020-ness were primed for escape into the undersea fantasia of South Africa's kelp forests, where Foster met her. Best-selling books like The Soul of an Octopus and Other Minds: The Octopus, the Sea, and the Deep Origins of Consciousness had whetted public curiosity about these uncannily intelligent creatures with whom humans last shared a common ancestor 600 million years ago. Yet while most writing about octopuses emphasizes their ostensibly alien, unknowable nature,1 and serious, science-minded nature documentaries elevate concern about biodiversity over sentiment for a single animal, My Octopus Teacher defied convention. It embraced Foster's feelings for the octopus, which over the course of a year evolved from curiosity to care--even to love. And though her own feelings were left for viewers to interpret, the film's indelible impression was of nature populated by species who are not only beautiful and exquisitely evolved and ecologically important, but highly sentient, too.

Artificial intelligence (AI) has had a profound impact on our society in recent years, but it's been around longer than you may realize. Many people attribute the beginning of AI to a paper written in 1950 by Alan Turing titled "Computer Machinery and Intelligence." The term artificial intelligence, however, was first coined in 1956 at a conference that took place at Dartmouth College in Hanover, New Hampshire. Since then, interest in AI has wavered. Its most recent resurgence can be attributed to IBM's Deep Blue chess-playing supercomputer and its question-answering machine Watson. Today, AI is part of our everyday lives – from facial recognition technology and ride-share apps to smart assistants.