In recent years a number of regularization methods have been introduced which force the latent activations of an autoencoder or deep neural network to conform to either a hyperspherical or Gaussian distribution, in order to encourage diversity in the latent vectors, or to minimize the implicit rank of the distribution in latent space. Variational Autoencoders (VAE) (Kingma and Welling, 2014) and related variational methods such as β-VAE (Higgins et al., 2017) force the latent distribution to match a known prior distribution by minimizing the Kullback-Leibler divergence. Normally, a standard Gaussian distribution is used as the prior, but alternatives such as the hyperspherical distribution have also been investigated in the literature due to certain advantages (Davidson et al., 2018). More recently, deterministic alternatives have been proposed such as Wasserstein AutoEncoder (WAE) (Tolstikhin et al., 2018), VQ-VAE (van den Oord et al., 2017) and RAE (Ghosh et al., 2020). Several existing methods encourage diversity by maximizing pairwise dissimilarity between items, drawing inspiration in part from a 1904 paper by J.J. Thomson in which various classical models are proposed for maintaining the electrons of an atom in an appropriate formation around the nucleus (Thomson, 1904). Hyperspherical Energy Minimization (Liu et al., 2018) has been used to regularize the hidden unit

Computer vision (CV) techniques try to mimic human capabilities of visual perception to support labor-intensive and time-consuming tasks like the recognition and localization of critical objects. Nowadays, CV increasingly relies on artificial intelligence (AI) to automatically extract useful information from images that can be utilized for decision support and business process automation. However, the focus of extant research is often exclusively on technical aspects when designing AI-based CV systems while neglecting socio-technical facets, such as trust, control, and autonomy. For this purpose, we consider the design of such systems from a hybrid intelligence (HI) perspective and aim to derive prescriptive design knowledge for CV-based HI systems. We apply a reflective, practice-inspired design science approach and accumulate design knowledge from six comprehensive CV projects. As a result, we identify four design-related mechanisms (i.e., automation, signaling, modification, and collaboration) that inform our derived meta-requirements and design principles. This can serve as a basis for further socio-technical research on CV-based HI systems.

We argue that an explainable artificial intelligence must possess a rationale for its decisions, be able to infer the purpose of observed behaviour, and be able to explain its decisions in the context of what its audience understands and intends. To address these issues we present four novel contributions. Firstly, we define an arbitrary task in terms of perceptual states, and discuss two extremes of a domain of possible solutions. Secondly, we define the intensional solution. Optimal by some definitions of intelligence, it describes the purpose of a task. An agent possessed of it has a rationale for its decisions in terms of that purpose, expressed in a perceptual symbol system grounded in hardware. Thirdly, to communicate that rationale requires natural language, a means of encoding and decoding perceptual states. We propose a theory of meaning in which, to acquire language, an agent should model the world a language describes rather than the language itself. If the utterances of humans are of predictive value to the agent's goals, then the agent will imbue those utterances with meaning in terms of its own goals and perceptual states. In the context of Peircean semiotics, a community of agents must share rough approximations of signs, referents and interpretants in order to communicate. Meaning exists only in the context of intent, so to communicate with humans an agent must have comparable experiences and goals. An agent that learns intensional solutions, compelled by objective functions somewhat analogous to human motivators such as hunger and pain, may be capable of explaining its rationale not just in terms of its own intent, but in terms of what its audience understands and intends. It forms some approximation of the perceptual states of humans.

Currently, the most widespread neural network architecture for training language models is the so called BERT which led to improvements in various NLP tasks. In general, the larger the number of parameters in a BERT model, the better the results obtained in these NLP tasks. Unfortunately, the memory consumption and the training duration drastically increases with the size of these models, though. In this article, we investigate various training techniques of smaller BERT models and evaluate them on five public German NER tasks of which two are introduced by this article. We combine different methods from other BERT variants like ALBERT, RoBERTa, and relative positional encoding. In addition, we propose two new fine-tuning techniques leading to better performance: CSE-tagging and a modified form of LCRF. Furthermore, we introduce a new technique called WWA which reduces BERT memory usage and leads to a small increase in performance.

Linear interpolation between initial neural network parameters and converged parameters after training with stochastic gradient descent (SGD) typically leads to a monotonic decrease in the training objective. This Monotonic Linear Interpolation (MLI) property, first observed by Goodfellow et al. (2014) persists in spite of the non-convex objectives and highly non-linear training dynamics of neural networks. Extending this work, we evaluate several hypotheses for this property that, to our knowledge, have not yet been explored. Using tools from differential geometry, we draw connections between the interpolated paths in function space and the monotonicity of the network - providing sufficient conditions for the MLI property under mean squared error. While the MLI property holds under various settings (e.g. network architectures and learning problems), we show in practice that networks violating the MLI property can be produced systematically, by encouraging the weights to move far from initialization. The MLI property raises important questions about the loss landscape geometry of neural networks and highlights the need to further study their global properties.

In the early 2010s, a crisis of reproducibility rocked the field of psychology. Following a period of reflection, the field has responded with radical reform of its scientific practices. More recently, similar questions about the reproducibility of machine learning research have also come to the fore. In this short paper, we present select ideas from psychology's reformation, translating them into relevance for a machine learning audience.

We extend the neural basis expansion analysis (NBEATS) to incorporate exogenous factors. The resulting method, called NBEATSx, improves on a well performing deep learning model, extending its capabilities by including exogenous variables and allowing it to integrate multiple sources of useful information. To showcase the utility of the NBEATSx model, we conduct a comprehensive study of its application to electricity price forecasting (EPF) tasks across a broad range of years and markets. We observe state-of-the-art performance, significantly improving the forecast accuracy by nearly 20% over the original NBEATS model, and by up to 5% over other well established statistical and machine learning methods specialized for these tasks. Additionally, the proposed neural network has an interpretable configuration that can structurally decompose time series, visualizing the relative impact of trend and seasonal components and revealing the modeled processes' interactions with exogenous factors. To assist related work we made the code available in https://github.com/cchallu/nbeatsx.

Most chatbot literature that focuses on improving the fluency and coherence of a chatbot, is dedicated to making chatbots more human-like. However, very little work delves into what really separates humans from chatbots -- humans intrinsically understand the effect their responses have on the interlocutor and often respond with an intention such as proposing an optimistic view to make the interlocutor feel better. This paper proposes an innovative framework to train chatbots to possess human-like intentions. Our framework includes a guiding chatbot and an interlocutor model that plays the role of humans. The guiding chatbot is assigned an intention and learns to induce the interlocutor to reply with responses matching the intention, for example, long responses, joyful responses, responses with specific words, etc. We examined our framework using three experimental setups and evaluated the guiding chatbot with four different metrics to demonstrate flexibility and performance advantages. Additionally, we performed trials with human interlocutors to substantiate the guiding chatbot's effectiveness in influencing the responses of humans to a certain extent. Code will be made available to the public.

The traditional path of drug development is lengthy, expensive, and suffers from high failure rates – scientists test millions of molecules, however, only a handful progress to preclinical or clinical testing. Embracing innovation, particularly automated technologies, is essential to reduce the complexity associated with drug discovery and circumvent the high cost and time spent bringing a medicine to market. The subsequent sections will highlight examples of how ML can be used for drug repurposing and to discover novel antibiotics. The application of ML strategies to enhance image-based profiling and accelerate drug discovery will also be discussed. Drug discovery is often thought of as a complex jigsaw puzzle where connecting workflows and data are essential pieces.

Sick has launched its first machine vision camera with a pre-installed deep learning app, making it easy to create custom inspections of complex or irregular-shaped goods, packaging and assemblies, especially those that have previously defied automation. The Sick Intelligent Inspection Deep Learning App runs on the company's recently-launched Inspector P621 2D programmable camera. The combined package allows machine-builders and end-users to set up vision classifications using AI (artificial intelligence) in a fraction of the time and cost needed to program traditional vision systems to perform challenging inspection tasks based on recognising preset rules and patterns. The new system can tackle applications where it was previously difficult to achieve consistent, repeatable quality inspections. It makes automation practical and affordable for complex tasks such as sorting fresh fruit and vegetables, checking the orientation of timber profiles by recognising the annual ring structure, checking leather car seats for creases or flaws, or inspecting the integrity of solder in surface-mount assemblies.