With the rise of autonomous vehicles, smart video surveillance, facial detection and various people counting applications, fast and accurate object detection systems are rising in demand. These systems involve not only recognizing and classifying every object in an image, but localizing each one by drawing the appropriate bounding box around it. This makes object detection a significantly harder task than its traditional computer vision predecessor, image classification.

Deep learning bears promise for drug discovery, including advanced image analysis, prediction of molecular structure and function, and automated generation of innovative chemical entities with bespoke properties. Despite the growing number of successful prospective applications, the underlying mathematical models often remain elusive to interpretation by the human mind. There is a demand for 'explainable' deep learning methods to address the need for a new narrative of the machine language of the molecular sciences. This review summarizes the most prominent algorithmic concepts of explainable artificial intelligence, and dares a forecast of the future opportunities, potential applications, and remaining challenges.

Multi-task learning (MTL) is broadly used across various applications of machine learning and has several advantages in comparison with the single-task learning. Since layers are shared between different tasks and features are not repeatedly calculated for each task, the amount of memory used is reduced and the inference speed is improved. In addition, if tasks share complimentary information, they act as regularizers for each other which results in the improvement of the prediction performance of each task [1]. This has been proven in various areas such as detection and classification [2], computer vision [3, 4], depth estimation [5], natural language processing [6-8] and drug discovery [9]. The goal of this review paper is to provide an overview of various deep multi-task learning (dMTL) solutions and possible improvements in performance through efficient auxiliary tasks selection.

A key factor for ensuring safety in Autonomous Vehicles (AVs) is to avoid any abnormal behaviors under undesirable and unpredicted circumstances. As AVs increasingly rely on Deep Neural Networks (DNNs) to perform safety-critical tasks, different methods for uncertainty quantification have recently been proposed to measure the inevitable source of errors in data and models. However, uncertainty quantification in DNNs is still a challenging task. These methods require a higher computational load, a higher memory footprint, and introduce extra latency, which can be prohibitive in safety-critical applications. In this paper, we provide a brief and comparative survey of methods for uncertainty quantification in DNNs along with existing metrics to evaluate uncertainty predictions. We are particularly interested in understanding the advantages and downsides of each method for specific AV tasks and types of uncertainty sources.

One of the major concerns for neural network training is that the non-convexity of the associated loss functions may cause bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks. First, we point out that wide neural nets may have sub-optimal local minima under certain assumptions. Second, we discuss a few rigorous results on the geometric properties of wide networks such as "no bad basin", and some modifications that eliminate sub-optimal local minima and/or decreasing paths to infinity. Third, we discuss visualization and empirical explorations of the landscape for practical neural nets. Finally, we briefly discuss some convergence results and their relation to landscape results.

Graph convolutional networks (GCN) have been recently applied to semi-supervised classification problems with fewer labeled data and higher-dimensional features. Existing GCNs mostly rely on a first-order Chebyshev approximation of the graph wavelet-kernels. Such a generic propagation model may not always be well suited for the datasets. This work revisits the fundamentals of graph wavelet and explores the utility of spectral wavelet-kernels to signal propagation in the vertex domain. We first derive the conditions for representing the graph wavelet-kernels via vertex propagation. We next propose alternative propagation models for GCN layers based on Taylor expansions. We further analyze the choices of detailed propagation models. We test the proposed Taylor-based GCN (TGCN) in citation networks and 3D point clouds to demonstrate its advantages over traditional GCN methods.

Deep neural networks are typically trained under a supervised learning framework where a model learns a single task using labeled data. Instead of relying solely on labeled data, practitioners can harness unlabeled or related data to improve model performance, which is often more accessible and ubiquitous. Self-supervised pre-training for transfer learning is becoming an increasingly popular technique to improve state-of-the-art results using unlabeled data. It involves first pre-training a model on a large amount of unlabeled data, then adapting the model to target tasks of interest. In this review, we survey self-supervised learning methods and their applications within the sequential transfer learning framework. We provide an overview of the taxonomy for self-supervised learning and transfer learning, and highlight some prominent methods for designing pre-training tasks across different domains. Finally, we discuss recent trends and suggest areas for future investigation.

Robustly determining the optimal number of clusters in a data set is an essential factor in a wide range of applications. Cluster enumeration becomes challenging when the true underlying structure in the observed data is corrupted by heavy-tailed noise and outliers. Recently, Bayesian cluster enumeration criteria have been derived by formulating cluster enumeration as maximization of the posterior probability of candidate models. This article generalizes robust Bayesian cluster enumeration so that it can be used with any arbitrary Real Elliptically Symmetric (RES) distributed mixture model. Our framework also covers the case of M-estimators that allow for mixture models, which are decoupled from a specific probability distribution. Examples of Huber's and Tukey's M-estimators are discussed. We derive a robust criterion for data sets with finite sample size, and also provide an asymptotic approximation to reduce the computational cost at large sample sizes. The algorithms are applied to simulated and real-world data sets, including radar-based person identification, and show a significant robustness improvement in comparison to existing methods.

The theory of evolutionary computation for discrete search spaces has made a lot of progress during the last ten years. This survey summarizes some of the most important recent results obtained in this research area. It reviews important methods such as drift analysis, discusses theoretical insight on parameter tuning and parameter control, and summarizes the advances made for stochastic and dynamic problems. Furthermore, the survey highlights important results in the area of combinatorial optimization with a focus on parameterized complexity and the optimization of submodular functions. Finally, it gives an overview on the large amount of new important results for estimation of distribution algorithms.

Building trustworthy knowledge graphs for cyber-physical social systems (CPSS) is a challenge. In particular, current approaches relying on human experts have limited scalability, while automated approaches are often not accountable to users resulting in knowledge graphs of questionable quality. This paper introduces a novel pervasive knowledge graph builder that brings together automation, experts' and crowd-sourced citizens' knowledge. The knowledge graph grows via automated link predictions using genetic programming that are validated by humans for improving transparency and calibrating accuracy. The knowledge graph builder is designed for pervasive devices such as smartphones and preserves privacy by localizing all computations. The accuracy, practicality, and usability of the knowledge graph builder is evaluated in a real-world social experiment that involves a smartphone implementation and a Smart City application scenario. The proposed knowledge graph building methodology outperforms the baseline method in terms of accuracy while demonstrating its efficient calculations on smartphones and the feasibility of the pervasive human supervision process in terms of high interactions throughput. These findings promise new opportunities to crowd-source and operate pervasive reasoning systems for cyber-physical social systems in Smart Cities.