As we seek to deploy machine learning models beyond virtual and controlled domains, it is critical to analyze not only the accuracy or the fact that it works most of the time, but if such a model is truly robust and reliable. This paper studies strategies to implement adversary robustly trained algorithms towards guaranteeing safety in machine learning algorithms. We provide a taxonomy to classify adversarial attacks and defenses, formulate the Robust Optimization problem in a min-max setting and divide it into 3 subcategories, namely: Adversarial (re)Training, Regularization Approach, and Certified Defenses. We survey the most recent and important results in adversarial example generation, defense mechanisms with adversarial (re)Training as their main defense against perturbations. We also survey mothods that add regularization terms that change the behavior of the gradient, making it harder for attackers to achieve their objective. Alternatively, we've surveyed methods which formally derive certificates of robustness by exactly solving the optimization problem or by approximations using upper or lower bounds. In addition, we discuss the challenges faced by most of the recent algorithms presenting future research perspectives.

The development and advancements of technology are rapidly changing the nature of work and the workforce. The relentless growing connectivity and cognitive technologies are making it possible where humans and machines can work side-by-side at a shared workplace, enhancing the abilities of the human workforce. Today, as the workplaces are evolving to a flexible workforce driven by technology advances, software, automation, IoT, robotics, and artificial intelligence, among others, almost every job in every discipline is being revived. With the introduction of new technology, companies now have opportunities to power the workplace and augment their workforce to perform tasks effortlessly. The increasing proliferation of intelligent automation into the workplace is taking away a lot of tedious, repetitive works that used to overwhelm workflow, freeing up employees to focus on more valuable tasks.

Combining intelligent streaming analytics with real-time digital twins for aggregate analysis offers several benefits in a variety of real-world applications. Digital twins are finding broader use and playing a more important role in innovation. Many digital twins rely on continuous intelligence (CI) and artificial intelligence (AI) to ingest streams of data from sensors. The real-time analysis of that data use then used to make sense of current conditions, the status of different elements in a system, and determine what actions should be taken (if any). Their rising importance was noted in a recent MIT Sloan Management Review article.

With the rise of autonomous vehicles, smart video surveillance, facial detection and various people counting applications, fast and accurate object detection systems are rising in demand. These systems involve not only recognizing and classifying every object in an image, but localizing each one by drawing the appropriate bounding box around it. This makes object detection a significantly harder task than its traditional computer vision predecessor, image classification.

Deep learning bears promise for drug discovery, including advanced image analysis, prediction of molecular structure and function, and automated generation of innovative chemical entities with bespoke properties. Despite the growing number of successful prospective applications, the underlying mathematical models often remain elusive to interpretation by the human mind. There is a demand for 'explainable' deep learning methods to address the need for a new narrative of the machine language of the molecular sciences. This review summarizes the most prominent algorithmic concepts of explainable artificial intelligence, and dares a forecast of the future opportunities, potential applications, and remaining challenges.

Multi-task learning (MTL) is broadly used across various applications of machine learning and has several advantages in comparison with the single-task learning. Since layers are shared between different tasks and features are not repeatedly calculated for each task, the amount of memory used is reduced and the inference speed is improved. In addition, if tasks share complimentary information, they act as regularizers for each other which results in the improvement of the prediction performance of each task [1]. This has been proven in various areas such as detection and classification [2], computer vision [3, 4], depth estimation [5], natural language processing [6-8] and drug discovery [9]. The goal of this review paper is to provide an overview of various deep multi-task learning (dMTL) solutions and possible improvements in performance through efficient auxiliary tasks selection.

A key factor for ensuring safety in Autonomous Vehicles (AVs) is to avoid any abnormal behaviors under undesirable and unpredicted circumstances. As AVs increasingly rely on Deep Neural Networks (DNNs) to perform safety-critical tasks, different methods for uncertainty quantification have recently been proposed to measure the inevitable source of errors in data and models. However, uncertainty quantification in DNNs is still a challenging task. These methods require a higher computational load, a higher memory footprint, and introduce extra latency, which can be prohibitive in safety-critical applications. In this paper, we provide a brief and comparative survey of methods for uncertainty quantification in DNNs along with existing metrics to evaluate uncertainty predictions. We are particularly interested in understanding the advantages and downsides of each method for specific AV tasks and types of uncertainty sources.

One of the major concerns for neural network training is that the non-convexity of the associated loss functions may cause bad landscape. The recent success of neural networks suggests that their loss landscape is not too bad, but what specific results do we know about the landscape? In this article, we review recent findings and results on the global landscape of neural networks. First, we point out that wide neural nets may have sub-optimal local minima under certain assumptions. Second, we discuss a few rigorous results on the geometric properties of wide networks such as "no bad basin", and some modifications that eliminate sub-optimal local minima and/or decreasing paths to infinity. Third, we discuss visualization and empirical explorations of the landscape for practical neural nets. Finally, we briefly discuss some convergence results and their relation to landscape results.

Graph convolutional networks (GCN) have been recently applied to semi-supervised classification problems with fewer labeled data and higher-dimensional features. Existing GCNs mostly rely on a first-order Chebyshev approximation of the graph wavelet-kernels. Such a generic propagation model may not always be well suited for the datasets. This work revisits the fundamentals of graph wavelet and explores the utility of spectral wavelet-kernels to signal propagation in the vertex domain. We first derive the conditions for representing the graph wavelet-kernels via vertex propagation. We next propose alternative propagation models for GCN layers based on Taylor expansions. We further analyze the choices of detailed propagation models. We test the proposed Taylor-based GCN (TGCN) in citation networks and 3D point clouds to demonstrate its advantages over traditional GCN methods.

Deep neural networks are typically trained under a supervised learning framework where a model learns a single task using labeled data. Instead of relying solely on labeled data, practitioners can harness unlabeled or related data to improve model performance, which is often more accessible and ubiquitous. Self-supervised pre-training for transfer learning is becoming an increasingly popular technique to improve state-of-the-art results using unlabeled data. It involves first pre-training a model on a large amount of unlabeled data, then adapting the model to target tasks of interest. In this review, we survey self-supervised learning methods and their applications within the sequential transfer learning framework. We provide an overview of the taxonomy for self-supervised learning and transfer learning, and highlight some prominent methods for designing pre-training tasks across different domains. Finally, we discuss recent trends and suggest areas for future investigation.