Monte Carlo methods provide one solution to represent neural network parameter posteriors as ensembles of networks, but this requires In this paper, we present a general framework large amounts of both storage and compute time (Neal, for distilling expectations with respect to the 1996; Welling and Teh, 2011). Bayesian posterior distribution of a deep neural network classifier, extending prior work on To help overcome these problems, Balan et al. (2015) introduced the Bayesian Dark Knowledge framework. The a model training method referred to as Bayesian proposed framework takes as input "teacher" Dark Knowledge (BDK). BDK attempts to compress (or and student model architectures and a general distill) the Bayesian posterior predictive distribution induced posterior expectation of interest. The distillation by the full parameter posterior of a "teacher" network method performs an online compression (represented via a set of Mote Carlo samples) into a of the selected posterior expectation using iteratively significantly more compact "student" network. The major generated Monte Carlo samples. We advantage of BDK is that the computational complexity focus on the posterior predictive distribution of prediction at test time is drastically reduced compared and expected entropy as distillation targets. We to directly computing predictions via Monte Carlo averages investigate several aspects of this framework over the set of teacher network samples (the teacher including the impact of uncertainty and the ensemble).

Recently, there has been a lot of interest in ensuring algorithmic fairness in machine learning where the central question is how to prevent sensitive information (e.g. knowledge about the ethnic group of an individual) from adding "unfair" bias to a learning algorithm (Feldman et al. (2015), Zemel et al. (2013)). This has led to several debiasing algorithms on word embeddings (Qian et al. (2019) , Bolukbasi et al. (2016)), coreference resolution (Zhao et al. (2018a)), semantic role labeling (Zhao et al. (2017)), etc. Most of these existing work deals with explicit sensitive features such as gender, occupations or race which doesn't work with data where such features are not captured due to privacy concerns. In this research work, we aim to achieve classification parity across explicit as well as implicit sensitive features. We define explicit cohorts as groups of people based on explicit sensitive attributes provided in the data (age, gender, race) whereas implicit cohorts are defined as groups of people with similar language usage. We obtain implicit cohorts by clustering embeddings of each individual trained on the language generated by them using a language model. We achieve two primary objectives in this work : [1.] We experimented and discovered classification performance differences across cohorts based on implicit and explicit features , [2] We improved classification parity by introducing modification to the loss function aimed to minimize the range of model performances across cohorts.

Conventional adversarial defenses reduce classification accuracy whether or not a model is under attacks. Moreover, most of image processing based defenses are defeated due to the problem of obfuscated gradients. In this paper, we propose a new adversarial defense which is a defensive transform for both training and test images inspired by perceptual image encryption methods. The proposed method utilizes a block-wise pixel shuffling method with a secret key. The experiments are carried out on both adaptive and non-adaptive maximum-norm bounded white-box attacks while considering obfuscated gradients. The results show that the proposed defense achieves high accuracy (91.55 %) on clean images and (89.66 %) on adversarial examples with noise distance of 8/255 on CIFAR-10 dataset. Thus, the proposed defense outperforms state-of-the-art adversarial defenses including latent adversarial training, adversarial training and thermometer encoding.

Recurrent Neural Networks (RNNs) are widely used for online regression due to their ability to generalize nonlinear temporal dependencies. As an RNN model, Long-Short-Term-Memory Networks (LSTMs) are commonly preferred in practice, as these networks are capable of learning long-term dependencies while avoiding the vanishing gradient problem. However, due to their large number of parameters, training LSTMs requires considerably longer training time compared to simple RNNs (SRNNs). In this paper, we achieve the online regression performance of LSTMs with SRNNs efficiently. To this end, we introduce a first-order training algorithm with a linear time complexity in the number of parameters. We show that when SRNNs are trained with our algorithm, they provide very similar regression performance with the LSTMs in two to three times shorter training time. We provide strong theoretical analysis to support our experimental results by providing regret bounds on the converge rate of our algorithm. Through an extensive set of experiments, we verify our theoretical work and demonstrate significant performance improvements of our algorithm with respect to LSTMs and the state-of-the-art training methods.

Due to their proven efficiency, machine-learning systems are deployed in a wide range of complex real-life problems. More specifically, Spiking Neural Networks (SNNs) emerged as a promising solution to the accuracy, resource-utilization, and energy-efficiency challenges in machine-learning systems. While these systems are going mainstream, they have inherent security and reliability issues. In this paper, we propose NeuroAttack, a cross-layer attack that threatens the SNNs integrity by exploiting low-level reliability issues through a high-level attack. Particularly, we trigger a fault-injection based sneaky hardware backdoor through a carefully crafted adversarial input noise. Our results on Deep Neural Networks (DNNs) and SNNs show a serious integrity threat to state-of-the art machine-learning techniques.

This book discusses the necessity and perhaps urgency for the regulation of algorithms on which new technologies rely; technologies that have the potential to re-shape human societies. From commerce and farming to medical care and education, it is difficult to find any aspect of our lives that will not be affected by these emerging technologies. At the same time, artificial intelligence, deep learning, machine learning, cognitive computing, blockchain, virtual reality and augmented reality, belong to the fields most likely to affect law and, in particular, administrative law. The book examines universally applicable patterns in administrative decisions and judicial rulings. First, similarities and divergence in behavior among the different cases are identified by analyzing parameters ranging from geographical location and administrative decisions to judicial reasoning and legal basis. As it turns out, in several of the cases presented, sources of general law, such as competition or labor law, are invoked as a legal basis, due to the lack of current specialized legislation. This book also investigates the role and significance of national and indeed supranational regulatory bodies for advanced algorithms and considers ENISA, an EU agency that focuses on network and information security, as an interesting candidate for a European regulator of advanced algorithms. Lastly, it discusses the involvement of representative institutions in algorithmic regulation.

The current Air Traffic Management (ATM) system worldwide has reached its limits in terms of predictability, efficiency and cost effectiveness. Different initiatives worldwide propose trajectory-oriented transformations that require high fidelity aircraft trajectory planning and prediction capabilities, supporting the trajectory life cycle at all stages efficiently. Recently proposed data-driven trajectory prediction approaches provide promising results. In this paper we approach the data-driven trajectory prediction problem as an imitation learning task, where we aim to imitate experts "shaping" the trajectory. Towards this goal we present a comprehensive framework comprising the Generative Adversarial Imitation Learning state of the art method, in a pipeline with trajectory clustering and classification methods. This approach, compared to other approaches, can provide accurate predictions for the whole trajectory (i.e. with a prediction horizon until reaching the destination) both at the pre-tactical (i.e. starting at the departure airport at a specific time instant) and at the tactical (i.e. from any state while flying) stages, compared to state of the art approaches.

Deep Learning (DL) techniques allow ones to train models from a dataset to solve tasks. DL has attracted much interest given its fancy performance and potential market value, while security issues are amongst the most colossal concerns. However, the DL models may be prone to the membership inference attack, where an attacker determines whether a given sample is from the training dataset. Efforts have been made to hinder the attack but unfortunately, they may lead to a major overhead or impaired usability. In this paper, we propose and implement DAMIA, leveraging Domain Adaptation (DA) as a defense aginist membership inference attacks. Our observation is that during the training process, DA obfuscates the dataset to be protected using another related dataset, and derives a model that underlyingly extracts the features from both datasets. Seeing that the model is obfuscated, membership inference fails, while the extracted features provide supports for usability. Extensive experiments have been conducted to validates our intuition. The model trained by DAMIA has a negligible footprint to the usability. Our experiment also excludes factors that may hinder the performance of DAMIA, providing a potential guideline to vendors and researchers to benefit from our solution in a timely manner.

The task of transliteration of named entities from one language into another is complicated and considered as one of the challenging tasks in machine translation (MT). To build a well performed transliteration system, we apply well-established techniques based on Hybrid Deep Learning. The system based on convolutional neural network (CNN) followed by Bi-LSTM and CRF. The proposed hybrid mechanism is examined on ANERCorp and Kalimat corpus. The results show that the neural machine translation approach can be employed to build efficient machine transliteration systems achieving state-of-the-art results for Arabic – English language.

Survival analysis has currently become a hot topic because it has been proven to be useful for understanding the relationships between patients’ covariates (e.g. clinical and genetic features) and the effectiveness of various treatment options. In this study, we study survival analysis of breast cancer patient with gene methylation data and clinical data. We propose a novel method for survival prediction by bidirectional LSTM network and ordinal Cox model. First, gene methylation expression data and clinical data are merged and filtered to keep matching. To reduce the gene expression feature dimension, weight gene co-expression network analysis (WGCNA) algorithm is used to obtain the cluster eigengenes. Then, the eigengenes will be input features for machine learning network. We build a cox proportional hazards model for survival analysis and use LSTM method to predict patient survival risk. We use the leave_one_out method for cross validation, and use the concordance index (C-index) to evaluate the prediction performance. Stringent cross-validation tests on the benchmark dataset demonstrated the efficacy of the proposed method, which achieves very competitive performances to existing state-of-the-art methods.