We propose a data-driven approach to online multi-object tracking (MOT) that uses a convolutional neural network (CNN) for data association in a tracking-by-detection framework. The problem of multi-target tracking aims to assign noisy detections to a-priori unknown and time-varying number of tracked objects across a sequence of frames. A majority of the existing solutions focus on either tediously designing cost functions or formulating the task of data association as a complex optimization problem that can be solved effectively. Instead, we exploit the power of deep learning to formulate the data association problem as inference in a CNN. To this end, we propose to learn a similarity function that combines cues from both image and spatial features of objects. Our solution learns to perform global assignments in 3D purely from data, handles noisy detections and a varying number of targets, and is easy to train. We evaluate our approach on the challenging KITTI dataset and show competitive results. Our code is available at https://git.uwaterloo.ca/wise-lab/fantrack.

Episodic memory plays an important role in the behavior of animals and humans. It allows the accumulation of information about current state of the environment in a task-agnostic way. This episodic representation can be later accessed by down-stream tasks in order to make their execution more efficient. In this work, we introduce the neural architecture with shared episodic memory (SEM) for learning and the sequential execution of multiple tasks. We explicitly split the encoding of episodic memory and task-specific memory into separate recurrent sub-networks. An agent augmented with SEM was able to effectively reuse episodic knowledge collected during other tasks to improve its policy on a current task in the Taxi problem. Repeated use of episodic representation in continual learning experiments facilitated acquisition of novel skills in the same environment.

In recent years, deep neural networks have achieved excellent performance in many application scenarios such as face recognition and automatic speech recognition (ASR) [21]. However, deep neural networks are difficult to be interpreted. This defect severely restricts the development of deep learning in a few application scenarios, where the model's interpretability is needed. Moreover, the deep neural networks are very data-hungry due to the large complexity of the models, which means that the model's performance can decrease significantly when the size of the training data decreases [12, 22]. In many real tasks, due to the high cost of data collection and labeling, the amount of labeled training data may be insufficient to train a deep neural network. In such a situation, traditional learning methods such as random forest (R.F.) [3], gradient boosting decision tree (GBDT) [15, 5], support-vector machines (SVMs) [7], etc., are still good choices. By realizing that the essence of deep learning lies in the layer-by-layer processing, in-model feature transformation, and sufficient model complexity [33], recently Zhou and Feng [32] proposed the deep forest model and the gcForest algorithm to achieve forest representation learning. It can achieve excellent performance on a broad range of tasks, and can even perform well on small or middle-scale of data. Later on, a more efficient improvement was presented [24], and it shows that forest is able to do auto-encoder which thought to be a specialty of neural networks [13].

Simulation of physical models has been one of main driven forces for scientific computing. Physical phenomena at different scales, e.g., macroscopic scale, microscopic scale, etc., are characterized by Newton's laws of motion, Darcy's law, Maxwell's equations, Schrödinger equation, etc. Solving these equations efficiently, especially those nonlinear ones, has challenged computational scientists for decades and led to remarkable development in algorithms and in computing hardware. As the rise of machine learning, particularly deep learning, many researchers have been attempting to adopt artificial neural networks (NN) to represent the high-dimensional solutions or the low-dimensional solution maps. This paper proposes a variational training framework for solving the solution map of low-dimensional physical models via NNs. Here we emphasize solving a solution map in contrast with fitting a solution map, where solving can be to some extent viewed as unsupervised learning with input functions only and fitting refers to supervised learning with both input functions and the corresponding solutions. Solving the solution map for physical models is feasible due to an intrinsic difference between the physical problems and other data-driven problems, e.g., handwriting recognition, speech recognition, spam detection, etc.

Clinical decision making is challenging because of pathological complexity, as well as large amounts of heterogeneous data generated as part of routine clinical care. In recent years, machine learning tools have been developed to aid this process. Intensive care unit (ICU) admissions represent the most data dense and time-critical patient care episodes. In this context, prediction models may help clinicians determine which patients are most at risk and prioritize care. However, flexible tools such as artificial neural networks (ANNs) suffer from a lack of interpretability limiting their acceptability to clinicians. In this work, we propose a novel interpretable Bayesian neural network architecture which offers both the flexibility of ANNs and interpretability in terms of feature selection. In particular, we employ a sparsity inducing prior distribution in a tied manner to learn which features are important for outcome prediction. We evaluate our approach on the task of mortality prediction using two real-world ICU cohorts. In collaboration with clinicians we found that, in addition to the predicted outcome results, our approach can provide novel insights into the importance of different clinical measurements. This suggests that our model can support medical experts in their decision making process.

The correctness of deep neural networks is well-known to be vulnerable to small, 'adversarial' perturbations of their inputs. Although studying these attacks is valuable, they do not necessarily conform to any real-world threat model. This has led to interest in the generation of (and robustness to) unrestricted adversarial inputs, which are not constructed as small perturbations of correctly-classified ground-truth inputs. We introduce a novel algorithm to generate realistic unrestricted adversarial inputs, in the sense that they cannot reliably be distinguished from the training dataset by a human. This is achieved by modifying generative adversarial networks: a generator neural network is trained to construct examples that deceive a fixed target network (so they are adversarial) while also deceiving the usual co-training discriminator network (so they are realistic). Our approach is demonstrated by the generation of unrestricted adversarial inputs for a trained image classifier that is robust to perturbation-based attacks. We find that human judges are unable to identify which image out of ten was generated by our method about 50 percent of the time, providing evidence that they are moderately realistic.

Semi-supervised learning is sought for leveraging the unlabelled data when labelled data is difficult or expensive to acquire. Deep generative models (e.g., Variational Autoencoder (VAE)) and semisupervised Generative Adversarial Networks (GANs) have recently shown promising performance in semi-supervised classification for the excellent discriminative representing ability. However, the latent code learned by the traditional VAE is not exclusive (repeatable) for a specific input sample, which prevents it from excellent classification performance. In particular, the learned latent representation depends on a non-exclusive component which is stochastically sampled from the prior distribution. Moreover, the semi-supervised GAN models generate data from pre-defined distribution (e.g., Gaussian noises) which is independent of the input data distribution and may obstruct the convergence and is difficult to control the distribution of the generated data. To address the aforementioned issues, we propose a novel Adversarial Variational Embedding (AVAE) framework for robust and effective semi-supervised learning to leverage both the advantage of GAN as a high quality generative model and VAE as a posterior distribution learner. The proposed approach first produces an exclusive latent code by the model which we call VAE++, and meanwhile, provides a meaningful prior distribution for the generator of GAN. The proposed approach is evaluated over four different real-world applications and we show that our method outperforms the state-of-the-art models, which confirms that the combination of VAE++ and GAN can provide significant improvements in semisupervised classification.

The paper presents a novel approach of spoofing wireless signals by using a general adversarial network (GAN) to generate and transmit synthetic signals that cannot be reliably distinguished from intended signals. It is of paramount importance to authenticate wireless signals at the PHY layer before they proceed through the receiver chain. For that purpose, various waveform, channel, and radio hardware features that are inherent to original wireless signals need to be captured. In the meantime, adversaries become sophisticated with the cognitive radio capability to record, analyze, and manipulate signals before spoofing. Building upon deep learning techniques, this paper introduces a spoofing attack by an adversary pair of a transmitter and a receiver that assume the generator and discriminator roles in the GAN and play a minimax game to generate the best spoofing signals that aim to fool the best trained defense mechanism. The output of this approach is two-fold. From the attacker point of view, a deep learning-based spoofing mechanism is trained to potentially fool a defense mechanism such as RF fingerprinting. From the defender point of view, a deep learning-based defense mechanism is trained against potential spoofing attacks when an adversary pair of a transmitter and a receiver cooperates. The probability that the spoofing signal is misclassified as the intended signal is measured for random signal, replay, and GAN-based spoofing attacks. Results show that the GAN-based spoofing attack provides a major increase in the success probability of wireless signal spoofing even when a deep learning classifier is used as the defense.

Composed of long chains of amino acids, proteins perform these myriad tasks by folding themselves into precise 3D structures that govern how they interact with other molecules. Because a protein's shape determines its function and the extent of its dysfunction in disease, efforts to illuminate protein structures are central to all of molecular biology -- and in particular, therapeutic science and the development of lifesaving and life-altering medicines. In recent years, computational methods have made significant strides in predicting how proteins fold based on knowledge of their amino acid sequence. If fully realized, these methods have the potential to transform virtually all facets of biomedical research. Current approaches, however, are limited in the scale and scope of the proteins that can be determined.

Neuroscientists and computer vision scientists say a new dataset of unprecedented size--comprising brain scans of four volunteers who each viewed 5,000 images--will help researchers better understand how the brain processes images. Researchers at Carnegie Mellon University and Fordham University, reporting today in the journal Scientific Data, said acquiring functional magnetic resonance imaging (fMRI) scans at this scale presented unique challenges. Each volunteer participated in 20 or more hours of MRI scanning, challenging both their perseverance and the experimenters' ability to coordinate across scanning sessions. The extreme design decision to run the same individuals over so many sessions was necessary for disentangling the neural responses associated with individual images. The resulting dataset, dubbed BOLD5000, allows cognitive neuroscientists to better leverage the deep learning models that have dramatically improved artificial vision systems.