The researchers from Massachusetts Institute of Technology (MIT) have developed a system that could bring deep learning neural networks to new -- and much smaller -- places, like the tiny computer chips in wearable medical devices, household appliances, and the 250 billion other objects that constitute the IoT. The system, called MCUNet, designs compact neural networks that deliver unprecedented speed and accuracy for deep learning on IoT devices, despite limited memory and processing power. The technology could facilitate the expansion of the IoT universe while saving energy and improving data security. MCUNet has two components needed for "tiny deep learning" -- the operation of neural networks on microcontrollers. One component is TinyEngine, an inference engine that directs resource management, akin to an operating system.

Deep learning (DL) has crept into almost all parts of Artificial Intelligence. DL methods constitute the state of the art for almost all tasks in image processing, natural language processing, recommendation systems…etc. Consequently, as more and more of these models are being deployed, the interpretability of these models is coming into question. Deep learning is a form of machine learning based on learning hierarchical, distributed models to solve difficult AI problems. Broadly speaking, these models work extremely well for most tasks, but because they have so many learnable parameters (order of millions), it's hard to predict what the effect of 1 single parameter, or a few of them, would be on the overall neural network.

Deep Learning and Machine Learning are one of the hottest tech fields to be in right now! The field is exploding with opportunities and career prospects. Machine/Deep Learning techniques are widely used in several sectors nowadays such as banking, healthcare, transportation and technology. Machine learning is the study of algorithms that teach computers to learn from experience. Through experience (i.e.: more training data), computers can continuously improve their performance. Deep Learning is a subset of Machine learning that utilizes multi-layer Artificial Neural Networks. Deep Learning is inspired by the human brain and mimics the operation of biological neurons. A hierarchical, deep artificial neural network is formed by connecting multiple artificial neurons in a layered fashion. The more hidden layers added to the network, the more

A Theory-guided Auto-Encoder (TgAE) framework is proposed for surrogate construction and is further used for uncertainty quantification and inverse modeling tasks. The framework is built based on the Auto-Encoder (or Encoder-Decoder) architecture of convolutional neural network (CNN) via a theory-guided training process. In order to achieve the theory-guided training, the governing equations of the studied problems can be discretized and the finite difference scheme of the equations can be embedded into the training of CNN. The residual of the discretized governing equations as well as the data mismatch constitute the loss function of the TgAE. The trained TgAE can be used to construct a surrogate that approximates the relationship between the model parameters and responses with limited labeled data. In order to test the performance of the TgAE, several subsurface flow cases are introduced. The results show the satisfactory accuracy of the TgAE surrogate and efficiency of uncertainty quantification tasks can be improved with the TgAE surrogate. The TgAE also shows good extrapolation ability for cases with different correlation lengths and variances. Furthermore, the parameter inversion task has been implemented with the TgAE surrogate and satisfactory results can be obtained.

Robust estimation of ground motions generated by scenario earthquakes is critical for many engineering applications. We leverage recent advances in Generative Adversarial Networks (GANs) to develop a new framework for synthesizing earthquake acceleration time histories. Our approach extends the Wasserstein GAN formulation to allow for the generation of ground-motions conditioned on a set of continuous physical variables. Our model is trained to approximate the intrinsic probability distribution of a massive set of strong-motion recordings from Japan. We show that the trained generator model can synthesize realistic 3-Component accelerograms conditioned on magnitude, distance, and $V_{s30}$. Our model captures the expected statistical features of the acceleration spectra and waveform envelopes. The output seismograms display clear P and S-wave arrivals with the appropriate energy content and relative onset timing. The synthesized Peak Ground Acceleration (PGA) estimates are also consistent with observations. We develop a set of metrics that allow us to assess the training process's stability and tune model hyperparameters. We further show that the trained generator network can interpolate to conditions where no earthquake ground motion recordings exist. Our approach allows the on-demand synthesis of accelerograms for engineering purposes.

We propose a novel neural network architecture based on dual quaternions which allow for a compact representation of informations with a main focus on describing rigid body movements. To cover the dynamic behavior inherent to rigid body movements, we propose recurrent architectures in the neural network. To further model the interactions between individual rigid bodies as well as external inputs efficiently, we incorporate a novel attention mechanism employing dual quaternion algebra. The introduced architecture is trainable by means of gradient based algorithms. We apply our approach to a parcel prediction problem where a rigid body with an initial position, orientation, velocity and angular velocity moves through a fixed simulation environment which exhibits rich interactions between the parcel and the boundaries.

Sentiment analysis of user-generated reviews or comments on products and services on social media can help enterprises to analyze the feedback from customers and take corresponding actions for improvement. To mitigate large-scale annotations, domain adaptation (DA) provides an alternate solution by learning a transferable model from another labeled source domain. Since the labeled data may be from multiple sources, multi-source domain adaptation (MDA) would be more practical to exploit the complementary information from different domains. Existing MDA methods might fail to extract some discriminative features in the target domain that are related to sentiment, neglect the correlations of different sources as well as the distribution difference among different sub-domains even in the same source, and cannot reflect the varying optimal weighting during different training stages. In this paper, we propose an instance-level multi-source domain adaptation framework, named curriculum cycle-consistent generative adversarial network (C-CycleGAN). Specifically, C-CycleGAN consists of three components: (1) pre-trained text encoder which encodes textual input from different domains into a continuous representation space, (2) intermediate domain generator with curriculum instance-level adaptation which bridges the gap across source and target domains, and (3) task classifier trained on the intermediate domain for final sentiment classification. C-CycleGAN transfers source samples at an instance-level to an intermediate domain that is closer to target domain with sentiment semantics preserved and without losing discriminative features. Further, our dynamic instance-level weighting mechanisms can assign the optimal weights to different source samples in each training stage. We conduct extensive experiments on three benchmark datasets and achieve substantial gains over state-of-the-art approaches.

Graph convolutional networks (GCNs) have been employed as a kind of significant tool on many graph-based applications recently. Inspired by convolutional neural networks (CNNs), GCNs generate the embeddings of nodes by aggregating the information of their neighbors layer by layer. However, the high computational and memory cost of GCNs due to the recursive neighborhood expansion across GCN layers makes it infeasible for training on large graphs. To tackle this issue, several sampling methods during the process of information aggregation have been proposed to train GCNs in a mini-batch Stochastic Gradient Descent (SGD) manner. Nevertheless, these sampling strategies sometimes bring concerns about insufficient information collection, which may hinder the learning performance in terms of accuracy and convergence. To tackle the dilemma between accuracy and efficiency, we propose to use aggregators with different granularities to gather neighborhood information in different layers. Then, a degree-based sampling strategy, which avoids the exponential complexity, is constructed for sampling a fixed number of nodes. Combining the above two mechanisms, the proposed model, named Mix-grained GCN (MG-GCN) achieves state-of-the-art performance in terms of accuracy, training speed, convergence speed, and memory cost through a comprehensive set of experiments on four commonly used benchmark datasets and a new Ethereum dataset.

In this paper, we introduce DSN (Deep Serial Number), a new watermarking approach that can prevent the stolen model from being deployed by unauthorized parties. Recently, watermarking in DNNs has emerged as a new research direction for owners to claim ownership of DNN models. However, the verification schemes of existing watermarking approaches are vulnerable to various watermark attacks. Different from existing work that embeds identification information into DNNs, we explore a new DNN Intellectual Property Protection mechanism that can prevent adversaries from deploying the stolen deep neural networks. Motivated by the success of serial number in protecting conventional software IP, we introduce the first attempt to embed a serial number into DNNs. Specifically, the proposed DSN is implemented in the knowledge distillation framework, where a private teacher DNN is first trained, then its knowledge is distilled and transferred to a series of customized student DNNs. During the distillation process, each customer DNN is augmented with a unique serial number, i.e., an encrypted 0/1 bit trigger pattern. Customer DNN works properly only when a potential customer enters the valid serial number. The embedded serial number could be used as a strong watermark for ownership verification. Experiments on various applications indicate that DSN is effective in terms of preventing unauthorized application while not sacrificing the original DNN performance. The experimental analysis further shows that DSN is resistant to different categories of attacks.

During a disaster event, images shared on social media helps crisis managers gain situational awareness and assess incurred damages, among other response tasks. Recent advances in computer vision and deep neural networks have enabled the development of models for real-time image classification for a number of tasks, including detecting crisis incidents, filtering irrelevant images, classifying images into specific humanitarian categories, and assessing the severity of damage. Despite several efforts, past works mainly suffer from limited resources (i.e., labeled images) available to train more robust deep learning models. In this study, we propose new datasets for disaster type detection, and informativeness classification, and damage severity assessment. Moreover, we relabel existing publicly available datasets for new tasks. We identify exact- and near-duplicates to form non-overlapping data splits, and finally consolidate them to create larger datasets. In our extensive experiments, we benchmark several state-of-the-art deep learning models and achieve promising results. We release our datasets and models publicly, aiming to provide proper baselines as well as to spur further research in the crisis informatics community.