This paper proposes a novel approach to train deep neural networks in a parallelized manner by unlocking the layer-wise dependency of backpropagation training. The approach employs additional modules called local critic networks besides the main network model to be trained, which estimate the output of the main network in order to obtain error gradients without complete feedforward and backward propagation processes. We propose a cascaded learning strategy for these local networks so that parallelized training of different layer groups is possible. Experimental results show the effectiveness of the proposed approach and suggest guidelines for determining appropriate algorithm parameters. In addition, we demonstrate that the approach can be also used for structural optimization of neural networks, computationally efficient progressive inference, and ensemble classification for performance improvement.

The Multimedia Neural Cognitive Computing (MNCC) model was designed based on the nervous mechanism and cognitive architecture. Furthermore, the semantic-oriented hierarchical Cross-media Neural Cognitive Computing (CNCC) framework was proposed based on MNCC, and formal description and analysis for CNCC was given. It would effectively improve the performance of semantic processing for multimedia information, and has far-reaching significance for exploration and realization brain-inspired computing. Keywords Deep learning·cognitive computing·brain-inspired computing·cross-media neural cognitive computing·multimedia neural cognitive computing 1 Introduction The brain-inspired computing (BIC) is the integration of neural cognitive science and information technology. It would realize state-of-the-art computing system which has advanced in energy consumption, computing ability and efficiency.

In this report we described our approach achieves $53\%$ of unweighted accuracy over $7$ emotions and $0.05$ and $0.09$ mean squared errors for arousal and valence in OMG emotion recognition challenge. Our results were obtained with ensemble of single modality models trained on voice and face data from video separately. We consider each stream as a sequence of frames. Next we estimated features from frames and handle it with recurrent neural network. As audio frame we mean short $0.4$ second spectrogram interval. For features estimation for face pictures we used own ResNet neural network pretrained on AffectNet database. Each short spectrogram was considered as a picture and processed by convolutional network too. As a base audio model we used ResNet pretrained in speaker recognition task. Predictions from both modalities were fused on decision level and improve single-channel approaches by a few percent

Anti-phishing aims to detect phishing content/documents in a pool of textual data. This is an important problem in cybersecurity that can help to guard users from fraudulent information. Natural language processing (NLP) offers a natural solution for this problem as it is capable of analyzing the textual content to perform intelligent recognition. In this work, we investigate state-of-the-art techniques for text categorization in NLP to address the problem of anti-phishing for emails (i.e, predicting if an email is phishing or not). These techniques are based on deep learning models that have attracted much attention from the community recently. In particular, we present a framework with hierarchical long short-term memory networks (H-LSTMs) and attention mechanisms to model the emails simultaneously at the word and the sentence level. Our expectation is to produce an effective model for anti-phishing and demonstrate the effectiveness of deep learning for problems in cybersecurity.

This paper introduces Random Multimodel Deep Learning (RMDL): a new ensemble, deep learning approach for classification. Deep learning models have achieved state-of-the-art results across many domains. RMDL solves the problem of finding the best deep learning structure and architecture while simultaneously improving robustness and accuracy through ensembles of deep learning architectures. RDML can accept as input a variety data to include text, video, images, and symbolic. This paper describes RMDL and shows test results for image and text data including MNIST, CIFAR-10, WOS, Reuters, IMDB, and 20newsgroup. These test results show that RDML produces consistently better performance than standard methods over a broad range of data types and classification problems.

Deep learning has transformed the computer vision, natural language processing and speech recognition. However, the following two critical questions are remaining obscure: (1) why deep neural networks generalize better than shallow networks? (2) Does it always hold that a deeper network leads to better performance? Specifically, letting $L$ be the number of convolutional and pooling layers in a deep neural network, and $n$ be the size of the training sample, we derive the upper bound on the expected generalization error for this network, i.e., \begin{eqnarray*} \mathbb{E}[R(W)-R_S(W)] \leq \exp{\left(-\frac{L}{2}\log{\frac{1}{\eta}}\right)}\sqrt{\frac{2\sigma^2}{n}I(S,W) } \end{eqnarray*} where $\sigma >0$ is a constant depending on the loss function, $0<\eta<1$ is a constant depending on the information loss for each convolutional or pooling layer, and $I(S, W)$ is the mutual information between the training sample $S$ and the output hypothesis $W$. This upper bound discovers: (1) As the network increases its number of convolutional and pooling layers $L$, the expected generalization error will decrease exponentially to zero. Layers with strict information loss, such as the convolutional layers, reduce the generalization error for the whole network. This answers the first question. However, (2) algorithms with zero expected generalization error does not imply a small test error or $\mathbb{E}[R(W)]$. This is because $\mathbb{E}[R_S(W)]$ will be large when the information for fitting the data is lost as the number of layers increases. This suggests that the claim `the deeper the better' is conditioned on a small training error or $\mathbb{E}[R_S(W)]$. (3) We further show that deep learning algorithms satisfy a weak notion of stability and the sample complexity of deep learning algorithms will decrease as $L$ increases.

Drag in your training data and Lobe automatically builds you a custom deep learning model. Then refine your model by adjusting settings and connecting pre-trained building blocks. Monitor training progress in real-time with interactive charts and test results that update live as your model improves. Cloud training lets you get results quickly, without slowing down your computer. Export your trained model to TensorFlow or CoreML and run it directly in your app on iOS and Android.

Twenty startups were funded in April 2018. Fifteen disclosed transaction amounts totaling $808 million of which the $600 million to SenseTime, the Alibaba-funded Chinese deep learning and facial recognition software provider focused on smart self-driving vehicle systems, was by far the largest. Seven acquisitions also occurred in April. The most notable was the acquisition by Teradyne (which previously acquired Universal Robots and Energid) of MiR (Mobile Industrial Robots) for $148 million with an additional $124 million predicated on very achievable milestones between now and 2020. SenseTime, a Chinese deep learning and facial recognition software provider focused on smart self-driving vehicle systems, raised $600 million in a Series C funding round led by Alibaba Group with participation by Temasek Holdings and Suning Commerce Group.

Summary: Our starting assumption that sequence problems (language, speech, and others) are the natural domain of RNNs is being challenged. Temporal Convolutional Nets (TCNs) which are our workhorse CNNs with a few new features are outperforming RNNs on major applications today. Looks like RNNs may well be history. It's only been since 2014 or 2015 when our DNN-powered applications passed the 95% accuracy point on text and speech recognition allowing for whole generations of chatbots, personal assistants, and instant translators. Convolutional Neural Nets (CNNs) are the acknowledged workhorse of image and video recognition while Recurrent Neural Nets (RNNs) became the same for all things language. One of the key differences is that CNNs can recognize features in static images (or video when considered one frame at a time) while RNNs exceled at text and speech which were recognized as sequence or time-dependent problems.

Go is the go-to game for machine learning researchers. It's what Google's DeepMind team famously used to show off its algorithms, and Facebook, too, recently announced that it was building a Go bot of its own. As the team announced at the company's F8 developer conference today, the ELF OpenGo bot has now achieved professional status after winning all 14 games it played against a group of top 30 human Go players recently. "We salute our friends at DeepMind for doing awesome work," Facebook CTO Mike Schroepfer said in today's keynote. "But we wondered: Are there some unanswered questions? What else can you apply these tools to."