With Artificial Intelligence By Example, develop your own method for future AI solutions. Acquire advanced AI, machine learning, and deep learning design skills. Description Topics included: Become an Adaptive Thinker • Think like a Machine • Apply Machine Thinking to a Human Problem • Become an Unconventional Innovator • Manage the Power of Machine Learning and Deep Learning • Don't Get Lost in Techniques – Focus on Optimizing • Your Solutions • When and How to Use Artificial Intelligence • Revolutions Designed for Some Corporations and Disruptive • Innovations for Small to Large Companies • Getting Your Neurons to Work • Applying Biomimicking to Artificial Intelligence • Conceptual Representation Learning • Automated Planning and Scheduling • AI and the Internet of Things (IoT) • Optimizing Blockchains with AI • Cognitive NLP Chatbots • Improve the Emotional Intelligence Deficiencies of Chatbots • Quantum Computers That Think

Computer programs are composed of three fundamental mechanisms: elementary operations, logical flow control and memory usage. In the history of neural networks [19] only the use of elementary operations have been extensively explored since so far, but during the last few years the coupling with an external piece of memory is gaining popularity [24]. Neural Turing Machines (NTMs) were developed in 2014 at Google DeepMind Labs [8] in an attempt to couple a neural network with an external memory component in order to improve long-term dependency learning in sequences prediction. Although recurrent neural networks (RNNs) are Turing-complete on their own [20], the difficulties that arise during their training (like the vanishing or the exploding gradient problems [18, 15]) prevented them from being employed in learning more complex tasks, for example algorithmic ones [27]. NTMs derive their name from the analogy with standard Turing Machines (TMs) [22] in addressing an infinite (or at least large enough to be considered so) portion of memory with an attentional mechanism similar to the read/write head of a TM. In contrast to a standard TM, a NTM is a "differentiable computer" that can be trained using gradient descent methods and can therefore learn its own "program" independently (attempts using Neuroevolution [9] and reinforcement learning [26] have also been made). In human brains, the most similar process to an algorithm is the concept of "working memory" [1]: this mechanism allows the brain to rapidly create "variables" [11] by storing short-term information and manipulating them in a rulebased way [17]. The analogy with an algorithm is evident, and a NTM is similar to this process because it can learn tasks in which it is required to manipulate rapidly-created variables. Also the attention mechanism in a NTM is similar to the way the working memory bounds its information in certain slots of memory in the brain [6], despite the fact that a NTM autonomously learns how to do that.

Most deep neural networks use simple, fixed activation functions, such as sigmoids or rectified linear units, regardless of domain or network structure. We introduce differential equation units (DEUs), an improvement to modern neural networks, which enables each neuron to learn a particular nonlinear activation function from a family of solutions to an ordinary differential equation. Specifically, each neuron may change its functional form during training based on the behavior of the other parts of the network. We show that using neurons with DEU activation functions results in a more compact network capable of achieving comparable, if not superior, performance when is compared to much larger networks.

There is a small number of case studies of automatic land cover classification on the coastal area. Here, I test extraction of seagrass beds, sandy area, oyster farming rafts at Mangoku-ura Lagoon, Miyagi, Japan by comparing manual tracing, simple image segmentation, and image transformation using deep learning. The result was used to extract the changes before and after the earthquake and tsunami. The output resolution was best in the image transformation method, which showed more than 69% accuracy for vegetation classification by an assessment using random points on independent test data. The distribution of oyster farming rafts was detected by the segmentation model. Assessment of the change before and after the earthquake by the manual tracing and image transformation result revealed increase of sand area and decrease of the vegetation. By the segmentation model only the decrease of the oyster farming was detected. These results demonstrate the potential to extract the spatial pattern of these elements after an earthquake and tsunami. Index Terms: Great East Japan Earthquake of 2011, Land use land cover (LULC), Zosteracea seagrass, cultured oyster, deep learning, Mangoku Bay

In this paper we describe a video surveillance system able to detect traffic events in videos acquired by fixed videocameras on highways. The events of interest consist in a specific sequence of situations that occur in the video, as for instance a vehicle stopping on the emergency lane. Hence, the detection of these events requires to analyze a temporal sequence in the video stream. We compare different approaches that exploit architectures based on Recurrent Neural Networks (RNNs) and Convolutional Neural Networks (CNNs). A first approach extracts vectors of features, mostly related to motion, from each video frame and exploits a RNN fed with the resulting sequence of vectors. The other approaches are based directly on the sequence of frames, that are eventually enriched with pixel-wise motion information. The obtained stream is processed by an architecture that stacks a CNN and a RNN, and we also investigate a transfer-learning-based model. The results are very promising and the best architecture will be tested online in real operative conditions.

Relation extraction models suffer from limited qualified training data. Using human annotators to label sentences is too expensive and does not scale well especially when dealing with large datasets. In this paper, we use Auxiliary Classifier Generative Adversarial Networks (AC-GANs) to generate high-quality relational sentences and to improve the performance of relation classifier in end-to-end models. In AC-GAN, the discriminator gives not only a probability distribution over the real source, but also a probability distribution over the relation labels. This helps to generate meaningful relational sentences.

Creating efficient deep neural networks involves repetitive manual optimization of the topology and the hyperparameters. This human intervention significantly inhibits the process. Recent publications propose various Neural Architecture Search (NAS) algorithms that automate this work. We have applied a customized NAS algorithm with network morphism and Bayesian optimization to the problem of cryptocurrency predictions, where it achieved results on par with our best manually designed models. This is consistent with the findings of other teams, while several known experiments suggest that given enough computing power, NAS algorithms can surpass state-of-the-art neural network models designed by humans. In this paper, we propose a blockchain network protocol that incentivises independent computing nodes to run NAS algorithms and compete in finding better neural network models for a particular task. If implemented, such network can be an autonomous and self-improving source of machine learning models, significantly boosting and democratizing the access to AI capabilities for many industries.

Recurrent neural networks (RNN) and their variants, such as Long-Short Term Memory (Hochreiter and Schmidhuber, 1997) and Gated Recurrent Unit (Cho et al., 2014), are commonly used as the default architecture or even the synonym of sequence modeling by deep learning practitioners (Goodfellow et al., 2016). In the meanwhile, especially for high-dimensional time series, we may also consider the autoregressive modeling or multi-task learning, null y t f (y t 1, y t 2,..., y t P), (1) where the output null y t and each input y t i are N -dimensional, and the lag P can be very large for accomodating sequential dependence. Some non-recurrent feed-forward networks with convolutional or other certain architectures have been proposed recently for sequence modeling, and are shown to have state-of-the-art accuracy. For example, some autoregressive networks, such as PixelCNN (Van den Oord et al., 2016b) and WaveNet (Van den Oord et al., 2016a) for image and audio sequence modeling, are compelling alternatives to the recurrent networks. This paper aims at the autoregressive model (1) with a large number of sequences. This problem can be implemented by a fully connected network with NP inputs and N outputs.

Numerous empirical evidence has corroborated that the noise plays a crucial rule in effective and efficient training of neural networks. The theory behind, however, is still largely unknown. This paper studies this fundamental problem through training a simple two-layer convolutional neural network model. Although training such a network requires solving a nonconvex optimization problem with a spurious local optimum and a global optimum, we prove that perturbed gradient descent and perturbed mini-batch stochastic gradient algorithms in conjunction with noise annealing is guaranteed to converge to a global optimum in polynomial time with arbitrary initialization. This implies that the noise enables the algorithm to efficiently escape from the spurious local optimum. Numerical experiments are provided to support our theory.

Capturing sentence semantics plays a vital role in a range of text mining applications. Despite continuous efforts on the development of related datasets and models in the general domain, both datasets and models are limited in biomedical and clinical domains. The BioCreative/OHNLP organizers have made the first attempt to annotate 1,068 sentence pairs from clinical notes and have called for a community effort to tackle the Semantic Textual Similarity (BioCreative/OHNLP STS) challenge. We developed models using traditional machine learning and deep learning approaches. For the post challenge, we focus on two models: the Random Forest and the Encoder Network. We applied sentence embeddings pre-trained on PubMed abstracts and MIMIC-III clinical notes and updated the Random Forest and the Encoder Network accordingly. The official results demonstrated our best submission was the ensemble of eight models. It achieved a Person correlation coefficient of 0.8328, the highest performance among 13 submissions from 4 teams. For the post challenge, the performance of both Random Forest and the Encoder Network was improved; in particular, the correlation of the Encoder Network was improved by ~13%. During the challenge task, no end-to-end deep learning models had better performance than machine learning models that take manually-crafted features. In contrast, with the sentence embeddings pre-trained on biomedical corpora, the Encoder Network now achieves a correlation of ~0.84, which is higher than the original best model. The ensembled model taking the improved versions of the Random Forest and Encoder Network as inputs further increased performance to 0.8528. Deep learning models with sentence embeddings pre-trained on biomedical corpora achieve the highest performance on the test set.