Intel is the world's biggest maker of processors for computers, but it hasn't been the fastest when it comes to capitalizing on the artificial intelligence computing explosion. Rival Nvidia and other AI processor startups have jumped into the market, and Intel has been playing catchup. But the big company has been moving fast. It acquired AI chip design firm Nervana in 2016 for $350 million, and Intel recently announced that its Xeon CPUs generated $1 billion in revenue in 2017 for use in AI applications. Intel believes that the overall market for AI chips will reach between $8 billion and $10 billion in revenue by 2022. And the company is focused on designing AI chips from the ground up, according to Gadi Singer, vice president and general manager of AI architecture at Intel. Singer, his boss Naveen Rao, and chip architect Jim Keller have been hitting the road lately to show that Intel isn't asleep at the switch when it comes to the hot AI chip design trends.

WHITE PAPER ver1.0 released on March 2018, pre-sale was completed successfully and handed over softcap on May and ICO main-sale is live. On July, our MVP(Minimum Value Product) will be updated on our website-Github link and completed 1st ICO. We introduced new technological concept; AI Mining and AI Deep Learning preparing 2nd ICO. We will launch'Sapiens' which provides resources and verify the resources on Ethereum platform on December. On June 2019, we will start our own platform'Creatura' that provides advanced machine learning and form AI Society.

In this paper we present the results of an investigation of the importance of verbs in a deep learning QA system trained on SQuAD dataset. We show that main verbs in questions carry little influence on the decisions made by the system - in over 90% of researched cases swapping verbs for their antonyms did not change system decision. We track this phenomenon down to the insides of the net, analyzing the mechanism of self-attention and values contained in hidden layers of RNN. Finally, we recognize the characteristics of the SQuAD dataset as the source of the problem. Our work refers to the recently popular topic of adversarial examples in NLP, combined with investigating deep net structure.

Leveraging these insights, we propose a "heating-up" strategy to train a classifier To overcome the sampling issue, a variety of hard mining strategies (Schroff et al., 2015; Mishchuk In this paper, we show that the temperature parameter in the softmax function, defined by Hinton et al. (2015) for knowledge transfer, plays an important role in determining the distribution of the Compared to the state-of-the-art methods in deep metric learning, the proposed "heating-up" method Siamese networks with contrastive loss (Chopra et al., 2005) was one of the earliest attempts to solve A reasonable solution to address the sampling issue is mining samples that are the most informative for training, also known as "hard mining". Semihard mining (Schroff et al., 2015) tries to find triplets in a training batch, for which the distance of Lifted structured loss (Song et al., 2016) exploits all Proxy NCA (Movshovitz-Attias et al., 2017) proposes to learn semantic proxies for training data and Applying hard mining with proxies is more efficient than with samples. In face verification, quite a few works have shown that training a classifier and using the output of the second last layer as embedding performs reasonably well (Wang et al., 2017b). This paper shows that the scalar can be seen as the temperature parameter of the softmax function in Hinton et al. (2015). The proposed "heating-up" idea is based on an observation that different We define 2 types of training samples as in Figure 1.

Learning long-term dependencies in extended temporal sequences requires credit assignment to events far back in the past. The most common method for training recurrent neural networks, back-propagation through time (BPTT), requires credit information to be propagated backwards through every single step of the forward computation, potentially over thousands or millions of time steps. This becomes computationally expensive or even infeasible when used with long sequences. Importantly, biological brains are unlikely to perform such detailed reverse replay over very long sequences of internal states (consider days, months, or years.) However, humans are often reminded of past memories or mental states which are associated with the current mental state. We consider the hypothesis that such memory associations between past and present could be used for credit assignment through arbitrarily long sequences, propagating the credit assigned to the current state to the associated past state. Based on this principle, we study a novel algorithm which only back-propagates through a few of these temporal skip connections, realized by a learned attention mechanism that associates current states with relevant past states. We demonstrate in experiments that our method matches or outperforms regular BPTT and truncated BPTT in tasks involving particularly long-term dependencies, but without requiring the biologically implausible backward replay through the whole history of states. Additionally, we demonstrate that the proposed method transfers to longer sequences significantly better than LSTMs trained with BPTT and LSTMs trained with full self-attention.

We propose an effective method for creating interpretable control agents, by re-purposing the function of a biological neural circuit model, to govern simulated and real world reinforcement learning (RL) test-beds. Inspired by the structure of the nervous system of the soil-worm, C. elegans, we introduce Neuronal Circuit Policies (NCPs) as a novel recurrent neural network instance with liquid time-constants, universal approximation capabilities and interpretable dynamics. We theoretically show that they can approximate any finite simulation time of a given continuous n-dimensional dynamical system, with n output units and some hidden units. We model instances of the policies and learn their synaptic and neuronal parameters to control standard RL tasks and demonstrate its application for autonomous parking of a real rover robot on a predefined trajectory. For reconfiguration of the purpose of the neural circuit, we adopt a search-based RL algorithm. We show that our neuronal circuit policies perform as good as deep neural network policies with the advantage of realizing interpretable dynamics at the cell-level. We theoretically find bounds for the time-varying dynamics of the circuits, and introduce a novel way to reason about networks' dynamics.

The design of neural network architectures is an important component for achieving state-of-the-art performance with machine learning systems across a broad array of tasks. Much work has endeavored to design and build architectures automatically through clever construction of a search space paired with simple learning algorithms. Recent progress has demonstrated that such meta-learning methods may exceed scalable human-invented architectures on image classification tasks. An open question is the degree to which such methods may generalize to new domains. In this work we explore the construction of meta-learning techniques for dense image prediction focused on the tasks of scene parsing, person-part segmentation, and semantic image segmentation. Constructing viable search spaces in this domain is challenging because of the multi-scale representation of visual information and the necessity to operate on high resolution imagery. Based on a survey of techniques in dense image prediction, we construct a recursive search space and demonstrate that even with efficient random search, we can identify architectures that outperform human-invented architectures and achieve state-of-the-art performance on three dense prediction tasks including 82.7\% on Cityscapes (street scene parsing), 71.3\% on PASCAL-Person-Part (person-part segmentation), and 87.9\% on PASCAL VOC 2012 (semantic image segmentation). Additionally, the resulting architecture is more computationally efficient, requiring half the parameters and half the computational cost as previous state of the art systems.

-- A deep learning network was used to predict future blood glucose levels, as this can permit diabetes patients to take action before imminent hyperglycaemia and hypoglycaemia. A sequential model with one long-short-term memory (LSTM) layer, one bidirectional LSTM layer and several fully connected layers was used to predict blood glucose levels for different prediction horizons. The method was trained and tested on 26 datasets from 20 real patients. The proposed network outperforms the baseline methods in terms of all evaluation criteria. Diabetes can be mainly classified into Type 1 diabetes (T1D) and Type 2 diabetes (T2D), where the former is caused by destruction of the pancreatic beta cells resulting in insulin deficiency, while the latter due to the ineffective use of insulin.

Forecasting multivariate time series data, such as prediction of electricity consumption, solar power production, and polyphonic piano pieces, has numerous valuable applications. However, complex and non-linear interdependencies between time steps and series complicate the task. To obtain accurate prediction, it is crucial to model long-term dependency in time series data, which can be achieved to some good extent by recurrent neural network (RNN) with attention mechanism. Typical attention mechanism reviews the information at each previous time step and selects the relevant information to help generate the outputs, but it fails to capture the temporal patterns across multiple time steps. In this paper, we propose to use a set of filters to extract time-invariant temporal patterns, which is similar to transforming time series data into its "frequency domain". Then we proposed a novel attention mechanism to select relevant time series, and use its "frequency domain" information for forecasting. We applied the proposed model on several real-world tasks and achieved the state-of-the-art performance in all of them with only one exception. We also show that to some degree the learned filters play the role of bases in discrete Fourier transform.

The specificty and sensitivity of resting state functional MRI (rs-fMRI) measurements depend on pre-processing choices, such as the parcellation scheme used to define regions of interest (ROIs). In this study, we critically evaluate the effect of brain parcellations on machine learning models applied to rs-fMRI data. Our experiments reveal a remarkable trend: On average, models with stochastic parcellations consistently perform as well as models with widely used atlases at the same spatial scale. We thus propose an ensemble learning strategy to combine the predictions from models trained on connectivity data extracted using different (e.g., stochastic) parcellations. We further present an implementation of our ensemble learning strategy with a novel 3D Convolutional Neural Network (CNN) approach. The proposed CNN approach takes advantage of the full-resolution 3D spatial structure of rs-fMRI data and fits non-linear predictive models. Our ensemble CNN framework overcomes the limitations of traditional machine learning models for connectomes that often rely on region-based summary statistics and/or linear models. We showcase our approach on a classification (autism patients versus healthy controls) and a regression problem (prediction of subject's age), and report promising results.