Deep Learning
DeepProteomics: Protein family classification using Shallow and Deep Networks
Vazhayil, Anu, R, Vinayakumar, KP, Soman
The knowledge regarding the function of proteins is necessary as it gives a clear picture of biological processes. Nevertheless, there are many protein sequences found and added to the databases but lacks functional annotation. The laboratory experiments take a considerable amount of time for annotation of the sequences. This arises the need to use computational techniques to classify proteins based on their functions. In our work, we have collected the data from Swiss-Prot containing 40433 proteins which is grouped into 30 families. We pass it to recurrent neural network(RNN), long short term memory(LSTM) and gated recurrent unit(GRU) model and compare it by applying trigram with deep neural network and shallow neural network on the same dataset. Through this approach, we could achieve maximum of around 78% accuracy for the classification of protein families.
Response Characterization for Auditing Cell Dynamics in Long Short-term Memory Networks
Hasani, Ramin M., Amini, Alexander, Lechner, Mathias, Naser, Felix, Grosu, Radu, Rus, Daniela
In this paper, we introduce a novel method to interpret recurrent neural networks (RNNs), particularly long short-term memory networks (LSTMs) at the cellular level. We propose a systematic pipeline for interpreting individual hidden state dynamics within the network using response characterization methods. The ranked contribution of individual cells to the network's output is computed by analyzing a set of interpretable metrics of their decoupled step and sinusoidal responses. As a result, our method is able to uniquely identify neurons with insightful dynamics, quantify relationships between dynamical properties and test accuracy through ablation analysis, and interpret the impact of network capacity on a network's dynamical distribution. Finally, we demonstrate generalizability and scalability of our method by evaluating a series of different benchmark sequential datasets.
Capsule Deep Neural Network for Recognition of Historical Graffiti Handwriting
Gordienko, Nikita, Kochura, Yuriy, Taran, Vlad, Peng, Gang, Gordienko, Yuri, Stirenko, Sergii
Automatic recognition of the historical letters (XI-XVIII centuries) carved on the stoned walls of St.Sophia cathedral in Kyiv (Ukraine) was demonstrated by means of capsule deep learning neural network. It was applied to the image dataset of the carved Glagolitic and Cyrillic letters (CGCL), which was assembled and pre-processed recently for recognition and prediction by machine learning methods (https://www.kaggle.com/yoctoman/graffiti-st-sophia-cathedral-kyiv). CGCL dataset contains >4000 images for glyphs of 34 letters which are hardly recognized by experts even in contrast to notMNIST dataset with the better images of 10 letters taken from different fonts. Despite the much worse quality of CGCL dataset and extremely low number of samples (in comparison to notMNIST dataset) the capsule network model demonstrated much better results than the previously used convolutional neural network (CNN). The validation accuracy (and validation loss) was higher (lower) for capsule network model than for CNN without data augmentation even. The area under curve (AUC) values for receiver operating characteristic (ROC) were also higher for the capsule network model than for CNN model: 0.88-0.93 (capsule network) and 0.50 (CNN) without data augmentation, 0.91-0.95 (capsule network) and 0.51 (CNN) with lossless data augmentation, and similar results of 0.91-0.93 (capsule network) and 0.9 (CNN) in the regime of lossless data augmentation only. The confusion matrixes were much better for capsule network than for CNN model and gave the much lower type I (false positive) and type II (false negative) values in all three regimes of data augmentation. These results supports the previous claims that capsule-like networks allow to reduce error rates not only on MNIST digit dataset, but on the other notMNIST letter dataset and the more complex CGCL handwriting graffiti letter dataset also.
TGE-PS: Text-driven Graph Embedding with Pairs Sampling
Chen, Liheng, Qu, Yanru, Wang, Zhenghui, Qiu, Lin, Zhang, Weinan, Chen, Ken, Zhang, Shaodian, Yu, Yong
In graphs with rich text information, constructing expressive graph representations requires incorporating textual information with structural information. Graph embedding models are becoming more and more popular in representing graphs, yet they are faced with two issues: sampling efficiency and text utilization. Through analyzing existing models, we find their training objectives are composed of pairwise proximities, and there are large amounts of redundant node pairs in Random Walk-based methods. Besides, inferring graph structures directly from texts (also known as zero-shot scenario) is a problem that requires higher text utilization. To solve these problems, we propose a novel Text-driven Graph Embedding with Pairs Sampling (TGE-PS) framework. TGE-PS uses Pairs Sampling (PS) to generate training samples which reduces ~99% training samples and is competitive compared to Random Walk. TGE-PS uses Text-driven Graph Embedding (TGE) which adopts word- and character-level embeddings to generate node embeddings. We evaluate TGE-PS on several real-world datasets, and experimental results demonstrate that TGE-PS produces state-of-the-art results in traditional and zero-shot link prediction tasks.
Democratizing Deep Learning – The Stanford Dawn Project
Summary: How about we develop a ML platform that any domain expert can use to build a deep learning model without help from specialist data scientists, in a fraction of the time and cost. The good news is the folks at the Stanford DAWN project are hard at work on just such a platform and the initial results are extraordinary. Last week we wrote that sufficient labeled training data was the single greatest cost factor holding back wide adoption of machine learning. It may be the most expensive, but it's far from the only thing holding ML back. After examining the work of the Stanford DAWN project these researchers propose that literally all the steps in the development process from data acquisition, to feature extraction, to model training, and all the way to productionizing the model are all deeply flawed.
Top 5 Best Artificial Intelligence(AI) Frameworks and Libraries for AI
The existence of Artificial intelligence has been for a long time in the industry. The impact of AI has never been limited moreover, it has become an important element in recent years. The huge improvements in various fields, especially in mobile App development, is due to the development of various libraries and framework. Thus we can say that AI has become a responsive IT field and has lots of research going into it. There are many AI solutions that came into existence due to the implementation and integration of libraries and framework. TensorFlow is a neural network library created by Google.
Cloud Tools Rev Up AI Dev Platforms
The latest release of analytics startup Dataiku's platform incorporates application container and orchestration capabilities along with expanded support for deep learning libraries to help launch enterprise AI project at scale. The data platform vendor said Monday (Sept. Dataiku 5 also includes new documentation features aimed at improving application development among distributed data science teams. The New York-based data science startup joins a growing list of analytics vendors embracing cloud-native tools like Docker containers and the Kubernetes cluster orchestrator. Dataiku's spin on containerization includes combining the infrastructure tools with the in-memory processing of Python and R programing languages as well as in-memory model training and scoring.
Cisco shoves more GPUs in AI server for deep learning, still doesn't play Crysis
Cisco has beefed up its C480 AI/machine learning sever, adding a faster GPU interconnect and more GPU slots while losing two CPU sockets. It was revealed in June 2017. Since then NetApp and Pure have introduced all-flash storage systems with more GPUs inside NVidia's DGX-1 GPU server system, interconnected by NVlink. Pure's can have four DGX-1s. Cisco has announced an updated C480 ML M5 – "ML" of course standing for machine learning – with 2 x Xeon SP CPUs, up to 3TB of memory, 8 x Nvidia Tesla V100-32G, up to 24 x SAS/SATA HDD/SSD (to 182TB), plus 6 x NVMe drives, and up to 4 x x100G VIC (Virtual Interface Card), and NVlink.
RJT1990/mantra
Mantra is a deep learning development kit that manages the various components in an deep learning project, and makes it much easier to do routine tasks like training in the cloud, model monitoring, model benchmarking and more. It works with your favourite deep learning libraries like TensorFlow, PyTorch and Keras. This is an very early alpha release: we'd love your comments on the general concept and whether you find it useful. Issues and pull requests are also welcome. If the project gets enough love, we'll work towards a stable release!