Deep Learning
Lessons Learned from Applying Deep Learning for NLP Without Big Data
As a data scientist, one of your most important skills should be choosing the right modeling techniques and algorithms for your problems. A few months ago I was trying to solve a text classification problem of classifying which news articles will be relevant for my customers. I had only a few thousand labeled examples so I started with simple classic machine learning modeling methods like Logistic regression on TF-IDF. These models that usually works well on text classification of long documents like news articles performed only slightly better than random on this task. After investigating the mistakes of my model I found that bag of words representation is just not enough for this task and I need a model the will use a deeper semantic understanding of the documents.
Stock Market Prediction by Recurrent Neural Network on LSTM Model
There are a lot of complicated financial indicators and also the fluctuation of the stock market is highly violent. However, as the technology is getting advanced, the opportunity to gain a steady fortune from the stock market is increased and it also helps experts to find out the most informative indicators to make a better prediction. The prediction of the market value is of great importance to help in maximizing the profit of stock option purchase while keeping the risk low. Recurrent neural networks (RNN) have proved one of the most powerful models for processing sequential data. Long Short-Term memory is one of the most successful RNNs architectures.
Getting Started With Google Colab โ Towards Data Science
Just let me code, already! You know it's out there. You know there's free GPU somewhere, hanging like a fat, juicy, ripe blackberry on a branch just slightly out of reach. Wondering how on earth to get it to work? For anyone who doesn't already know, Google has done the coolest thing ever by providing a free cloud service based on Jupyter Notebooks that supports free GPU.
Top Machine Learning Solutions
In today's hyper-fast cloud computing era, machine learning solutions drive exponential progress in improving systems. Machine learning's ability to leverage Big Data analytics and identify patterns offers critical competitive advantage to today's businesses. Often used in combination with artificial intelligence and deep learning, machine learning uses sophisticated statistical modeling. These complex systems may reside in private cloud or public cloud. In any case, the passage of time boosts machine learning: as more data is added to a task and analyzed over time, ML produces more accurate the results.
Elon Musk Confirms Exit From Openai Over Disagreements to Focus on Tesla, Spacex
Billionaire Elon Musk confirmed that he exited OpenAI, an artificial intelligence research group, on "good terms" amid disagreements with team members over the project's direction. Musk also cited a desire to focus on "solving a painfully large number of engineering and manufacturing problems at Tesla (especially) and SpaceX." On Twitter, Musk suggested he had encountered some conflicts of interest because Tesla was competing for some of the same people that OpenAI wanted to recruit. "I didn't agree with some of what OpenAI team wanted to do. Add that all up, and it was just better to part ways on good terms."
DIViS: Domain Invariant Visual Servoing for Collision-Free Goal Reaching
Robots should understand both semantics and physics to be functional in the real world. While robot platforms provide means for interacting with the physical world they cannot autonomously acquire object-level semantics without needing human. In this paper, we investigate how to minimize human effort and intervention to teach robots perform real world tasks that incorporate semantics. We study this question in the context of visual servoing of mobile robots and propose DIViS, a Domain Invariant policy learning approach for collision free Visual Servoing. DIViS incorporates high level semantics from previously collected static human-labeled datasets and learns collision free servoing entirely in simulation and without any real robot data. However, DIViS can directly be deployed on a real robot and is capable of servoing to the user-specified object categories while avoiding collisions in the real world. DIViS is not constrained to be queried by the final view of goal but rather is robust to servo to image goals taken from initial robot view with high occlusions without this impairing its ability to maintain a collision free path. We show the generalization capability of DIViS on real mobile robots in more than 90 real world test scenarios with various unseen object goals in unstructured environments. DIViS is compared to prior approaches via real world experiments and rigorous tests in simulation. For supplementary videos, see: \href{https://fsadeghi.github.io/DIViS}{https://fsadeghi.github.io/DIViS}
Discovery of Natural Language Concepts in Individual Units of CNNs
Na, Seil, Choe, Yo Joong, Lee, Dong-Hyun, Kim, Gunhee
Although deep convolutional networks have achieved improved performance in many natural language tasks, they have been treated as black boxes because they are difficult to interpret. Especially, little is known about how they represent language in their intermediate layers. In an attempt to understand the representations of deep convolutional networks trained on language tasks, we show that individual units are selectively responsive to specific morphemes, words, and phrases, rather than responding to arbitrary and uninterpretable patterns. In order to quantitatively analyze such an intriguing phenomenon, we propose a concept alignment method based on how units respond to the replicated text. We conduct analyses with different architectures on multiple datasets for classification and translation tasks and provide new insights into how deep models understand natural language.
Classifying textual data: shallow, deep and ensemble methods
Anderlucci, Laura, Guastadisegni, Lucia, Viroli, Cinzia
Nowadays the increasing and rapid progress of technology and the availability of electronic documents from a variety of sources have made a huge amount of textual data available. Hence, one of the prominent research topics of statistical andmachine learning communities is to provide suitable and feasible methods to extract high-quality information from unstructured textual data (Lata and Loar, 2018) for the different purposes of clustering, classification and document retrieval (Khan et al., 2010). This work originates from an empirical problem of classification of the content ofcalls made to the customer service of an important mobile phone company inItaly. The received calls are written down by an operator and classified into relevant classes (e.g.
On the Impact of the Activation Function on Deep Neural Networks Training
Hayou, Soufiane, Doucet, Arnaud, Rousseau, Judith
The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation. Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Samuel et al (2017) who showed that for deep feedforward neural networks only a specific choice of hyperparameters known as the `Edge of Chaos' can lead to good performance. While the work by Samuel et al (2017) discuss trainability issues, we focus here on training acceleration and overall performance. We give a comprehensive theoretical analysis of the Edge of Chaos and show that we can indeed tune the initialization parameters and the activation function in order to accelerate the training and improve the performance.
Deep Learning Based Autoencoder for Interference Channel
Wu, Dehao, Nekovee, Maziar, Wang, Yue
Deep learning (DL) based autoencoder has shown great potential to significantly enhance the physical layer performance. In this paper, we present a DL based autoencoder for interference channel. Based on a characterization of a k-user Gaussian interference channel, where the interferences are classified as different levels from weak to very strong interferences based on a coupling parameter {\alpha}, a DL neural network (NN) based autoencoder is designed to train the data set and decode the received signals. The performance such a DL autoencoder for different interference scenarios are studied, with {\alpha} known or partially known, where we assume that {\alpha} is predictable but with a varying up to 10\% at the training stage. The results demonstrate that DL based approach has a significant capability to mitigate the effect induced by a poor signal-to-noise ratio (SNR) and a high interference-to-noise ratio (INR). However, the enhancement depends on the knowledge of {\alpha} as well as the interference levels. The proposed DL approach performs well with {\alpha} up to 10\% offset for weak interference level. For strong and very strong interference channel, the offset of {\alpha} needs to be constrained to less than 5\% and 2\%, respectively, to maintain similar performance as {\alpha} is known.