Deep Learning
Predictive-State Decoders: Encoding the Future into Recurrent Networks
Venkatraman, Arun, Rhinehart, Nicholas, Sun, Wen, Pinto, Lerrel, Hebert, Martial, Boots, Byron, Kitani, Kris M., Bagnell, J. Andrew
Recurrent neural networks (RNNs) are a vital modeling technique that rely on internal states learned indirectly by optimization of a supervised, unsupervised, or reinforcement training loss. RNNs are used to model dynamic processes that are characterized by underlying latent states whose form is often unknown, precluding its analytic representation inside an RNN. In the Predictive-State Representation (PSR) literature, latent state processes are modeled by an internal state representation that directly models the distribution of future observations, and most recent work in this area has relied on explicitly representing and targeting sufficient statistics of this probability distribution. We seek to combine the advantages of RNNs and PSRs by augmenting existing state-of-the-art recurrent neural networks with Predictive-State Decoders (PSDs), which add supervision to the network's internal state representation to target predicting future observations. Predictive-State Decoders are simple to implement and easily incorporated into existing training pipelines via additional loss regularization. We demonstrate the effectiveness of PSDs with experimental results in three different domains: probabilistic filtering, Imitation Learning, and Reinforcement Learning. In each, our method improves statistical performance of state-of-the-art recurrent baselines and does so with fewer iterations and less data.
House Price Prediction Using LSTM
Chen, Xiaochen, Wei, Lai, Xu, Jiaxin
In this paper, we use the house price data ranging from January 2004 to October 2016 to predict the average house price of November and December in 2016 for each district in Beijing, Shanghai, Guangzhou and Shenzhen. We apply Autoregressive Integrated Moving Average model to generate the baseline while LSTM networks to build prediction model. These algorithms are compared in terms of Mean Squared Error. The result shows that the LSTM model has excellent properties with respect to predict time series. Also, stateful LSTM networks and stack LSTM networks are employed to further study the improvement of accuracy of the house prediction model.
How Facebook Is Using Artificial Intelligence
The interaction between machines and people is expected to become more lucid and personalized as Artificial Intelligence tools become more advanced and learn to adapt to the dynamic environment around us. This makes AI one of the most compelling advanced technologies for company like Facebook, Inc. (FB), whose primary mode of exchange with people is through technology. Facebook has shown a steady commitment to integrating AI across its services to enhance and enable superior customer engagements. Here's a look at how Facebook is working with AI. With a vision that "artificial intelligence can play a big role in helping bring the world closer together," Facebook has opened a new AI research lab in Montreal as part of Facebook AI Research (FAIR).
Practical Deep Learning with PyTorch - Udemy
Although many courses are very mathematical or too practical in nature, this course strikes a careful balance between the two to provide a solid foundation in deep learning for you to explore further if you are interested in research in the field of deep learning and/or applied deep learning. It is purposefully made for anyone without a strong background in mathematics. And for those with a strong background, it would accelerate your learning in understanding the different models in deep learning. This entire course is delivered in a Python Notebook such that you can follow along the videos and replicate the results. You can practice and tweak the models until you truly understand every line of code as we go along.
Machine Learning at HPC User Forum: Drilling into Specific Use Cases
The 66th HPC User Forum held September 5-7, in Milwaukee, Wisconsin, at the elegant and historic Pfister Hotel, highlighting the 1893 Victorian dรฉcor and art of "The Grand Hotel Of The West," contrasted nicely with presentations on the latest trends in modern computing โ deep learning, machine learning and AI. Over the course of two days of presentations, a couple common themes became obvious: First, that machine and deep learning are focused currently on specific rather than general use cases and second, that ML and DL need to be part of an integrated workflow to be effective. This was exemplified by Dr. Maarten Sierhuis from Nissan Research Facility Silicon Valley with his presentation "Technologies for Making Self-Driving Vehicles the Norm." The challenge that Nissan and other deep learning practitioners face is that current deep learning algorithms are programmed to learn to do one thing extremely well โ the specific use case: image recognition of stop signs for example. Once an algorithm learns to recognize stop signs, the same amount of discrete learning must apply for every other road sign a vehicle may encounter.
Deep Learning Research Review: Natural Language Processing
The traditional approach to NLP involved a lot of domain knowledge of linguistics itself. Understanding terms such as phonemes and morphemes were pretty standard as there are whole linguistic classes dedicated to their study. Let's look at how traditional NLP would try to understand the following word.
Adaptive Convolutional Filter Generation for Natural Language Understanding
Shen, Dinghan, Min, Martin Renqiang, Li, Yitong, Carin, Lawrence
Convolutional neural networks (CNNs) have recently emerged as a popular building block for natural language processing (NLP). Despite their success, most existing CNN models employed in NLP are not expressive enough, in the sense that all input sentences share the same learned (and static) set of filters. Motivated by this problem, we propose an adaptive convolutional filter generation framework for natural language understanding, by leveraging a meta network to generate input-aware filters. We further generalize our framework to model question-answer sentence pairs and propose an adaptive question answering (AdaQA) model; a novel two-way feature abstraction mechanism is introduced to encapsulate co-dependent sentence representations. We investigate the effectiveness of our framework on document categorization and answer sentence-selection tasks, achieving state-of-the-art performance on several benchmark datasets.
Deep learning must happen at the edge, too - SiliconANGLE
In between meeting with customers, crowdchatting with our communities and hosting theCUBE, the research team at Wikibon, owned by the same company as SiliconANGLE, finds time to meet and discuss trends and topics regarding digital business transformation and technology markets. We look at topics from the standpoints of business, the Internet of Things, big data, application, cloud and infrastructure modernization. We use the results of our research meetings to explore new research topics, further current research projects and share insights. This is the sixth summary of findings from these regular meetings, which we plan to publish every week. The combination of faster, cheaper and memory-rich hardware, coupled with unprecedented streams of data, has renewed interest in an old favorite: artificial intelligence. But this time AI and its progeny, "machine learning" and "deep learning," are generating real returns in a wide array of industries and applications.
Clever Machines Learn How to Be Curious (And Play Super Mario Bros.)
You probably can't remember what it feels like to play Super Mario Bros. for the very first time, but try to picture it. An 8-bit game world blinks into being: baby blue sky, tessellated stone ground, and in between, a squat, red-suited man standing still--waiting. He's facing rightward; you nudge him farther in that direction. A few more steps reveal a row of bricks hovering overhead and what looks like an angry, ambulatory mushroom. Another twitch of the game controls makes the man spring up, his four-pixel fist pointed skyward. Maybe try combining nudge-rightward and spring-skyward?