Goto

Collaborating Authors

 Deep Learning


Transfer Learning for Electricity Price Forecasting

arXiv.org Machine Learning

The task has been studied in different markets separately and learning interdependent information in between different markets is an understudied field. Recently, deep learning methods have showcased superior performance in predicting electricity prices [1]. In particular, recurrent neural networks have been able to learn sequential information in time-series type data sets [2]. Most of the the literature on the application of neural networks for electricity price forecasting has relied on single market data and available large amounts of data from different markets have not been utilized. Transfer Learning is a major tool to improve the performance on image classification problems. The networks can be trained on similar problems before finally being trained on the final problem to leverage from the data to the fullest. In this paper, we utilize the concept of transfer learning for electricity price forecasting by using data from five different markets. Our major novelties can be listed as: 1. We investigate the different ways to combine data from different elec-2 tricity markets, when training neural networks, 2. We propose the transfer learning scheme to leverage from different market data, when training recurrent neural networks (RNN) for the task of price prediction.


Hierarchical nucleation in deep neural networks

arXiv.org Machine Learning

Deep convolutional networks (DCNs) learn meaningful representations where data that share the same abstract characteristics are positioned closer and closer. Understanding these representations and how they are generated is of unquestioned practical and theoretical interest. In this work we study the evolution of the probability density of the ImageNet dataset across the hidden layers in some state-of-the-art DCNs. We find that the initial layers generate a unimodal probability density getting rid of any structure irrelevant for classification. In subsequent layers density peaks arise in a hierarchical fashion that mirrors the semantic hierarchy of the concepts. Density peaks corresponding to single categories appear only close to the output and via a very sharp transition which resembles the nucleation process of a heterogeneous liquid. This process leaves a footprint in the probability density of the output layer where the topography of the peaks allows reconstructing the semantic relationships of the categories.


AI Startup Lets Foresters See the Wood for the Trees

#artificialintelligence

AI startup Trefos is helping foresters see the wood for the trees. Using custom lidar and camera-mounted drones, the Philadelphia-based company collects data for high-resolution, 3D forest maps. These metrics allow government agencies and the forestry industry to estimate the volume of timber and biomass in an area of forest, as well as the amount of carbon stored in the trees. With this unprecedented detail, foresters can make more informed decisions when, for example, evaluating the need for controlled burns to clear biomass and reduce the risk of wildfires. "Forests are often very dense, with a very repetitive layout," said Steven Chen, founder and CEO of the startup, a member of the NVIDIA Inception program, which supports startups from product development to deployment. "We can use deep learning algorithms to detect trees, isolate them from the surrounding branches and vines, and use those as landmarks."


How AI Helps Historians Solve Ancient Puzzles

#artificialintelligence

The adoption of artificial intelligence and machine learning techniques is making research on historical artifacts and handwritten records faster and more efficient. Artificial intelligence (AI) and machine learning techniques are helping historians restore or recreate archaeological artifacts from photos of fragments. Ayellet Tal of Israel's Technion University said applying algorithmic techniques to historical research can improve AI's capabilities. AI models can be taught how to reverse the erosion process, predict what the original fragments looked like, and test whether fragments fit together. Said Tal, "We want to transform archaeology and we want to advance computer vision because these tasks are where current algorithms fail."


Converting text news into video stories using Deep Learning

#artificialintelligence

What these companies have in common is that they both use cutting-edge deep learning technologies such as generative adversarial networks, facial point detection, body pose estimation, and cloud technologies to synthesize human faces and videos that cannot be distinguished from reality. Today, I want to show and deconstruct a new product in the synthetic media category that we have been developing for the last few months called NIUS.TV. Catching up on news on mobile is still painful. Reading while commuting, exercising, or waiting is difficult. NIUS.TV is a next-generation mobile-first news aggregator that converts text news on topics you love into video stories narrated by an AI anchor.


Free MIT Courses on Calculus: The Key to Understanding Deep Learning - KDnuggets

#artificialintelligence

It is difficult, perhaps, to link this to neural networks, but the basic intuition of calculus is achieved. If you are looking for a more full treatment of this branch of mathematics, you will want to seek out some more robust learning tools. Here are 3 courses and a textbook to help out, all from MIT's Open Courseware initiative, which will cover everything you need to know about calculus to understand deep learning -- and far beyond.


Explainer: What Is Machine Learning?

#artificialintelligence

Machine learning (ML) has become a hot topic in the last few years, but what you may not realize is that the concept of machine learning has been around for decades. The design of machine-learning systems used to this day is based on the human brain model described by Donald Hebb in 1949 in his book "The Organization of Behavior." Hebb noted that when cells in the brain fire in a repeated pattern, synaptic knobs are formed or enlarge if they already exist. The same principle is applied to nodes in a digital neural network. Nodes develop relationships that grow stronger if they are activated simultaneously and weaken if they fire separately.


Reverb: a framework for experience replay

#artificialintelligence

The use of experience plays a key role in reinforcement learning (RL). How best to use this data is one of the central problems of this field. As RL agents have advanced over recent years, taking on bigger and more complex problems (Atari, Go, StarCraft, Dota), the generated data has grown in both size and complexity. To cope with this complexity many RL systems split the learning problem into two distinct parts: experience producers (actors) and experience consumers (learners) — allowing these different parts to run in parallel. Often a data storage system lies at the intersection between these two components. The question of how to efficiently store and transport the data is itself a challenging engineering problem.


Model Compression: needs and importance

#artificialintelligence

Whether you're new to computer vision or an expert, you've probably heard about AlexNet winning the ImageNet challenge in 2012. That was the turning point in computer vision history because it showed that deep learning models can perform tasks which were considered very difficult for computers, with an unprecedented level of accuracy. But did you know that AlexNet had 62 million trainable parameters? Another popular model VGGNet which came out in 2014 had even more, 138 million trainable parameters. You might be thinking… I know that the deeper the model is, the better it will perform.


DNNBrain: a unifying toolbox for mapping deep neural networks and brains

#artificialintelligence

Deep neural networks (DNNs) have attained human-level performance on dozens of challenging tasks through an end-to-end deep learning strategy. Deep learning gives rise to data representations with multiple levels of abstraction; however, it does not explicitly provide any insights into the internal operations of DNNs. Its success appeals to neuroscientists not only to apply DNNs to model biological neural systems, but also to adopt concepts and methods from cognitive neuroscience to understand the internal representations of DNNs. Although general deep learning frameworks such as PyTorch and TensorFlow could be used to allow such cross-disciplinary studies, the use of these frameworks typically requires high-level programming expertise and comprehensive mathematical knowledge. A toolbox specifically designed for cognitive neuroscientists to map DNNs and brains is urgently needed.