Short-term load forecasting is a critical element of power systems energy management systems. In recent years, probabilistic load forecasting (PLF) has gained increased attention for its ability to provide uncertainty information that helps to improve the reliability and economics of system operation performances. This paper proposes a two-stage probabilistic load forecasting framework by integrating point forecast as a key probabilistic forecasting feature into PLF. In the first stage, all related features are utilized to train a point forecast model and also obtain the feature importance. In the second stage the forecasting model is trained, taking into consideration point forecast features, as well as selected feature subsets. During the testing period of the forecast model, the final probabilistic load forecast results are leveraged to obtain both point forecasting and probabilistic forecasting. Numerical results obtained from ISO New England demand data demonstrate the effectiveness of the proposed approach in the hour-ahead load forecasting, which uses the gradient boosting regression for the point forecasting and quantile regression neural networks for the probabilistic forecasting.

Takagi-Sugeno-Kang (TSK) fuzzy systems are very useful machine learning models for regression problems. However, to our knowledge, there has not existed an efficient and effective training algorithm that enables them to deal with big data. Inspired by the connections between TSK fuzzy systems and neural networks, we extend three powerful neural network optimization techniques, i.e., mini-batch gradient descent, regularization, and AdaBound, to TSK fuzzy systems, and also propose a novel DropRule technique specifically for training TSK fuzzy systems. Our final algorithm, mini-batch gradient descent with regularization, DropRule and AdaBound (MBGD-RDA), can achieve fast convergence in training TSK fuzzy systems, and also superior generalization performance in testing. It can be used for training TSK fuzzy systems on datasets of any size; however, it is particularly useful for big datasets, on which currently no other efficient training algorithms exist.

Short-term load forecasting (STLF) is essential for the reliable and economic operation of power systems. Though many STLF methods were proposed over the past decades, most of them focused on loads at high aggregation levels only. Thus, low-aggregation load forecast still requires further research and development. Compared with the substation or city level loads, individual loads are typically more volatile and much more challenging to forecast. To further address this issue, this paper first discusses the characteristics of small-and-medium enterprise (SME) and residential loads at different aggregation levels and quantifies their predictability with approximate entropy. Various STLF techniques, from the conventional linear regression to state-of-the-art deep learning, are implemented for a detailed comparative analysis to verify the forecasting performances as well as the predictability using an Irish smart meter dataset. In addition, the paper also investigates how using data processing improves individual-level residential load forecasting with low predictability. Effectiveness of the discussed method is validated with numerical results.

Amazon's decision to design its second-generation Amazon Echo smart speaker with a removable sleeve makes the device customizable. But there's a hidden feature I wasn't aware of until I installed Ninety7's Sky Tote Portable Battery Base: a set of electrical contacts tucked behind a rubber plug on the bottom of the speaker. These contacts enable the Echo 2 to draw power directly from the Sky Tote's large battery, which is charged in turn by the AC adapter that comes with the Echo 2. Operating your Echo 2 on battery power allows you to take the smart speaker with you into any room in your home or even into the yard if your Wi-Fi network is strong enough to reach there. That's a much less expensive alternative to putting an Echo in every room for smart home control, or investing in a multi-room audio system so you can listen to music everywhere. You can also take a battery-powered Echo 2 on picnics and camping trips if you create a mobile hotspot with your smartphone.

The neural network is a powerful computing framework that has been exploited by biological evolution and by humans for solving diverse problems. Although the computational capabilities of neural networks are determined by their structure, the current understanding of the relationships between a neural network's architecture and function is still primitive. Here we reveal that neural network's modular architecture plays a vital role in determining the neural dynamics and memory performance of the network of threshold neurons. In particular, we demonstrate that there exists an optimal modularity for memory performance, where a balance between local cohesion and global connectivity is established, allowing optimally modular networks to remember longer. Our results suggest that insights from dynamical analysis of neural networks and information spreading processes can be leveraged to better design neural networks and may shed light on the brain's modular organization.

Machine learning offers remarkable benefits for improving workplaces and working conditions amongst others in the recycling industry. Here e.g. hand-sorting of medium value scrap is labor intensive and requires experienced and skilled workers. On the one hand, they have to be highly concentrated for making proper readings and analyses of the material, but on the other hand, this work is monotonous. Therefore, a machine learning approach is proposed for a quick and reliable automated identification of alloys in the recycling industry, while the mere scrap handling is regarded to be left in the hands of the workers. To this end, a set of twelve tool and high-speed steels from the field were selected to be identified by their spectrum induced by electric arcs. For data acquisition, the optical emission spectrometer Thorlabs CCS 100 was used. Spectra have been post-processed to be fed into the supervised machine learning algorithm. The development of the machine learning software is conducted according to the steps of the VDI 2221 standard method. For programming Python 3 as well as the python-library sklearn were used. By systematic parameter variation, the appropriate machine learning algorithm was selected and validated. Subsequent validation steps showed that the automated identification process using a machine learning approach and the optical emission spectrometry is applicable, reaching a maximum F1 score of 96.9 %. This performance is as good as the performance of a highly trained worker using visual grinding spark identification. The tests were based on a self-generated set of 600 spectra per single alloy (7,200 spectra in total) which were produced using an industry workshop device.

Reinforcement learning algorithms can be used to optimally solve dynamic decision-making and control problems. With continuous-valued state and input variables, reinforcement learning algorithms must rely on function approximators to represent the value function and policy mappings. Commonly used numerical approximators, such as neural networks or basis function expansions, have two main drawbacks: they are black-box models offering no insight in the mappings learned, and they require significant trial and error tuning of their meta-parameters. In this paper, we propose a new approach to constructing smooth value functions by means of symbolic regression. We introduce three off-line methods for finding value functions based on a state transition model: symbolic value iteration, symbolic policy iteration, and a direct solution of the Bellman equation. The methods are illustrated on four nonlinear control problems: velocity control under friction, one-link and two-link pendulum swing-up, and magnetic manipulation. The results show that the value functions not only yield well-performing policies, but also are compact, human-readable and mathematically tractable. This makes them potentially suitable for further analysis of the closed-loop system. A comparison with alternative approaches using neural networks shows that our method constructs well-performing value functions with substantially fewer parameters.

In classification problems (i.e. when the labels Y are discrete), the scaling of the test error with respect to n is determined by characterizations of the VC-dimension [2]/Rademacher complexity [3] of the function class, which in the worst case increases with its number of parameters. In regression (i.e. when the labels Y are continuous), the mean-squared error of the ordinary least-squares estimate is characterized by the condition number of the regression matrix, which is reasonable for appropriate ratios of d/n but tends to increase astronomically as d approaches n. The qualitative fear is the same: if the function class is too complex, it starts to overfit noise and can generalize poorly to unseen test data. But there is a gap between "can" and "will" -- and indeed this conventional wisdom has been challenged by the recent advent of deeper and deeper neural networks. In particular, a thought-provoking paper [4] noted that several deep neural networks generalize well despite achieving zero or close to zero training error, and being so expressive that they even have the ability to fit pure noise. As they put it, "understanding deep learning requires rethinking generalization". How can we reconcile the fact that good interpolative solutions exist with the classical bias-variance tradeoff? These phenomena are being actively investigated in a statistical sense [5,6] and a computational sense [7-9] in classification problems and/or noiseless models.

In this paper, we analyze the finite sample complexity of stochastic system identification using modern tools from machine learning and statistics. An unknown discrete-time linear system evolves over time under Gaussian noise without external inputs. The objective is to recover the system parameters as well as the Kalman filter gain, given a single trajectory of output measurements over a finite horizon of length $N$. Based on a subspace identification algorithm and a finite number of $N$ output samples, we provide non-asymptotic high-probability upper bounds for the system parameter estimation errors. Our analysis uses recent results from random matrix theory, self-normalized martingales and SVD robustness, in order to show that with high probability the estimation errors decrease with a rate of $1/\sqrt{N}$. Our non-asymptotic bounds not only agree with classical asymptotic results, but are also valid even when the system is marginally stable.

A detailed understanding of wind turbine performance status classification can improve operations and maintenance in the wind energy industry. Due to different engineering properties of wind turbines, the standard supervised learning models used for classification do not generalize across data sets obtained from different wind sites. We propose two methods to deal with the transferability of the trained models: first, data normalization in the form of power curve alignment, and second, a robust method based on convolutional neural networks and feature-space extension. We demonstrate the success of our methods on real-world data sets with industrial applications. Keywords: Machine learning, classification, generalization, CNN, wind turbine, wind energy 1. Introduction Classification of operational status is an important step for performance analysis of wind farms from data of SCADA (Supervisory Control and Data Acquisition) type.