Energy
Staff Data Engineer at Span.IO, Inc. - San Francisco
SPAN develops products that accelerate the rapid adoption of renewable energy in the home. The flagship SPAN Smart Panel is the first true evolution for the traditional home electric panel, harnessing enhanced technology for metering, monitoring, and control. An expanded product suite of intelligent, integrated solutions radically lowers the cost and complexity of energy upgrades–including solar, batteries and EVs–empowering homeowners to be active, resilient and informed players in the energy market. We are seeking a Staff Data Engineer to join our team building the cloud based glue that gives our users access to the rich information and controls provided by the SPAN Panel. Our system collects a large volume of energy monitoring data that needs to be stored, processed, and exposed in different ways for different end users.
Dismantling Sellafield: the epic task of shutting down a nuclear site
If you take the cosmic view of Sellafield, the superannuated nuclear facility in north-west England, its story began long before the Earth took shape. About 9bn years ago, tens of thousands of giant stars ran out of fuel, collapsed upon themselves, and then exploded. Flung out by such explosions, trillions of tonnes of uranium traversed the cold universe and wound up near our slowly materialising solar system. And here, over roughly 20m years, the uranium and other bits of space dust and debris cohered to form our planet in such a way that the violent tectonics of the young Earth pushed the uranium not towards its hot core but up into the folds of its crust. Within reach, so to speak, of the humans who eventually came along circa 300,000BC, and who mined the uranium beginning in the 1500s, learned about its radioactivity in 1896 and started feeding it into their nuclear reactors 70-odd years ago, making electricity that could be relayed to their houses to run toasters and light up Christmas trees. Sellafield compels this kind of gaze into the abyss of deep time because it is a place where multiple time spans – some fleeting, some cosmic – drift in and out of view. Laid out over six square kilometres, Sellafield is like a small town, with nearly a thousand buildings, its own roads and even a rail siding – all owned by the government, and requiring security clearance to visit. Sellafield's presence, at the end of a road on the Cumbrian coast, is almost hallucinatory. Then, having driven through a high-security gate, you're surrounded by towering chimneys, pipework, chugging cooling plants, everything dressed in steampunk. The sun bounces off metal everywhere. In some spots, the air shakes with the noise of machinery. It feels like the most manmade place in the world. Since it began operating in 1950, Sellafield has had different duties. First it manufactured plutonium for nuclear weapons.
Advancing Fusion Energy Research With Machine Learning
Machine learning is becoming an increasingly important tool in fusion research, allowing scientists to make new discoveries and improve fusion reaction efficiency. Researchers discussed the potential for using machine learning in fusion research at a recent workshop sponsored by the US Department of Energy, and identified several key areas for further study. One of the most difficult challenges in fusion research is accurately modeling and predicting the behavior of plasma, the superheated gas that powers fusion reactions. Traditional methods for simulating plasma rely on computationally intensive mathematical models, which can be difficult to solve and necessitate a significant amount of computational power. Machine learning algorithms, on the other hand, can be used to analyze large datasets and identify patterns and relationships that human experts would not be able to detect.
Ungeneralizable Contextual Logistic Bandit in Credit Scoring
Manopanjasiri, Pojtanut, Visantavarakul, Kantapong, Kiatsupaibul, Seksan
The application of reinforcement learning in credit scoring has created a unique setting for contextual logistic bandit that does not conform to the usual exploration-exploitation tradeoff but rather favors exploration-free algorithms. Through sufficient randomness in a pool of observable contexts, the reinforcement learning agent can simultaneously exploit an action with the highest reward while still learning more about the structure governing that environment. Thus, it is the case that greedy algorithms consistently outperform algorithms with efficient exploration, such as Thompson sampling. However, in a more pragmatic scenario in credit scoring, lenders can, to a degree, classify each borrower as a separate group, and learning about the characteristics of each group does not infer any information to another group. Through extensive simulations, we show that Thompson sampling dominates over greedy algorithms given enough timesteps which increase with the complexity of underlying features.
Probabilistic Constraint Tightening Techniques for Trajectory Planning with Predictive Control
Goulet, Nathan, Wang, Qian, Ayalew, Beshah
In order for automated mobile vehicles to navigate in the real world with minimal collision risks, it is necessary for their planning algorithms to consider uncertainties from measurements and environmental disturbances. In this paper, we consider analytical solutions for a conservative approximation of the mutual probability of collision between two robotic vehicles in the presence of such uncertainties. Therein, we present two methods, which we call unitary scaling and principal axes rotation, for decoupling the bivariate integral required for efficient approximation of the probability of collision between two vehicles including orientation effects. We compare the conservatism of these methods analytically and numerically. By closing a control loop through a model predictive guidance scheme, we observe through Monte-Carlo simulations that directly implementing collision avoidance constraints from the conservative approximations remains infeasible for real-time planning. We then propose and implement a convexification approach based on the tightened collision constraints that significantly improves the computational efficiency and robustness of the predictive guidance scheme.
Identifying AGN host galaxies with convolutional neural networks
Guo, Ziting, Wu, John F., Sharon, Chelsea E.
Active galactic nuclei (AGN) are supermassive black holes with luminous accretion disks found in some galaxies, and are thought to play an important role in galaxy evolution. However, traditional optical spectroscopy for identifying AGN requires time-intensive observations. We train a convolutional neural network (CNN) to distinguish AGN host galaxies from non-active galaxies using a sample of 210,000 Sloan Digital Sky Survey galaxies. We evaluate the CNN on 33,000 galaxies that are spectrally classified as composites, and find correlations between galaxy appearances and their CNN classifications, which hint at evolutionary processes that effect both galaxy morphology and AGN activity. With the advent of the Vera C. Rubin Observatory, Nancy Grace Roman Space Telescope, and other wide-field imaging telescopes, deep learning methods will be instrumental for quickly and reliably shortlisting AGN samples for future analyses.
Let's consider more general nonlinear approaches to study teleconnections of climate variables
Bueso, D., Piles, M., Camps-Valls, G.
The recent work by (Rieger et al 2021) is concerned with the problem of extracting features from spatio-temporal geophysical signals. The authors introduce the complex rotated MCA (xMCA) to deal with lagged effects and non-orthogonality of the feature representation. This method essentially (1) transforms the signals to a complex plane with the Hilbert transform; (2) applies an oblique (Varimax and Promax) rotation to remove the orthogonality constraint; and (3) performs the eigendecomposition in this complex space (Horel et al, 1984). We argue that this method is essentially a particular case of the method called rotated complex kernel principal component analysis (ROCK-PCA) introduced in (Bueso et al., 2019, 2020), where we proposed the same approach: first transform the data to the complex plane with the Hilbert transform and then apply the varimax rotation, with the only difference that the eigendecomposition is performed in the dual (kernel) Hilbert space. The latter allows us to generalize the xMCA solution by extracting nonlinear (curvilinear) features when nonlinear kernel functions are used. Hence, the solution of xMCA boils down to ROCK-PCA when the inner product is computed in the input data space instead of in the high-dimensional (possibly infinite) kernel Hilbert space to which data has been mapped. In this short correspondence we show theoretical proof that xMCA is a special case of ROCK-PCA and provide quantitative evidence that more expressive and informative features can be extracted when working with kernels; results of the decomposition of global sea surface temperature (SST) fields are shown to illustrate the capabilities of ROCK-PCA to cope with nonlinear processes, unlike xMCA.
A scalable framework for annotating photovoltaic cell defects in electroluminescence images
Otamendi, Urtzi, Martinez, Inigo, Olaizola, Igor G., Quartulli, Marco
The correct functioning of photovoltaic (PV) cells is critical to ensuring the optimal performance of a solar plant. Anomaly detection techniques for PV cells can result in significant cost savings in operation and maintenance (O&M). Recent research has focused on deep learning techniques for automatically detecting anomalies in Electroluminescence (EL) images. Automated anomaly annotations can improve current O&M methodologies and help develop decision-making systems to extend the life-cycle of the PV cells and predict failures. This paper addresses the lack of anomaly segmentation annotations in the literature by proposing a combination of state-of-the-art data-driven techniques to create a Golden Standard benchmark. The proposed method stands out for (1) its adaptability to new PV cell types, (2) cost-efficient fine-tuning, and (3) leverage public datasets to generate advanced annotations. The methodology has been validated in the annotation of a widely used dataset, obtaining a reduction of the annotation cost by 60%.
Towards Hardware-Specific Automatic Compression of Neural Networks
Krieger, Torben, Klein, Bernhard, Fröning, Holger
Compressing neural network architectures is important to allow the deployment of models to embedded or mobile devices, and pruning and quantization are the major approaches to compress neural networks nowadays. Both methods benefit when compression parameters are selected specifically for each layer. Finding good combinations of compression parameters, so-called compression policies, is hard as the problem spans an exponentially large search space. Effective compression policies consider the influence of the specific hardware architecture on the used compression methods. We propose an algorithmic framework called Galen to search such policies using reinforcement learning utilizing pruning and quantization, thus providing automatic compression for neural networks. Contrary to other approaches we use inference latency measured on the target hardware device as an optimization goal. With that, the framework supports the compression of models specific to a given hardware target. We validate our approach using three different reinforcement learning agents for pruning, quantization and joint pruning and quantization. Besides proving the functionality of our approach we were able to compress a ResNet18 for CIFAR-10, on an embedded ARM processor, to 20% of the original inference latency without significant loss of accuracy. Moreover, we can demonstrate that a joint search and compression using pruning and quantization is superior to an individual search for policies using a single compression method.
Spatially-resolved Thermometry from Line-of-Sight Emission Spectroscopy via Machine Learning
Kang, Ruiyuan, Kyritsis, Dimitrios C., Liatsis, Panos
A methodology is proposed, which addresses the caveat that line-of-sight emission spectroscopy presents in that it cannot provide spatially resolved temperature measurements in nonhomogeneous temperature fields. The aim of this research is to explore the use of data-driven models in measuring temperature distributions in a spatially resolved manner using emission spectroscopy data. Two categories of data-driven methods are analyzed: (i) Feature engineering and classical machine learning algorithms, and (ii) end-to-end convolutional neural networks (CNN). In total, combinations of fifteen feature groups and fifteen classical machine learning models, and eleven CNN models are considered and their performances explored. The results indicate that the combination of feature engineering and machine learning provides better performance than the direct use of CNN. Notably, feature engineering which is comprised of physics-guided transformation, signal representation-based feature extraction and Principal Component Analysis is found to be the most effective. Moreover, it is shown that when using the extracted features, the ensemble-based, light blender learning model offers the best performance with RMSE, RE, RRMSE and R values of 64.3, 0.017, 0.025 and 0.994, respectively. The proposed method, based on feature engineering and the light blender model, is capable of measuring nonuniform temperature distributions from low-resolution spectra, even when the species concentration distribution in the gas mixtures is unknown.