Energy
Materials-Discovery Workflows Guided by Symbolic Regression: Identifying Acid-Stable Oxides for Electrocatalysis
Nair, Akhil S., Foppa, Lucas, Scheffler, Matthias
The efficiency of active learning (AL) approaches to identify materials with desired properties relies on the knowledge of a few parameters describing the property. However, these parameters are unknown if the property is governed by a high intricacy of many atomistic processes. Here, we develop an AL workflow based on the sure-independence screening and sparsifying operator (SISSO) symbolic-regression approach. SISSO identifies the few, key parameters correlated with a given materials property via analytical expressions, out of many offered primary features. Crucially, we train ensembles of SISSO models in order to quantify mean predictions and their uncertainty, enabling the use of SISSO in AL. By combining bootstrap sampling to obtain training datasets with Monte-Carlo feature dropout, the high prediction errors observed by a single SISSO model are improved. Besides, the feature dropout procedure alleviates the overconfidence issues observed in the widely used bagging approach. We demonstrate the SISSO-guided AL workflow by identifying acid-stable oxides for water splitting using high-quality DFT-HSE06 calculations. From a pool of 1470 materials, 12 acid-stable materials are identified in only 30 AL iterations. The materials property maps provided by SISSO along with the uncertainty estimates reduce the risk of missing promising portions of the materials space that were overlooked in the initial, possibly biased dataset.
Modeling, Planning, and Control for Hybrid UAV Transition Maneuvers
Small unmanned aerial vehicles (UAVs) have become standard tools in reconnaissance and surveying for both civilian and defense applications. In the future, UAVs will likely play a pivotal role in autonomous package delivery, but current multi-rotor candidates suffer from poor energy efficiency leading to insufficient endurance and range. In order to reduce the power demands of package delivery UAVs while still maintaining necessary hovering capabilities, companies like Amazon are experimenting with hybrid Vertical Take-Off and Landing (VTOL) platforms. Tailsitter VTOLs offer a mechanically simple and cost-effective solution compared to other hybrid VTOL configurations, and while advances in hardware and microelectronics have optimized the tailsitter for package delivery, the software behind its operation has largely remained a critical barrier to industry adoption. Tailsitters currently lack a generic, computationally efficient method of control that can provide strong safety and robustness guarantees over the entire flight domain. Further, tailsitters lack a closed-form method of designing dynamically feasible transition maneuvers between hover and cruise. In this paper, we survey the modeling and control methods currently implemented on small-scale tailsitter UAVs, and attempt to leverage a nonlinear dynamic model to design physically realizable, continuous-pitch transition maneuvers at constant altitude. Primary results from this paper isolate potential barriers to constant-altitude transition, and a novel approach to bypassing these barriers is proposed. While initial results are unsuccessful at providing feasible transition, this work acts as a stepping stone for future efforts to design new transition maneuvers that are safe, robust, and computationally efficient.
Radon Implicit Field Transform (RIFT): Learning Scenes from Radar Signals
Bao, Daqian, Saad-Falcon, Alex, Romberg, Justin
Data acquisition in array signal processing (ASP) is costly because achieving high angular and range resolutions necessitates large antenna apertures and wide frequency bandwidths, respectively. The data requirements for ASP problems grow multiplicatively with the number of viewpoints and frequencies, significantly increasing the burden of data collection, even for simulation. Implicit Neural Representations (INRs) -- neural network-based models of 3D objects and scenes -- offer compact and continuous representations with minimal radar data. They can interpolate to unseen viewpoints and potentially address the sampling cost in ASP problems. In this work, we select Synthetic Aperture Radar (SAR) as a case from ASP and propose Radon Implicit Field Transform (RIFT). RIFT consists of two components: a classical forward model for radar (Generalized Radon Transform, GRT), and an INR based scene representation learned from radar signals. This method can be extended to other ASP problems by replacing the GRT with appropriate algorithms corresponding to different data modalities. In our experiments, we first synthesize radar data using the GRT. We then train the INR model on this synthetic data by minimizing the reconstruction error of the radar signal. After training, we render the scene using the trained INR and evaluate our scene representation against the ground truth scene. Due to the lack of existing benchmarks, we introduce two main new error metrics: phase-Root Mean Square Error (p-RMSE) for radar signal interpolation, and magnitude-Structural Similarity Index measure(m-SSIM) for scene reconstruction. These metrics adapt traditional error measures to account for the complex nature of radar signals. Compared to traditional scene models in radar signal processing, with only 10% data footprint, our RIFT model achieves up to 188% improvement in scene reconstruction.
Characterizing Jupiter's interior using machine learning reveals four key structures
Ziv, Maayan, Galanti, Eli, Howard, Saburo, Guillot, Tristan, Kaspi, Yohai
The internal structure of Jupiter is constrained by the precise gravity field measurements by NASA's Juno mission, atmospheric data from the Galileo entry probe, and Voyager radio occultations. Not only are these observations few compared to the possible interior setups and their multiple controlling parameters, but they remain challenging to reconcile. As a complex, multidimensional problem, characterizing typical structures can help simplify the modeling process. We used NeuralCMS, a deep learning model based on the accurate concentric Maclaurin spheroid (CMS) method, coupled with a fully consistent wind model to efficiently explore a wide range of interior models without prior assumptions. We then identified those consistent with the measurements and clustered the plausible combinations of parameters controlling the interior. We determine the plausible ranges of internal structures and the dynamical contributions to Jupiter's gravity field. Four typical interior structures are identified, characterized by their envelope and core properties. This reduces the dimensionality of Jupiter's interior to only two effective parameters. Within the reduced 2D phase space, we show that the most observationally constrained structures fall within one of the key structures, but they require a higher 1 bar temperature than the observed value. We provide a robust framework for characterizing giant planet interiors with consistent wind treatment, demonstrating that for Jupiter, wind constraints strongly impact the gravity harmonics while the interior parameter distribution remains largely unchanged. Importantly, we find that Jupiter's interior can be described by two effective parameters that clearly distinguish the four characteristic structures and conclude that atmospheric measurements may not fully represent the entire envelope.
A Comprehensive Survey and Guide to Multimodal Large Language Models in Vision-Language Tasks
Liang, Chia Xin, Tian, Pu, Yin, Caitlyn Heqi, Yua, Yao, An-Hou, Wei, Ming, Li, Wang, Tianyang, Bi, Ziqian, Liu, Ming
This survey and application guide to multimodal large language models(MLLMs) explores the rapidly developing field of MLLMs, examining their architectures, applications, and impact on AI and Generative Models. Starting with foundational concepts, we delve into how MLLMs integrate various data types, including text, images, video and audio, to enable complex AI systems for cross-modal understanding and generation. It covers essential topics such as training methods, architectural components, and practical applications in various fields, from visual storytelling to enhanced accessibility. Through detailed case studies and technical analysis, the text examines prominent MLLM implementations while addressing key challenges in scalability, robustness, and cross-modal learning. Concluding with a discussion of ethical considerations, responsible AI development, and future directions, this authoritative resource provides both theoretical frameworks and practical insights. It offers a balanced perspective on the opportunities and challenges in the development and deployment of MLLMs, and is highly valuable for researchers, practitioners, and students interested in the intersection of natural language processing and computer vision.
A Uranium-Mining Boom Is Sweeping Through Texas
This story originally appeared on Inside Climate News and is part of the Climate Desk collaboration. In the old ranchlands of South Texas, dormant uranium mines are coming back online. A collection of new ones hope to start production soon, extracting radioactive fuel from the region's shallow aquifers. These mines are the leading edge of what government and industry leaders in Texas hope will be a nuclear renaissance, as America's latent nuclear sector begins to stir again. Texas is currently developing a host of high-tech industries that require enormous amounts of electricity, from cryptocurrency mines and artificial intelligence to hydrogen production and seawater desalination.
Chugoku Electric restarts Shimane reactor for first time in 13 years
Chugoku Electric Power on Saturday restarted its Shimane nuclear power station in western Japan, shuttered since shortly after the 2011 Fukushima meltdown, the company said. The long-delayed restart of the 820 plant's megawatt (MW) No. 2 reactor, which was shut down in January 2012, boosts the number of Japan's operational reactors to 14, with a combined capacity of 13,253 MW. Japan's demand for liquefied natural gas and thermal coal is expected to fall next year, with Tohoku Electric Power also recently resuming operations of the 825 MW No. 2 reactor at its Onagawa nuclear power plant in northern Japan. The increased operation of nuclear power plants is expected to help Japan meet the growing power demand from semiconductor plants and data centers that support artificial intelligence applications. The government anticipates power output to grow to between 1.35 trillion and 1.5 trillion kilowatt-hours (kWh) by 2050, from 1 trillion kWh projected for this decade, as Japan establishes more data centers, chip factories and other energy-intensive businesses.
STONet: A novel neural operator for modeling solute transport in micro-cracked reservoirs
Haghighat, Ehsan, Adeli, Mohammad Hesan, Mousavi, S Mohammad, Juanes, Ruben
In this work, we develop a novel neural operator, the Solute Transport Operator Network (STONet), to efficiently model contaminant transport in micro-cracked reservoirs. The model combines different networks to encode heterogeneous properties effectively. By predicting the concentration rate, we are able to accurately model the transport process. Numerical experiments demonstrate that our neural operator approach achieves accuracy comparable to that of the finite element method. The previously introduced Enriched DeepONet architecture has been revised, motivated by the architecture of the popular multi-head attention of transformers, to improve its performance without increasing the compute cost. The computational efficiency of the proposed model enables rapid and accurate predictions of solute transport, facilitating the optimization of reservoir management strategies and the assessment of environmental impacts. The data and code for the paper will be published at https://github.com/ehsanhaghighat/STONet.
M$^3$PC: Test-time Model Predictive Control for Pretrained Masked Trajectory Model
Wen, Kehan, Hu, Yutong, Mu, Yao, Ke, Lei
Recent work in Offline Reinforcement Learning (RL) has shown that a unified Transformer trained under a masked auto-encoding objective can effectively capture the relationships between different modalities (e.g., states, actions, rewards) within given trajectory datasets. However, this information has not been fully exploited during the inference phase, where the agent needs to generate an optimal policy instead of just reconstructing masked components from unmasked ones. Given that a pretrained trajectory model can act as both a Policy Model and a World Model with appropriate mask patterns, we propose using Model Predictive Control (MPC) at test time to leverage the model's own predictive capability to guide its action selection. Empirical results on D4RL and RoboMimic show that our inference-phase MPC significantly improves the decision-making performance of a pretrained trajectory model without any additional parameter training. Furthermore, our framework can be adapted to Offline to Online (O2O) RL and Goal Reaching RL, resulting in more substantial performance gains when an additional online interaction budget is provided, and better generalization capabilities when different task targets are specified. The Masked Modeling paradigm has a simple, self-supervised training objective: predicting a randomly masked subset of the original sequence. It has become a powerful technique for generation or representation learning for sequential data, e.g., language tokens (Devlin et al., 2018) or image patches (He et al., 2022). Unlike autoregressive models like GPT (Brown et al., 2020), which condition only on the past context in the "left", bidirectional models trained with this objective learn to model the context from both sides, leading to richer representations and deeper understandings of the data's underlying dependencies. Given that a sequential decision-making trajectory inherently involves a sequence of states s and actions a, and other optional augmented properties like return-to-go (RTG) g (Chen et al., 2021) or approximate state-action value v (Yamagata et al., 2023) across T timesteps, the mask modeling paradigm can be adapted easily for sequential decision-making tasks. For example, in the case of Reinforcement Learning, the policy output P(a|s) at each time step can be regarded as predicting a masked action a conditioned on given states s.
Leveraging Time-Series Foundation Model for Subsurface Well Logs Prediction and Anomaly Detection
Koeshidayatullah, Ardiansyah, Al-Fakih, Abdulrahman, Kaka, SanLinn Ismael
The rise in energy demand highlights the importance of suitable subsurface storage, requiring detailed and accurate subsurface characterization often reliant on high-quality borehole well log data. However, obtaining complete well-log data is costly and time-consuming, with missing data being common due to borehole conditions or tool errors. While machine learning and deep learning algorithms have been implemented to address these issues, they often fail to capture the intricate, nonlinear relationships and long-term dependencies in complex well log sequences. Additionally, prior AI-driven models typically require retraining when introduced to new datasets and are constrained to deployment in the same basin. In this study, we explored and evaluated the potential of a time-series foundation model leveraging transformer architecture and a generative pre-trained approach for predicting and detecting anomalies in borehole well log data. Specifically, we fine-tuned and adopted the TimeGPT architecture to forecast key log responses and detect anomalies with high accuracy. Our proposed model demonstrated excellent performance, achieving R2 of up to 87% and a mean absolute percentage error (MAPE) as low as 1.95%. Additionally, the model's zero-shot capability successfully identified subtle yet critical anomalies, such as drilling hazards or unexpected geological formations, with an overall accuracy of 93%. The model represents a significant advancement in predictive accuracy and computational efficiency, enabling zero-shot inference through fine-tuning. Its application in well-log prediction enhances operational decision-making while reducing risks associated with subsurface exploration. These findings demonstrate the model's potential to transform well-log data analysis, particularly in complex geological settings.