Materials
Entropy stable conservative flux form neural networks
Liu, Lizuo, Li, Tongtong, Gelb, Anne, Lee, Yoonsang
We propose an entropy-stable conservative flux form neural network (CFN) that integrates classical numerical conservation laws into a data-driven framework using the entropy-stable, second-order, and non-oscillatory Kurganov-Tadmor (KT) scheme. The proposed entropy-stable CFN uses slope limiting as a denoising mechanism, ensuring accurate predictions in both noisy and sparse observation environments, as well as in both smooth and discontinuous regions. Numerical experiments demonstrate that the entropy-stable CFN achieves both stability and conservation while maintaining accuracy over extended time domains. Furthermore, it successfully predicts shock propagation speeds in long-term simulations, {\it without} oracle knowledge of later-time profiles in the training data.
GraphXForm: Graph transformer for computer-aided molecular design with application to extraction
Pirnay, Jonathan, Rittig, Jan G., Wolf, Alexander B., Grohe, Martin, Burger, Jakob, Mitsos, Alexander, Grimm, Dominik G.
Generative deep learning has become pivotal in molecular design for drug discovery and materials science. A widely used paradigm is to pretrain neural networks on string representations of molecules and fine-tune them using reinforcement learning on specific objectives. However, string-based models face challenges in ensuring chemical validity and enforcing structural constraints like the presence of specific substructures. We propose to instead combine graph-based molecular representations, which can naturally ensure chemical validity, with transformer architectures, which are highly expressive and capable of modeling long-range dependencies between atoms. Our approach iteratively modifies a molecular graph by adding atoms and bonds, which ensures chemical validity and facilitates the incorporation of structural constraints. We present GraphXForm, a decoder-only graph transformer architecture, which is pretrained on existing compounds and then fine-tuned using a new training algorithm that combines elements of the deep cross-entropy method with self-improvement learning from language modeling, allowing stable fine-tuning of deep transformers with many layers. We evaluate GraphXForm on two solvent design tasks for liquid-liquid extraction, showing that it outperforms four state-of-the-art molecular design techniques, while it can flexibly enforce structural constraints or initiate the design from existing molecular structures.
Graph Fourier Neural ODEs: Bridging Spatial and Temporal Multiscales in Molecular Dynamics
Sun, Fang, Huang, Zijie, Wang, Haixin, Cao, Yadi, Luo, Xiao, Wang, Wei, Sun, Yizhou
Molecular dynamics simulations are crucial for understanding complex physical, chemical, and biological processes at the atomic level. However, accurately capturing interactions across multiple spatial and temporal scales remains a significant challenge. We present a novel framework that jointly models spatial and temporal multiscale interactions in molecular dynamics. Our approach leverages Graph Fourier Transforms to decompose molecular structures into different spatial scales and employs Neural Ordinary Differential Equations to model the temporal dynamics in a curated manner influenced by the spatial modes. We evaluate our model on the MD17 dataset, demonstrating consistent performance improvements over state-of-the-art baselines across multiple molecules, particularly under challenging conditions such as irregular timestep sampling and long-term prediction horizons. Ablation studies confirm the significant contributions of both spatial and temporal multiscale modeling components.
Symmetry Adapted Residual Neural Network Diabatization: Conical Intersections in Aniline Photodissociation
We present a symmetry adapted residual neural network (SAResNet) diabatization method to construct quasi-diabatic Hamiltonians that accurately represent ab initio adiabatic energies, energy gradients, and nonadiabatic couplings for moderate sized systems. Our symmetry adapted neural network inherits from the pioneering symmetry adapted polynomial and fundamental invariant neural network diabatization methods to exploit the power of neural network along with the transparent symmetry adaptation of polynomial for both symmetric and asymmetric irreducible representations. In addition, our symmetry adaptation provides a unified framework for symmetry adapted polynomial and symmetry adapted neural network, enabling the adoption of the residual neural network architecture, which is a powerful descendant of the pioneering feedforward neural network. Our SAResNet is applied to construct the full 36-dimensional coupled diabatic potential energy surfaces for aniline N-H bond photodissociation, with 2,269 data points and 32,640 trainable parameters and 190 cm-1 root mean square deviation in energy. In addition to the experimentally observed {\pi}{\pi}* and {\pi}Rydberg/{\pi}{\sigma}* states, a higher state (HOMO - 1 {\pi} to Rydberg/{\sigma}* excitation) is found to introduce an induced geometric phase effect thus indirectly participate in the photodissociation process.
Performance Evaluation of Deep Learning Models for Water Quality Index Prediction: A Comparative Study of LSTM, TCN, ANN, and MLP
Ismail, Muhammad, Abbas, Farkhanda, Shah, Shahid Munir, Aljawarneh, Mahmoud, Dhomeja, Lachhman Das, Abbas, Fazila, Shoaib, Muhammad, Alrefaei, Abdulwahed Fahad, Albeshr, Mohammed Fahad
Increased population, urbanization, adoption of modern life styles, and congested population structures pose problems of sewage disposal and pollution of surface waters like lakes. Natural water gets polluted because of weathering of rocks, seepage of soils, and mining processes, etc. [1]. Water quality assessment is used to assess the quality of water based on multiple parameters such as temperature, electrical conductivity, nitrate, phosphorus, potassium, dissolved oxygen, etc. Water Quality Index (WQI) aggregates data from these parameters and produces a single numer that is helpful for the water quality assessment [2]. It facilitates a thorough judgment of water conditions in an environment and directs resource management strategies along with the appropriate treatment plan for it [3-5]. Traditionally, WQI is estimated using different mathematical procedures [6], however, recently, Machine Learning (ML) methods are used for its more feasible and costeffective estimation [7]. Because of their robust nature to handle complex data patterns, these methods have become a viable paradigm of improved predictions.
What Features in Prompts Jailbreak LLMs? Investigating the Mechanisms Behind Attacks
Kirch, Nathalie Maria, Field, Severin, Casper, Stephen
While `jailbreaks' have been central to research on the safety and reliability of LLMs (large language models), the underlying mechanisms behind these attacks are not well understood. Some prior works have used linear methods to analyze jailbreak prompts or model refusal. Here, however, we compare linear and nonlinear methods to study the features in prompts that contribute to successful jailbreaks. We do this by probing for jailbreak success based only on the portions of the latent representations corresponding to prompt tokens. First, we introduce a dataset of 10,800 jailbreak attempts from 35 attack methods. We then show that different jailbreaking methods work via different nonlinear features in prompts. Specifically, we find that while probes can distinguish between successful and unsuccessful jailbreaking prompts with a high degree of accuracy, they often transfer poorly to held-out attack methods. We also show that nonlinear probes can be used to mechanistically jailbreak the LLM by guiding the design of adversarial latent perturbations. These mechanistic jailbreaks are able to jailbreak Gemma-7B-IT more reliably than 34 of the 35 techniques that it was trained on. Ultimately, our results suggest that jailbreaks cannot be thoroughly understood in terms of universal or linear prompt features alone.
MolCap-Arena: A Comprehensive Captioning Benchmark on Language-Enhanced Molecular Property Prediction
Edwards, Carl, Lu, Ziqing, Hajiramezanali, Ehsan, Biancalani, Tommaso, Ji, Heng, Scalia, Gabriele
Bridging biomolecular modeling with natural language information, particularly through large language models (LLMs), has recently emerged as a promising interdisciplinary research area. LLMs, having been trained on large corpora of scientific documents, demonstrate significant potential in understanding and reasoning about biomolecules by providing enriched contextual and domain knowledge. However, the extent to which LLM-driven insights can improve performance on complex predictive tasks (e.g., toxicity) remains unclear. Further, the extent to which relevant knowledge can be extracted from LLMs also remains unknown. In this study, we present Molecule Caption Arena: the first comprehensive benchmark of LLM-augmented molecular property prediction. We evaluate over twenty LLMs, including both general-purpose and domain-specific molecule captioners, across diverse prediction tasks. To this goal, we introduce a novel, battle-based rating system. Our findings confirm the ability of LLM-extracted knowledge to enhance state-of-the-art molecular representations, with notable model-, prompt-, and dataset-specific variations. Code, resources, and data are available at github.com/Genentech/molcap-arena.
Biohybrid Microrobots Based on Jellyfish Stinging Capsules and Janus Particles for In Vitro Deep-Tissue Drug Penetration
Park, Sinwook, Barak, Noga, Lotan, Tamar, Yossifon, Gilad
Microrobots engineered from self-propelling active particles, extend the reach of robotic operations to submillimeter dimensions and are becoming increasingly relevant for various tasks, such as manipulation of micro/nanoscale cargo, particularly targeted drug delivery. However, achieving deep-tissue penetration and drug delivery remain a challenge. This work developed a novel biohybrid microrobot consisting of jellyfish stinging capsules, which act as natural nanoinjectors for efficient penetration and delivery, assembled onto an active Janus particle (JP). While microrobot transport and navigation was externally controlled by magnetic field-induced rolling, capsule loading onto the JP surface was controlled by electric field. Following precise navigation of the biohybrid microrobots to the vicinity of target tissues, the capsules were activated by a specific enzyme introduced to the solution, which then triggered tubule ejection and release of the preloaded molecules. Use of such microrobots for penetration of and delivery of the preloaded drug/toxin to targeted cancer spheroids and live Caenorhabditis elegans was demonstrated in-vitro. The findings offer insights for future development of bio-inspired microrobots capable of deep penetration and drug delivery. Future directions may involve encapsulation of various drugs within different capsule types for enhanced versatility. This study may also inspire in-vivo applications involving deep tissue drug delivery.
Expert-level protocol translation for self-driving labs
Shi, Yu-Zhe, Meng, Fanxu, Hou, Haofei, Bi, Zhangqian, Xu, Qiao, Ruan, Lecheng, Wang, Qining
Recent development in Artificial Intelligence (AI) models has propelled their application in scientific discovery, but the validation and exploration of these discoveries require subsequent empirical experimentation. The concept of self-driving laboratories promises to automate and thus boost the experimental process following AI-driven discoveries. However, the transition of experimental protocols, originally crafted for human comprehension, into formats interpretable by machines presents significant challenges, which, within the context of specific expert domain, encompass the necessity for structured as opposed to natural language, the imperative for explicit rather than tacit knowledge, and the preservation of causality and consistency throughout protocol steps. Presently, the task of protocol translation predominantly requires the manual and labor-intensive involvement of domain experts and information technology specialists, rendering the process time-intensive. To address these issues, we propose a framework that automates the protocol translation process through a three-stage workflow, which incrementally constructs Protocol Dependence Graphs (PDGs) that approach structured on the syntax level, completed on the semantics level, and linked on the execution level. Quantitative and qualitative evaluations have demonstrated its performance at par with that of human experts, underscoring its potential to significantly expedite and democratize the process of scientific discovery by elevating the automation capabilities within self-driving laboratories.
Explainable few-shot learning workflow for detecting invasive and exotic tree species
Gevaert, Caroline M., Pedro, Alexandra Aguiar, Ku, Ou, Cheng, Hao, Chandramouli, Pranav, Javan, Farzaneh Dadrass, Nattino, Francesco, Georgievska, Sonja
Deep Learning methods are notorious for relying on extensive labeled datasets to train and assess their performance. This can cause difficulties in practical situations where models should be trained for new applications for which very little data is available. While few-shot learning algorithms can address the first problem, they still lack sufficient explanations for the results. This research presents a workflow that tackles both challenges by proposing an explainable few-shot learning workflow for detecting invasive and exotic tree species in the Atlantic Forest of Brazil using Unmanned Aerial Vehicle (UAV) images. By integrating a Siamese network with explainable AI (XAI), the workflow enables the classification of tree species with minimal labeled data while providing visual, case-based explanations for the predictions. Results demonstrate the effectiveness of the proposed workflow in identifying new tree species, even in data-scarce conditions. With a lightweight backbone, e.g., MobileNet, it achieves a F1-score of 0.86 in 3-shot learning, outperforming a shallow CNN. A set of explanation metrics, i.e., correctness, continuity, and contrastivity, accompanied by visual cases, provide further insights about the prediction results. This approach opens new avenues for using AI and UAVs in forest management and biodiversity conservation, particularly concerning rare or under-studied species.