Evolutionary Systems
MERGE$^3$: Efficient Evolutionary Merging on Consumer-grade GPUs
Mencattini, Tommaso, Minut, Adrian Robert, Crisostomi, Donato, Santilli, Andrea, Rodolà, Emanuele
Evolutionary model merging enables the creation of high-performing multi-task models but remains computationally prohibitive for consumer hardware. We introduce MERGE$^3$, an efficient framework that makes evolutionary merging feasible on a single GPU by reducing fitness computation costs 50$\times$ while preserving performance. MERGE$^3$ achieves this by Extracting a reduced dataset for evaluation, Estimating model abilities using Item Response Theory (IRT), and Evolving optimal merges via IRT-based performance estimators. Our method enables state-of-the-art multilingual and cross-lingual merging, transferring knowledge across languages with significantly lower computational overhead. We provide theoretical guarantees and an open-source library, democratizing high-quality model merging.
Reward-Based Collision-Free Algorithm for Trajectory Planning of Autonomous Robots
Hoyos, Jose D., Zhou, Tianyu, Lu, Zehui, Mou, Shaoshuai
This paper introduces a new mission planning algorithm for autonomous robots that enables the reward-based selection of an optimal waypoint sequence from a predefined set. The algorithm computes a feasible trajectory and corresponding control inputs for a robot to navigate between waypoints while avoiding obstacles, maximizing the total reward, and adhering to constraints on state, input and its derivatives, mission time window, and maximum distance. This also solves a generalized prize-collecting traveling salesman problem. The proposed algorithm employs a new genetic algorithm that evolves solution candidates toward the optimal solution based on a fitness function and crossover. During fitness evaluation, a penalty method enforces constraints, and the differential flatness property with clothoid curves efficiently penalizes infeasible trajectories. The Euler spiral method showed promising results for trajectory parameterization compared to minimum snap and jerk polynomials. Due to the discrete exploration space, crossover is performed using a dynamic time-warping-based method and extended convex combination with projection. A mutation step enhances exploration. Results demonstrate the algorithm's ability to find the optimal waypoint sequence, fulfill constraints, avoid infeasible waypoints, and prioritize high-reward ones. Simulations and experiments with a ground vehicle, quadrotor, and quadruped are presented, complemented by benchmarking and a time-complexity analysis.
Survey on Recent Progress of AI for Chemistry: Methods, Applications, and Opportunities
Hu, Ding, Hua, Pengxiang, Huang, Zhen
The development of artificial intelligence (AI) techniques has brought revolutionary changes across various realms. In particular, the use of AI-assisted methods to accelerate chemical research has become a popular and rapidly growing trend, leading to numerous groundbreaking works. In this paper, we provide a comprehensive review of current AI techniques in chemistry from a computational perspective, considering various aspects in the design of methods. We begin by discussing the characteristics of data from diverse sources, followed by an overview of various representation methods. Next, we review existing models for several topical tasks in the field, and conclude by highlighting some key challenges that warrant further attention.
Transfer Learning of Surrogate Models via Domain Affine Transformation Across Synthetic and Real-World Benchmarks
Pan, Shuaiqun, Vermetten, Diederick, López-Ibáñez, Manuel, Bäck, Thomas, Wang, Hao
Surrogate models are frequently employed as efficient substitutes for the costly execution of real-world processes. However, constructing a high-quality surrogate model often demands extensive data acquisition. A solution to this issue is to transfer pre-trained surrogate models for new tasks, provided that certain invariances exist between tasks. This study focuses on transferring non-differentiable surrogate models (e.g., random forest) from a source function to a target function, where we assume their domains are related by an unknown affine transformation, using only a limited amount of transfer data points evaluated on the target. Previous research attempts to tackle this challenge for differentiable models, e.g., Gaussian process regression, which minimizes the empirical loss on the transfer data by tuning the affine transformations. In this paper, we extend the previous work to the random forest model and assess its effectiveness on a widely-used artificial problem set - Black-Box Optimization Benchmark (BBOB) testbed, and on four real-world transfer learning problems. The results highlight the significant practical advantages of the proposed method, particularly in reducing both the data requirements and computational costs of training surrogate models for complex real-world scenarios.
Towards Autonomous Experimentation: Bayesian Optimization over Problem Formulation Space for Accelerated Alloy Development
Khatamsaz, Danial, Wagner, Joseph, Vela, Brent, Arroyave, Raymundo, Allaire, Douglas L.
Accelerated discovery in materials science demands autonomous systems capable of dynamically formulating and solving design problems. In this work, we introduce a novel framework that leverages Bayesian optimization over a problem formulation space to identify optimal design formulations in line with decision-maker preferences. By mapping various design scenarios to a multi attribute utility function, our approach enables the system to balance conflicting objectives such as ductility, yield strength, density, and solidification range without requiring an exact problem definition at the outset. We demonstrate the efficacy of our method through an in silico case study on a Mo-Nb-Ti-V-W alloy system targeted for gas turbine engine blade applications. The framework converges on a sweet spot that satisfies critical performance thresholds, illustrating that integrating problem formulation discovery into the autonomous design loop can significantly streamline the experimental process. Future work will incorporate human feedback to further enhance the adaptability of the system in real-world experimental settings.
Neural Genetic Search in Discrete Spaces
Kim, Hyeonah, Choi, Sanghyeok, Son, Jiwoo, Park, Jinkyoo, Kwon, Changhyun
Effective search methods are crucial for improving the performance of deep generative models at test time. In this paper, we introduce a novel test-time search method, Neural Genetic Search (NGS), which incorporates the evolutionary mechanism of genetic algorithms into the generation procedure of deep models. The core idea behind NGS is its crossover, which is defined as parent-conditioned generation using trained generative models. This approach offers a versatile and easy-to-implement search algorithm for deep generative models. We demonstrate the effectiveness and flexibility of NGS through experiments across three distinct domains: routing problems, adversarial prompt generation for language models, and molecular design.
Review for NeurIPS paper: Black-Box Ripper: Copying black-box models using generative evolutionary algorithms
Additional Feedback: The paper can be imporved by including experiments and comparison with baseline on more practical dataset. My main concerns on the dataset and results in many classes have been addressed. Black-box model stealing is also discussed in [1]. Considering previous work[1] in the context, using EA on a pre-trained GAN for model stealing is not so novel. The novelty is limited in using EA in this task.
Review for NeurIPS paper: Black-Box Ripper: Copying black-box models using generative evolutionary algorithms
The basic idea of evolving a data set that mimics a black box model of a teacher is intuitive and interesting, although not entirely novel. But the combination of EA and GAN algorithms for realizing a solution to this problem is. The experimental results presented show superiority over the tested baselines, and the paper is well written and easy to understand. One limitation is that the method only evaluated on small data sets and the description of the EA that is used needs to be better explained. The options of several reviewers were raised as a result of clarifications provided in the user response, and the consensus recommendation on this paper is to accept. Please be sure to attend to the reviewer comments as you prepare your final version.
Adaptive Learning-based Model Predictive Control Strategy for Drift Vehicles
Zhou, Bei, Hu, Cheng, Zeng, Jun, Li, Zhouheng, Betz, Johannes, Xie, Lei, Su, Hongye
Drift vehicle control offers valuable insights to support safe autonomous driving in extreme conditions, which hinges on tracking a particular path while maintaining the vehicle states near the drift equilibrium points (DEP). However, conventional tracking methods are not adaptable for drift vehicles due to their opposite steering angle and yaw rate. In this paper, we propose an adaptive path tracking (APT) control method to dynamically adjust drift states to follow the reference path, improving the commonly utilized predictive path tracking methods with released computation burden. Furthermore, existing control strategies necessitate a precise system model to calculate the DEP, which can be more intractable due to the highly nonlinear drift dynamics and sensitive vehicle parameters. To tackle this problem, an adaptive learning-based model predictive control (ALMPC) strategy is proposed based on the APT method, where an upper-level Bayesian optimization is employed to learn the DEP and APT control law to instruct a lower-level MPC drift controller. This hierarchical system architecture can also resolve the inherent control conflict between path tracking and drifting by separating these objectives into different layers. The ALMPC strategy is verified on the Matlab-Carsim platform, and simulation results demonstrate its effectiveness in controlling the drift vehicle to follow a clothoid-based reference path even with the misidentified road friction parameter.
Refining Integration-by-Parts Reduction of Feynman Integrals with Machine Learning
von Hippel, Matt, Wilhelm, Matthias
Perturbative Quantum Field Theory has proven to be a vastly successful theoretical framework for calculating precision predictions, with applications ranging from collider physics to gravitational-wave physics. A crucial step in the calculation of precision predictions is the reduction of the occurring Feynman integrals to a much smaller set of so-called master integrals, using integration-by-parts (IBP) identities [1-3]. This IBP reduction is a major bottleneck in precision calculations, requiring hundred thousands of CPU hours in current applications [4] and obstructing other applications altogether. IBP identities relate Feynman integrals with different integer exponents of the propagators as well as irreducible scalar products (ISP) in the numerator. They can easily be derived for general values of the exponents, see e.g.