Model-based reinforcement learning (RL) algorithms, which learn a dynamics model from logged experience and perform conservative planning under the learned model, have emerged as a promising paradigm for offline reinforcement learning (offline RL). However, practical variants of such model-based algorithms rely on explicit uncertainty quantification for incorporating conservatism. Uncertainty estimation with complex models, such as deep neural networks, can be difficult and unreliable. We empirically find that uncertainty estimation is not accurate and leads to poor performance in certain scenarios in offline model-based RL. We overcome this limitation by developing a new model-based offline RL algorithm, COMBO, that trains a value function using both the offline dataset and data generated using rollouts under the model while also additionally regularizing the value function on out-of-support state-action tuples generated via model rollouts.

This paper focuses on Bayesian Optimization (BO) for objectives on combinatorial search spaces, including ordinal and categorical variables. Despite the abundance of potential applications of Combinatorial BO, including chipset configuration search and neural architecture search, only a handful of methods have been proposed. We introduce COMBO, a new Gaussian Process (GP) BO.

Foundation models (FMs) trained with different objectives and data learn diverse representations, making some more effective than others for specific downstream tasks. Existing adaptation strategies, such as parameter-efficient fine-tuning, focus on individual models and do not exploit the complementary strengths across models. Probing methods offer a promising alternative by extracting information from frozen models, but current techniques do not scale well with large feature sets and often rely on dataset-specific hyperparameter tuning. We propose Combined backBones (ComBo), a simple and scalable probing-based adapter that effectively integrates features from multiple models and layers. ComBo compresses activations from layers of one or more FMs into compact token-wise representations and processes them with a lightweight transformer for task-specific prediction. Crucially, ComBo does not require dataset-specific tuning or backpropagation through the backbone models. However, not all models are equally relevant for all tasks. To address this, we introduce a mechanism that leverages ComBo's joint multi-backbone probing to efficiently evaluate each backbone's task-relevance, enabling both practical model comparison and improved performance through selective adaptation. On the 19 tasks of the VTAB-1k benchmark, ComBo outperforms previous probing methods, matches or surpasses more expensive alternatives, such as distillation-based model merging, and enables efficient probing of tuned models. Our results demonstrate that ComBo offers a practical and general-purpose framework for combining diverse representations from multiple FMs.

COMBO satisfies a policy improvement guarantee in the offline setting.

Applying BO with GPs on combinatorial spaces is, therefore, not straightforward.

We thank all the reviewers for their insightful and encouraging feedback. Due to the discrete nature of COMBO's search space, the implementation detail is slightly different. In contrast, in COMBO's combinatorial graphs, we have spray vertices. As R2 suggested, this heuristic promotes exploitation. Using random vertices for exploration is similar to Spearmint.

Bayesian Optimization (BO) has the potential to solve various combinatorial tasks, ranging from materials science to neural architecture search. However, BO requires specialized kernels to effectively model combinatorial domains. Recent efforts have introduced several combinatorial kernels, but the relationships among them are not well understood. To bridge this gap, we develop a unifying framework based on heat kernels, which we derive in a systematic way and express as simple closed-form expressions. Using this framework, we prove that many successful combinatorial kernels are either related or equivalent to heat kernels, and validate this theoretical claim in our experiments. Moreover, our analysis confirms and extends the results presented in Bounce: certain algorithms' performance decreases substantially when the unknown optima of the function do not have a certain structure. In contrast, heat kernels are not sensitive to the location of the optima. Lastly, we show that a fast and simple pipeline, relying on heat kernels, is able to achieve state-of-the-art results, matching or even outperforming certain slow or complex algorithms.

Gear The Bambu Lab A1 is the best multi-color 3D printer for beginners and it's up to $260 off at Amazon Bambu Lab 3D printers rarely go on-sale, but Amazon has every version of the auto-leveling A1 deeply discounted right now. We may earn revenue from the products available on this page and participate in affiliate programs. Bambu Labs makes some of our very favorite 3D printers on the market, but they rarely go on sale. Even on big shopping holidays, they tend to hover at or around retail prices. Right now, Amazon has one of the company's most popular (and accessible) models on huge discount.

Gear Cameras Up your content creation game with these discounted DJI action cameras during Amazon's Big Deal Days sale Whether you're an aspiring YouTuber or just want better home movies, these DJI filmmaking tools are worth the investment. We may earn revenue from the products available on this page and participate in affiliate programs. Smartphones have great cameras built-in, but sometimes you need to bring a camera into a dangerous situation. DJI's Osmo action cameras are built super tough so you can put them in harm's way without worrying about ruining the device on which you watch Tik Toks. Right now, Amazon has DJI action cameras (as well as camera-stabilizing gimbals) on sale for their cheapest prices of the year during the Prime Big Deal Days sale.

Applying BO with GPs on combinatorial spaces is, therefore, not straightforward.