Look Up Mashable's Best: E-readers, robovacs, laptops, earbuds, smart home and more Say More Safety Net Creator Hub Versus Gift Ideas For Everyone On Your List Mashable Selects Switch Off Trending Now In My Bag VidCon with Mashable All Series It's National ICEE Day: How to get a free ICEE on Aug. 18 Soumya is a deals writer who covers consumer tech, shopping deals, and the products people use every day. With experience writing about everything from AI tools and software to smartphones and home gadgets, she enjoys breaking down product research into clear, useful recommendations. When she's not tracking deals, she's usually comparing products, digging through reviews, and figuring out what actually makes a purchase worth it. All products featured here are independently selected by our editors and writers. If you buy something through links on our site, Mashable may earn an affiliate commission.

In-context learning is a promising approach for online policy learning of offline reinforcement learning (RL) methods, which can be achieved at inference time without gradient optimization. However, this method is hindered by significant computational costs resulting from the gathering of large training trajectory sets and the need to train large Transformer models. We address this challenge by introducing an In-context Exploration-Exploitation (ICEE) algorithm, designed to optimize the efficiency of in-context policy learning. Unlike existing models, ICEE performs an exploration-exploitation trade-off at inference time within a Transformer model, without the need for explicit Bayesian inference. Consequently, ICEE can solve Bayesian optimization problems as efficiently as Gaussian process biased methods do, but in significantly less time. Through experiments in grid world environments, we demonstrate that ICEE can learn to solve new RL tasks using only tens of episodes, marking a substantial improvement over the hundreds of episodes needed by the previous in-context learning method.