When you purchase through links in our articles, we may earn a small commission. Here are 15 geeky gifts and gadgets I'd buy for myself! 'Tis the season to get the perfect nerd gift. It could be for yourself or even your geeky friends. I don't know about you, but I love the holidays.

One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dimensional representation using a deep network.

World models are defined as a compressed spatial and temporal learned representation of an environment. The learned representation is typically a neural network, making transfer of the learned environment dynamics and explainability a challenge. In this paper, we propose an approach, Finite Automata Extraction (FAE), that learns a neuro-symbolic world model from gameplay video represented as programs in a novel domain-specific language (DSL): Retro Coder. Compared to prior world model approaches, FAE learns a more precise model of the environment and more general code than prior DSL-based approaches.

Last week, the Guardian spoke to the team behind Lili, a video game retelling of Macbeth, shown at the Cannes film festival. The headline quote from the piece was "Shakespeare would be writing for games today", which I have heard many times, and does make a lot of sense. Shakespeare worked in the Elizabethan theatre, a period in which plays were considered popularist entertainment hardly worthy of analysis or preservation – just like video games today! The authorities were also concerned about the lewd and violent nature of plays and the effect they may have on the impressionable masses – ditto! But if we agree that a 21st-century Shakespeare would be making games, what sort would he be making?

Microsoft and Google have each created models that can dream up virtual worlds, with significant limitations. And people have been using Grok, the gen-AI chatbot from Elon Musk's xAI, to make rudimentary clones of old arcade games. All you have to do is type "write me Pong" and AI (sort of) does the rest, albeit quite badly. On Feb 21, xAI employee Taylor Silveira claimed to have created an accurate version of 1980 coin-op Pac-Man using Grok 3, all the ghosts moving perfectly around their maze while Pac-Man chomps down dots, power pills and fruit. The takeaway is that AI can apparently write simple video games in seconds, so long as you have a good command of the software.

This paper introduces an extension to the arbitration graph framework designed to enhance the safety and robustness of autonomous systems in complex, dynamic environments. Building on the flexibility and scalability of arbitration graphs, the proposed method incorporates a verification step and structured fallback layers in the decision-making process. This ensures that only verified and safe commands are executed while enabling graceful degradation in the presence of unexpected faults or bugs. The approach is demonstrated using a Pac-Man simulation and further validated in the context of autonomous driving, where it shows significant reductions in accident risk and improvements in overall system safety. The bottom-up design of arbitration graphs allows for an incremental integration of new behavior components. The extension presented in this work enables the integration of experimental or immature behavior components while maintaining system safety by clearly and precisely defining the conditions under which behaviors are considered safe. The proposed method is implemented as a ready to use header-only C++ library, published under the MIT License. Together with the Pac-Man demo, it is available at github.com/KIT-MRT/arbitration_graphs.

One of the main challenges in reinforcement learning (RL) is generalisation. In typical deep RL methods this is achieved by approximating the optimal value function with a low-dimensional representation using a deep network. While this approach works well in many domains, in domains where the optimal value function cannot easily be reduced to a low-dimensional representation, learning can be very slow and unstable. This paper contributes towards tackling such challenging domains, by proposing a new method, called Hybrid Reward Architecture (HRA). HRA takes as input a decomposed reward function and learns a separate value function for each component reward function. Because each component typically only depends on a subset of all features, the corresponding value function can be approximated more easily by a low-dimensional representation, enabling more effective learning. We demonstrate HRA on a toy-problem and the Atari game Ms. Pac-Man, where HRA achieves above-human performance.

Pac-Man is a classic arcade game that deserves all the love. Guiding an abstract mouth around a ghost-patrolled maze in pursuit of dots is pure joy. As good as it is, I never imagined the greedy yellow circle would bring my family closer together, but that's exactly what happened this summer. Ever since the Arcade1Up Pac-Man Deluxe Arcade Machine displaced a tatty old cat tree in the corner of my office, I have been battling for the high score with my eldest teen. As a teenager, figuring out what you want to do and who you want to be is tough at the best of times.

An artificial intelligence can accurately predict how a monkey plays the Pac-Man video game and mimic the animals' eye movements when they do this. Tianming Yang at the Chinese Academy of Sciences and his colleagues trained two rhesus monkeys to play Pac-Man by rewarding them with juice for collecting all the dots in a maze and evading capture by ghosts.

A long term goal of Interactive Reinforcement Learning is to incorporate nonexpert human feedback to solve complex tasks. Some state-of-the-art methods have approached this problem by mapping human information to rewards and values and iterating over them to compute better control policies. In this paper we argue for an alternate, more effective characterization of human feedback: Policy Shaping. We introduce Advise, a Bayesian approach that attempts to maximize the information gained from human feedback by utilizing it as direct policy labels. We compare Advise to state-of-the-art approaches and show that it can outperform them and is robust to infrequent and inconsistent human feedback.