pong
From Pong to GTA VI: Play your way through 50 years of video games
On November 19, Rockstar Games releases Grand Theft Auto VI, ending a 13-year wait for the sequel to one of the best-selling games ever made. Few games have arrived with more riding on them. Its first trailer drew more than 90 million YouTube views in 24 hours in December 2023, a record for a game reveal. It arrives in an industry that generated $201.6bn in revenue in 2025, according to market research firm Newzoo. That business started with two paddles and a square of light. Using AI tools, Al Jazeera built six short levels inspired by the games that defined each decade, from Pong to GTA VI.
A rare two-player Computer Space arcade machine is up for auction
Before Atari, there was Syzygy Engineering, and before Home Pong, there was the Computer Space arcade machine. And then just a little bit after that, in July 1973 to be precise, there was the two-player Computer Space arcade cabinet. This sparkling emerald beauty is what we're talking about today, 56 years on. Released to the public in late 1971, the single-player version of Computer Space is widely recognized as the first arcade video game. It was designed by Nolan Bushnell and Ted Dabney of Syzygy Engineering, and distributed by Nutting Associates.
Verifiable Reinforcement Learning via Policy Extraction
While deep reinforcement learning has successfully solved many challenging control tasks, its real-world applicability has been limited by the inability to ensure the safety of learned policies. We propose an approach to verifiable reinforcement learning by training decision tree policies, which can represent complex policies (since they are nonparametric), yet can be efficiently verified using existing techniques (since they are highly structured). The challenge is that decision tree policies are difficult to train. We propose VIPER, an algorithm that combines ideas from model compression and imitation learning to learn decision tree policies guided by a DNN policy (called the oracle) and its Q-function, and show that it substantially outperforms two baselines. We use VIPER to (i) learn a provably robust decision tree policy for a variant of Atari Pong with a symbolic state space, (ii) learn a decision tree policy for a toy game based on Pong that provably never loses, and (iii) learn a provably stable decision tree policy for cart-pole. In each case, the decision tree policy achieves performance equal to that of the original DNN policy.
b6f8dc086b2d60c5856e4ff517060392-Supplemental.pdf
InEXPAND,weaugmenteachhuman evaluated state to 5 states. To verify 5issufficient, we also experimented with the numbers of augmentations required in each state to get the best performance. AGIL [50] was designed to utilize saliency map collected via human gaze. The network architectures are shown in Figure 1. Hence, we view the output of attention network as the prediction of whether a pixel should be included in a human annotated boundingbox.
Attention Trajectories as a Diagnostic Axis for Deep Reinforcement Learning
Beylier, Charlotte, Selder, Hannah, Fleig, Arthur, Hofmann, Simon M., Scherf, Nico
While deep reinforcement learning agents demonstrate high performance across domains, their internal decision processes remain difficult to interp ret when evaluated only through performance metrics. In particular, it is poorly understoo d which input features agents rely on, how these dependencies evolve during training, and how t hey relate to behavior. We introduce a scientific methodology for analyzing the learni ng process through quantitative analysis of saliency. This approach aggregates saliency in formation at the object and modality level into hierarchical attention profiles, quantifyin g how agents allocate attention over time, thereby forming attention trajectories throughout t raining. Applied to Atari benchmarks, custom Pong environments, and muscle-actuated biom echanical user simulations in visuomotor interactive tasks, this methodology uncovers a lgorithm-specific attention biases, reveals unintended reward-driven strategies, and diagnos es overfitting to redundant sensory channels. These patterns correspond to measurable behavio ral differences, demonstrating empirical links between attention profiles, learning dynam ics, and agent behavior. To assess robustness of the attention profiles, we validate our finding s across multiple saliency methods and environments. The results establish attention traj ectories as a promising diagnostic axis for tracing how feature reliance develops during train ing and for identifying biases and vulnerabilities invisible to performance metrics alone.
Verifiable Reinforcement Learning via Policy Extraction
While deep reinforcement learning has successfully solved many challenging control tasks, its real-world applicability has been limited by the inability to ensure the safety of learned policies. We propose an approach to verifiable reinforcement learning by training decision tree policies, which can represent complex policies (since they are nonparametric), yet can be efficiently verified using existing techniques (since they are highly structured). The challenge is that decision tree policies are difficult to train. We propose VIPER, an algorithm that combines ideas from model compression and imitation learning to learn decision tree policies guided by a DNN policy (called the oracle) and its Q-function, and show that it substantially outperforms two baselines. We use VIPER to (i) learn a provably robust decision tree policy for a variant of Atari Pong with a symbolic state space, (ii) learn a decision tree policy for a toy game based on Pong that provably never loses, and (iii) learn a provably stable decision tree policy for cart-pole. In each case, the decision tree policy achieves performance equal to that of the original DNN policy.
Learning Game-Playing Agents with Generative Code Optimization
Kuang, Zhiyi, Rong, Ryan, Yuan, YuCheng, Nie, Allen
We present a generative optimization approach for learning game-playing agents, where policies are represented as Python programs and refined using large language models (LLMs). Our method treats decision-making policies as self-evolving code, with current observation as input and an in-game action as output, enabling agents to self-improve through execution traces and natural language feedback with minimal human intervention. Applied to Atari games, our game-playing Python program achieves performance competitive with deep reinforcement learning (RL) baselines while using significantly less training time and much fewer environment interactions. This work highlights the promise of programmatic policy representations for building efficient, adaptable agents capable of complex, long-horizon reasoning.