Revisiting Recurrent Reinforcement Learning with Memory Monoids

Since these efficient models do not share sequence length We discover that the recurrent update of limitations with past models, we question whether the use these models is a monoid, leading us to formally of segments is still necessary. After highlighting the empirical define a novel memory monoid framework. We and theoretical shortcomings of segments, we propose revisit the traditional approach to batching in recurrent an alternative batching method. Our method improves RL, highlighting both theoretical and empirical sample efficiency across various tasks and memory models, deficiencies. Leveraging the properties while simplifying implementation. of memory monoids, we propose a new batching method that improves sample efficiency, increases the return, and simplifies the implementation Contributions of recurrent loss functions in RL. 1. We propose the memory monoid, a unifying framework for efficient sequence models.