A Convolution and Attention Based Encoder for Reinforcement Learning under Partial Observability

B. Observation History The core contribution of this work is a novel history encoder for processing historical observations, which integrates two key operations: depthwise separable convolution and multi-head attention. The background of these operations is briefly reviewed below. Depthwise separable convolution [33] is a streamlined variant of standard convolution that reduces both parameter count and computational cost. It decomposes the operation into two steps: (1) a depthwise convolution, which applies a single filter to each input channel, and (2) a pointwise convolution, which uses a 1 1 convolution to linearly combine the outputs of the depthwise stage. This factorization enables efficient extraction of spatial and cross-channel features while maintaining strong representational capacity. It has been widely adopted in lightweight neural architectures such as MobileNet [34] and is particularly well suited to real-time and resource-constrained applications. Multi-head attention [9] is a fundamental component of Transformer architectures, enabling the model to capture diverse patterns across different representation subspaces.