Attention or memory? Neurointerpretable agents in space and time