Space Time Recurrent Memory Network