Long Context Modeling with Ranked Memory-Augmented Retrieval