Memory Population in Continual Learning via Outlier Elimination