Why Train More? Effective and Efficient Membership Inference via Memorization