Breaking the Finite-Sample Barrier in Entropy Coupling