Simulate Less, Expect More: Bringing Robot Swarms to Life via Low-Fidelity Simulations