Train Once, Deploy Anywhere: Realize Data-Efficient Dynamic Object Manipulation