On the Unique Recovery of Transport Maps and Vector Fields from Finite Measure-Valued Data