OmniD: Generalizable Robot Manipulation Policy via Image-Based BEV Representation

Ensuring robust generalization across diverse environments and scenarios remains a central challenge for real-world embodied systems. The generalization challenges primarily manifest in positional variations, background interference, viewpoint shifts, morphological differences, illumination changes, and environmental dynamics[1, 2]. To provide a clearer critique for the model's generalization capability, inspired by [3], we formally define in-distribution (ID), out-of-distribution (OOD) evaluations, and combinatorial-distribution (CD) for embodied scenarios. Taking object position generalization as an example, as shown in Figure 1: when the spatial distribution of pumpkins in test data aligns with the training distribution, it constitutes an ID scenario; significantly divergent distributions indicate OOD cases, while intermediate variations correspond to CD with varying discrepancy levels. Building upon this generalization capability formalization, we systematically evaluate existing methodologies' effectiveness. Methods like DP[4], ACT[5], etc [6] are capable of performing complex manipulation tasks and get a high ID success rate. They are prone to overfit to the specific ID scenario and fail to generalize to OOD. Even minor camera pose perturbations or subtle background variations can lead to significant performance degradation.