Transferring Foundation Models for Generalizable Robotic Manipulation