Expanding the Boundaries of Vision Prior Knowledge in Multi-modal Large Language Models