Adaptive Multi-Modality Prompt Learning