DiffPrompter: Differentiable Implicit Visual Prompts for Semantic-Segmentation in Adverse Conditions