Towards Evaluating Robustness of Prompt Adherence in Text to Image Models