To Cool or not to Cool? Temperature Network Meets Large Foundation Models via DRO