On the Transferability of Whisper-based Representations for "In-the-Wild" Cross-Task Downstream Speech Applications