Unispeaker: A Unified Approach for Multimodality-driven Speaker Generation