Robust Acoustic and Semantic Contextual Biasing in Neural Transducers for Speech Recognition