MAFIA: Multi-Adapter Fused Inclusive LanguAge Models