Towards an empirical understanding of MoE design choices