Specifically, we introduce Multiway Transformer, where each block contains a pool of modality-specific experts and a shared self-attention layer.

The increasing reliance on ML models in high-stakes tasks has raised a major concern about fairness violations.

Hypothesis (L TH): a randomly-initialized dense neural network contains an extremely sparse subnet-work (i.e., a winning lottery ticket) such that, when trained from scratch with weights being reset to its initialization, can achieve similar performance to the original dense network within similar

Existing tensor decomposition objectives (e.g., Frobenius norm) are designed for fitting raw data under