While deep learning's performance has led it to become the dominant paradigm in machine learning,

Specifically, we introduce Multiway Transformer, where each block contains a pool of modality-specific experts and a shared self-attention layer.

As language models grow ever larger, the need for large-scale high-quality text datasets has never been more pressing, especially in multilingual settings.

Alignment with high-resource standard languages is often assumed to aid the modeling of related low-resource varieties. We challenge this assumption by demonstrating that excessive representational entanglement with a dominant variety, such as Modern Standard Arabic (MSA) in relation to Arabic dialects, can actively hinder generative modeling. We present the first comprehensive causal study of this phenomenon by analyzing and directly intervening in the internal representation geometry of large language models (LLMs). Our key contribution is an online variational probing framework that continuously estimates the subspace of the standard variety during fine-tuning, enabling projection-based decoupling from this space. While our study uses Arabic as a case due to its unusually rich parallel resources across 25 dialects, the broader motivation is methodological: dialectal MT serves as a controlled proxy for generative tasks where comparable multi-variety corpora are unavailable. Across 25 dialects, our intervention improves generation quality by up to +4.9 chrF++ and +2.0 on average compared to standard fine-tuning, despite a measured tradeoff in standard-language performance. These results provide causal evidence that subspace dominance by high-resource varieties can restrict generative capacity for related varieties. More generally, we unify geometric and information-theoretic probing with subspace-level causal interventions, offering practical tools for improving generative modeling in closely related language families and, more broadly, for controlling representational allocation in multilingual and multi-domain LLMs.

In this paper, we present the SALAMANDRATA family of models, an improved iteration of SALAMANDRA LLMs (Gonzalez-Agirre et al., 2025) specifically trained to achieve strong performance in translation-related tasks for 38 European languages. SALAMANDRATA comes in two scales: 2B and 7B parameters. For both versions, we applied the same training recipe with a first step of continual pre-training on parallel data, and a second step of supervised fine-tuning on high-quality instructions. The BSC submission to the WMT25 General Machine Translation shared task is based on the 7B variant of SALAMANDRATA. We first adapted the model vocabulary to support the additional non-European languages included in the task. This was followed by a second phase of continual pre-training and supervised fine-tuning, carefully designed to optimize performance across all translation directions for this year's shared task. For decoding, we employed two quality-aware strategies: Minimum Bayes Risk Decoding and Tuned Re-ranking using COMET and COMET-KIWI respectively. We publicly release both the 2B and 7B versions of SALAMANDRATA, along with the newer SALAMANDRATA-V2 model, on Hugging Face1

A.1 Summary of Commonly Used Metrics for T ext Generation Table 1: Summary of commonly used metrics for text generation. For settings and tasks, we only list the ones justified by the original paper for each metric. We conduct experiments on WMT19, and the results are shown in Tab. 2. We don't observe A.3 Prompt Set In Tab. 3, we list the full prompt set for both s h direction and h r direction. Prompt Set s h Last Tersely Succinctly In summation To put it succinctly After In brief All in all To summarize Bringing up the rear Behind In short In outline In a nutshell To come to the point Lastly Concisely In closing In conclusion In the final analysis In sum In precis In passing In winding up Without wasting words To end In a word To conclude Last in order At the end of the day Curtly Compactly Summarising In a few words Without waste of words Crisply Summarily In the rear As a final point Finally yet importantly At last To sum up Summarizing Not least of all To put it in a nutshell Pithily Basically Laconically To put it briefly When all is said and done Shortly In the end At the rear Not to mince words To cut a long story short In fine At the end To be brief Last but not least Not to beat about the bush Finally In essence Last of all Just as importantly In drawing things to a close Briefly Ultimately Elliptically To put it concisely Not to put too fine a point on ith r As To wit As it were Case in point As an illustration sc. That is Especially That is to say To give an example i.e.

Auto-regressive language models have been widely used for text generation. With the recent development of large-scale pre-trained language models (Radford et al., 2019; Brown et al., 2020; Raffel et al., 2020; Lewis et al., 2020), they have achieved state-of-the-art performances in applications