Our benchmark supports evaluation of language models using prompt-based learning as well as fine-tuning.

In most applications, the output labels of learning problems have some structure that is crucial to consider. This includes natural language processing applications, where the output may be a sentence, a sequence of parts-of-speech tags, a parse tree, or a dependency graph.

Text-to-SQL parsing, which aims at converting natural language questions into executable SQLs, has gained increasing attention in recent years.

Evaluating grammatical error correction requires metrics that reflect the diversity of valid human corrections rather than privileging a single reference. Existing frameworks, largely edit-based and English-centric, rely on rigid alignments between system and reference edits, limiting their applicability in multilingual and generative settings. This paper introduces a formal framework for \textit{fluency-based multi-reference evaluation}, framing $n$-gram similarity as an aggregation problem over multiple legitimate corrections. Within this formulation, we instantiate GLEU through four aggregation strategies--\textsc{select-best}, \textsc{simple-average}, \textsc{weighted-average}, and \textsc{merged-counts}--and analyze their properties of boundedness, monotonicity, and sensitivity to reference variation. Empirical results on Czech, Estonian, Ukrainian, and Chinese corpora show that these strategies capture complementary aspects of fluency and coverage. The framework unifies multi-reference evaluation into a principled, fluency-oriented approach that incorporates linguistic diversity without penalizing legitimate variation.

This work investigates how a multilingual transformer model represents morphosyntactic properties of questions. We introduce the Question Type and Complexity (QTC) dataset with sentences across seven languages, annotated with type information and complexity metrics including dependency length, tree depth, and lexical density. Our evaluation extends probing methods to regression labels with selectivity controls to quantify gains in generalizability. We compare layer-wise probes on frozen Glot500-m (Imani et al., 2023) representations against subword TF-IDF baselines, and a fine-tuned model. Results show that statistical features classify questions effectively in languages with explicit marking, while neural probes capture fine-grained structural complexity patterns better. We use these results to evaluate when contextual representations outperform statistical baselines and whether parameter updates reduce the availability of pre-trained linguistic information.

Sec. B, we provide more examples of the similarity distribution with/without the event and visualize To investigate the flexibility of our approach, we combine LSLD with different SOT A methods for the A VVP task. The experiments show that our denoised labels are indeed influential and can be properly employed on different SOT A methods. Effectiveness of modifying class names in prompts. Table 2, we can see that the segment-level visual metric improves by 1.7 points when we add playing As we transform objects like Accordion into human behavior (i.e. Table 2: Study the impact of varying class names to make the prompt more contextual.

Conventional methods tend to suffer from overfitting with limited supervision, as well as catastrophic forgetting due to parameter updates.