The performance differential of large language models (LLM) between languages hinders their effective deployment in many regions, inhibiting the potential economic and societal value of generative AI tools in many communities. However, the development of functional LLMs in many languages (i.e., multilingual LLMs) is bottlenecked by the lack of high-quality evaluation resources in languages other than English. Moreover, current practices in multilingual benchmark construction often translate English resources, ignoring the regional and cultural knowledge of the environments in which multilingual systems would be used. In this work, we construct an evaluation suite of 197,243 QA pairs from local exam sources to measure the capabilities of multilingual LLMs in a variety of regional contexts. The rapid advancement of AI technologies underscores the importance of developing LLMs that are proficient across diverse linguistic and cultural contexts, ensuring fair and equitable performance for stakeholders from various language groups. However, the lack of high-quality evaluation benchmarks in many languages discourages practitioners from training multilingual LLMs to meet this challenge. This evaluation gap limits the effective deployment of LLMs for many regions, exacerbates digital divides, and inhibits the economic and societal value of AI tools in many underserved communities. The source of this gap is the multitude of challenges in evaluating LLMs for multilingual contexts. First, at a meta-level, the majority of benchmarks for LLMs are only in English (Hendrycks et al., 2020, inter alia). Technical challenges also abound due to the manner in which multilingual datasets are often collected. Certain datasets are constructed using manually applied templates, resulting in low prompt and completion diversity (Muennighoff et al., 2022). Many more are composed of translations from high-resource languages (e.g., English; Holtermann et al., 2024; Myung et al., 2024; Lai et al., 2023; Foroutan et al., 2023). These datasets often contain errors (Ponti et al., 2020; Plaza et al., 2024) and create translationese artifacts (Vanmassenhove et al., 2021; Hartung et al., 2023; Savoldi et al., 2021; Ji et al., 2023).

Two fundamental problems in the statistical analysis of Gaussian databases are correlation detection (or testing) and alignment recovery. Correlation detection of two databases is basically a hypothesis testing problem; under the null hypothesis, the databases are independent, and under the alternate hypothesis, there exists a permutation, for which the databases are correlated. In this task, the main objective is to attain the best trade-off between the type-I and type-II error probabilities. In the problem of database alignment recovery, we make an assumption that the two databases are correlated, and want to estimate the underlying permutation. The objective is to minimize the probability of alignment error. While alignment recovery of databases with n sequences, each containing d Gaussian entries has been recently studied in [1] and correlation detection of such Gaussian databases has been lately explored in [2], it seems very natural to tackle the two individual problems together as a joint problem of correlation detection and alignment recovery. In addition, we also refer to the problem of partial alignment recovery, in which one would like to estimate only part of the underlying alignment between the databases. The main reasons for preferring partial alignment recovery instead of full alignment recovery are as follows.