The EU's Artificial Intelligence Act (AI Act) is a significant step towards responsible AI development, but lacks clear technical interpretation, making it difficult to assess models' compliance. This work presents COMPL-AI, a comprehensive framework consisting of (i) the first technical interpretation of the EU AI Act, translating its broad regulatory requirements into measurable technical requirements, with the focus on large language models (LLMs), and (ii) an open-source Act-centered benchmarking suite, based on thorough surveying and implementation of state-of-the-art LLM benchmarks. By evaluating 12 prominent LLMs in the context of COMPL-AI, we reveal shortcomings in existing models and benchmarks, particularly in areas like robustness, safety, diversity, and fairness. This work highlights the need for a shift in focus towards these aspects, encouraging balanced development of LLMs and more comprehensive regulation-aligned benchmarks. Simultaneously, COMPL-AI for the first time demonstrates the possibilities and difficulties of bringing the Act's obligations to a more concrete, technical level. As such, our work can serve as a useful first step towards having actionable recommendations for model providers, and contributes to ongoing efforts of the EU to enable application of the Act, such as the drafting of the GPAI Code of Practice.

Automated planning is a form of declarative problem solving which has recently drawn attention from the machine learning (ML) community. ML has been applied to planning either as a way to test `reasoning capabilities' of architectures, or more pragmatically in an attempt to scale up solvers with learned domain knowledge. In practice, planning problems are easy to solve but hard to optimise. However, ML approaches still struggle to solve many problems that are often easy for both humans and classical planners. In this paper, we thus propose a new ML approach that allows users to specify background knowledge (BK) through Datalog rules to guide both the learning and planning processes in an integrated fashion. By incorporating BK, our approach bypasses the need to relearn how to solve problems from scratch and instead focuses the learning on plan quality optimisation. Experiments with BK demonstrate that our method successfully scales and learns to plan efficiently with high quality solutions from small training data generated in under 5 seconds.

Diffusion models have seen notable success in continuous domains, leading to the development of discrete diffusion models (DDMs) for discrete variables. While parallel sampling methods like τ-leaping accelerate this process, they introduce Compounding Decoding Error (CDE), where discrepancies arise between the true distribution and the approximation from parallel token generation, leading to degraded sample quality. More precisely, we derive a practical upper bound on CDE and propose an efficient algorithm for searching for the optimal sampling schedule. Extensive experiments across image, music, and text generation show that JYS significantly improves sampling quality, establishing it as a versatile framework for enhancing DDM performance for fast sampling. Diffusion models (Sohl-Dickstein et al., 2015; Song et al., 2021b; Ho et al., 2020; Song et al., 2021a; Karras et al., 2022) have achieved remarkable success in generation tasks within the continuous domain. However, certain modalities, such as text and music, inherently possess discrete features. Nevertheless, like their continuous counterparts, DDMs encounter a significant bottleneck in sampling speed due to their progressive refinement process. In contrast to continuous-domain diffusion models, where sampling dynamics are driven by samplewise differential equations (Song et al., 2021b), allowing for the direct application of wellestablished numerical methods to accelerate generation, enhancing speed in DDMs poses a significant challenge. To address this, researchers have proposed fast and efficient samplers, including notable methods such as the τ-leaping (Campbell et al., 2022; Lezama et al., 2022; Sun et al., 2023) and k-Gillespie algorithms (Zhao et al., 2024), which facilitate parallel sampling of multiple tokens in a single step. However, this parallel but independent sampling introduces Compounding Decoding Error (CDE) (Lezama et al., 2022), which arises from a mismatch between the training and inference distributions of intermediate latents during parallel sampling. Specifically, while each token is generated according to its marginal distribution, the joint distribution deviates from the learned distribution. To mitigate this issue, the predictor-corrector (PC) sampler (Campbell et al., 2022) has been proposed.

In recent years, training data attribution (TDA) methods have emerged as a promising direction for the interpretability of neural networks. While research around TDA is thriving, limited effort has been dedicated to the evaluation of attributions. Similar to the development of evaluation metrics for traditional feature attribution approaches, several standalone metrics have been proposed to evaluate the quality of TDA methods across various contexts. However, the lack of a unified framework that allows for systematic comparison limits trust in TDA methods and stunts their widespread adoption. To address this research gap, we introduce Quanda, a Python toolkit designed to facilitate the evaluation of TDA methods. Beyond offering a comprehensive set of evaluation metrics, Quanda provides a uniform interface for seamless integration with existing TDA implementations across different repositories, thus enabling systematic benchmarking. The toolkit is user-friendly, thoroughly tested, well-documented, and available as an open-source library on PyPi and under https://github.com/dilyabareeva/quanda.

With the pervasive deployment of Machine Learning (ML) models in real-world applications, verifying and auditing properties of ML models have become a central concern. In this work, we focus on three properties: robustness, individual fairness, and group fairness. We discuss two approaches for auditing ML model properties: estimation with and without reconstruction of the target model under audit. Though the first approach is studied in the literature, the second approach remains unexplored. For this purpose, we develop a new framework that quantifies different properties in terms of the Fourier coefficients of the ML model under audit but does not parametrically reconstruct it. We propose the Active Fourier Auditor (AFA), which queries sample points according to the Fourier coefficients of the ML model, and further estimates the properties. We derive high probability error bounds on AFA's estimates, along with the worst-case lower bounds on the sample complexity to audit them. Numerically we demonstrate on multiple datasets and models that AFA is more accurate and sample-efficient to estimate the properties of interest than the baselines.

The heterogeneity of neurological and mental health disorders has been a key confound to disease understanding, treatment development and outcome prediction, as patient populations are thought to include multiple disease pathways that selectively respond to treatment (Kapur et al., 2012). These challenges are reflected in poor treatment outcomes; for instance, in depression, approximately only 40% of patients remit after first-line antidepressant treatment or psychotherapy (Amick et al., 2015; Cuijpers et al., 2014; Fava and Davidson, 1996; Trivedi et al., 2006). Diagnostic categories in psychiatry have historically been defined based on signs and symptoms, prioritising diagnostic agreement between clinicians, rather than underlying biological mechanisms (Freedman et al., 2013; Robins and Guze, 1970). Resultingly, the usefulness of supervised machine learning methods as diagnostic tools for mental health disorders (i.e., classifying patients vs. healthy controls) is questionable, as they may simply inherit the flaws of current diagnostic categories. Additional challenges in neurological and mental health disorders are comorbidity (i.e., individuals with one disorder often develop another disorder during their lifespan) and that different disorders can share similar symptoms (Kessler et al., 2005). To address the limitations of current diagnostic categories in psychiatry, the National Institute of Mental Health launched the Research Domain Criteria framework (RDoC) in 2009 (https://www.nimh.nih.gov/research/ 2 research-funded-by-nimh/rdoc) as an attempt to move beyond diagnostic categories and ground psychiatry within neurobiological constructs that combine multiple levels of measures or sources of information (Insel et al., 2010). Multivariate methods, such as Canonical Correlation Analysis (CCA) and related methods, that do not rely on the diagnostic categories, have been widely used to uncover latent disease dimensions capturing associations between brain imaging and non-imaging data (e.g., self-report questionnaires, cognitive tests and genetics). The identified latent dimensions provide information on how a set of non-imaging features (e.g.

The problem of adversarial robustness has been studied extensively for neural networks. However, for boosted decision trees and decision stumps there are almost no results, even though they are widely used in practice (e.g. We show in this paper that for boosted decision stumps the \textit{exact} min-max robust loss and test error for an l_\infty -attack can be computed in O(T\log T) time per input, where T is the number of decision stumps and the optimal update step of the ensemble can be done in O(n 2\,T\log T), where n is the number of data points. For boosted trees we show how to efficiently calculate and optimize an upper bound on the robust loss, which leads to state-of-the-art robust test error for boosted trees on MNIST (12.5\% for \epsilon_\infty 0.3), FMNIST (23.2\% for \epsilon_\infty 0.1), and CIFAR-10 (74.7\% for \epsilon_\infty 8/255). Moreover, the robust test error rates we achieve are competitive to the ones of provably robust convolutional networks.

We study the black-box attacks on graph neural networks (GNNs) under a novel and realistic constraint: attackers have access to only a subset of nodes in the network, and they can only attack a small number of them. A node selection step is essential under this setup. We demonstrate that the structural inductive biases of GNN models can be an effective source for this type of attacks. Specifically, by exploiting the connection between the backward propagation of GNNs and random walks, we show that the common gradient-based white-box attacks can be generalized to the black-box setting via the connection between the gradient and an importance score similar to PageRank. In practice, we find attacks based on this importance score indeed increase the classification loss by a large margin, but they fail to significantly increase the mis-classification rate. Our theoretical and empirical analyses suggest that there is a discrepancy between the loss and mis-classification rate, as the latter presents a diminishing-return pattern when the number of attacked nodes increases.

Kamala Harris' campaign is livestreaming a Tim Walz rally from Arizona and Azeroth tonight. This will be the first time the Harris campaign will stream live gameplay on its Twitch account. The event, which will take place at 6:30 pm ET, will include a live feed of Walz's Arizona speech with Preheat, a Twitch creator, playing the game and providing commentary. Preheat is expected to highlight their connection to the campaign and encourage viewers to make a plan to vote. The Harris campaign launched its Twitch account in August where it streamed the vice president's acceptance speech at the Democratic National Convention.

The United States Department of Justice's proposed remedies to break up Google's search dominance could weaken its main profit engine and stall its advances in artificial intelligence, even though a final outcome may be years away, analysts say. The Justice Department said on Tuesday it may ask a judge to force Google to divest parts of its business, such as its Chrome browser and Android operating system, that the Alphabet-owned company used to maintain an illegal monopoly in online search. It is only one of the many potential fixes prosecutors are considering. Barring Google from collecting sensitive user data, requiring it to make search results and indexes available to rivals, letting websites opt out of their content being used to train AI products and making Google report to a "court-appointed technical committee" are also on the table. The remedies strike at the heart of the internet empire that has made Google synonymous with search and can reduce its revenue while giving its rivals more room to grow.