Africa
AI's Blind Spots: Geographic Knowledge and Diversity Deficit in Generated Urban Scenario
Beneduce, Ciro, Luca, Massimiliano, Lepri, Bruno
Image generation models are revolutionizing many domains, and urban analysis and design is no exception. While such models are widely adopted, there is a limited literature exploring their geographic knowledge, along with the biases they embed. In this work, we generated 150 synthetic images for each state in the USA and related capitals using FLUX 1 and Stable Diffusion 3.5, two state-of-the-art models for image generation. We embed each image using DINO-v2 ViT-S/14 and the Fréchet Inception Distances to measure the similarity between the generated images. We found that while these models have implicitly learned aspects of USA geography, if we prompt the models to generate an image for "United States" instead of specific cities or states, the models exhibit a strong representative bias toward metropolis-like areas, excluding rural states and smaller cities. {\color{black} In addition, we found that models systematically exhibit some entity-disambiguation issues with European-sounding names like Frankfort or Devon.
Critical Appraisal of Fairness Metrics in Clinical Predictive AI
Matos, João, Van Calster, Ben, Celi, Leo Anthony, Dhiman, Paula, Gichoya, Judy Wawira, Riley, Richard D., Russell, Chris, Khalid, Sara, Collins, Gary S.
Predictive artificial intelligence (AI) offers an opportunity to improve clinical practice and patient outcomes, but risks perpetuating biases if fairness is inadequately addressed. However, the definition of "fairness" remains unclear. We conducted a scoping review to identify and critically appraise fairness metrics for clinical predictive AI. We defined a "fairness metric" as a measure quantifying whether a model discriminates (societally) against individuals or groups defined by sensitive attributes. We searched five databases (2014-2024), screening 820 records, to include 41 studies, and extracted 62 fairness metrics. Metrics were classified by performance-dependency, model output level, and base performance metric, revealing a fragmented landscape with limited clinical validation and overreliance on threshold-dependent measures. Eighteen metrics were explicitly developed for healthcare, including only one clinical utility metric. Our findings highlight conceptual challenges in defining and quantifying fairness and identify gaps in uncertainty quantification, intersectionality, and real-world applicability.
Identifiability of Deep Polynomial Neural Networks
Usevich, Konstantin, Dérand, Clara, Borsoi, Ricardo, Clausel, Marianne
Polynomial Neural Networks (PNNs) possess a rich algebraic and geometric structure. However, their identifiability -- a key property for ensuring interpretability -- remains poorly understood. In this work, we present a comprehensive analysis of the identifiability of deep PNNs, including architectures with and without bias terms. Our results reveal an intricate interplay between activation degrees and layer widths in achieving identifiability. As special cases, we show that architectures with non-increasing layer widths are generically identifiable under mild conditions, while encoder-decoder networks are identifiable when the decoder widths do not grow too rapidly. Our proofs are constructive and center on a connection between deep PNNs and low-rank tensor decompositions, and Kruskal-type uniqueness theorems. This yields both generic conditions determined by the architecture, and effective conditions that depend on the network's parameters. We also settle an open conjecture on the expected dimension of PNN's neurovarieties, and provide new bounds on the activation degrees required for it to reach its maximum.
Performative Validity of Recourse Explanations
König, Gunnar, Fokkema, Hidde, Freiesleben, Timo, Mendler-Dünner, Celestine, von Luxburg, Ulrike
When applicants get rejected by an algorithmic decision system, recourse explanations provide actionable suggestions for how to change their input features to get a positive evaluation. A crucial yet overlooked phenomenon is that recourse explanations are performative: When many applicants act according to their recommendations, their collective behavior may change statistical regularities in the data and, once the model is refitted, also the decision boundary. Consequently, the recourse algorithm may render its own recommendations invalid, such that applicants who make the effort of implementing their recommendations may be rejected again when they reapply. In this work, we formally characterize the conditions under which recourse explanations remain valid under performativity. A key finding is that recourse actions may become invalid if they are influenced by or if they intervene on non-causal variables. Based on our analysis, we caution against the use of standard counterfactual explanations and causal recourse methods, and instead advocate for recourse methods that recommend actions exclusively on causal variables.
Refined Causal Graph Structure Learning via Curvature for Brain Disease Classification
Febrinanto, Falih Gozi, Simango, Adonia, Xu, Chengpei, Zhou, Jingjing, Ma, Jiangang, Tyagi, Sonika, Xia, Feng
The field of neuroscience has been revolutionized by the advent of brain imaging technologies, particularly functional magnetic resonance imaging in the resting state (rest fMRI) (Khalilullah et al, 2023; Vasilkovska et al, 2023; Liu et al, 2024). This powerful tool allows the measurement of blood-oxygen-level-dependent (BOLD) signals in predefined Regions of Interest (ROIs) within the brain, offering an unprecedented avenue for revealing information about potential diseases such as autism spectrum disorder (ASD) and schizophrenia (Philiastides et al, 2021; Kocak, 2021). Various brain atlases, including Harvard-Oxford (Makris et al, 2006) and Craddock 200 (Craddock et al, 2012) parcellations, have been used to define these ROIs. Furthermore, ROIs can be interestingly modelled as graph data, where the ROIs themselves represent nodes, and the connections between ROIs represent edges of graphs (Cui et al, 2022b). This graph-based data structure, inheriting the graph theory technique, has been instrumental in revealing meaningful relationships between ROIs in brain networks to diagnose brain diseases more effectively (Alsubaie et al, 2024; Ren and Xia, 2024). With the current popularity of deep learning, recent frameworks have developed graph neural networks (GNNs) (Xia et al, 2021; Febrinanto et al, 2023c) to extend the merits of modelling graph-structured data for detecting brain diseases with brain networks based on fMRI signals as input (Kan et al, 2022b; Li et al, 2021; Kan et al, 2022a; Cui et al, 2022a; ElGazzar et al, 2022; Febrinanto et al, 2023a). These techniques perform more accurately than typical machine learning or deep learning techniques. However, there is still a high consensus on how to construct or define an appropriate graph structure in brain networks in terms of two processes: 1) how do we generate the graphs?
Development of a Multiprocessing Interface Genetic Algorithm for Optimising a Multilayer Perceptron for Disease Prediction
Iliyas, Iliyas Ibrahim, Boukari, Souley, Gital, Abdulsalam Yau
This study introduces a framework that integrates nonlinear feature extraction, classification, and efficient optimization. First, kernel principal component analysis with a radial basis function kernel reduces dimensionality while preserving 95% of the variance. Second, a multilayer perceptron (MLP) learns to predict disease status. Finally, a modified multiprocessing genetic algorithm (MIGA) optimizes MLP hyperparameters in parallel over ten generations. We evaluated this approach on three datasets: the Wisconsin Diagnostic Breast Cancer dataset, the Parkinson's Telemonitoring dataset, and the chronic kidney disease dataset. The MLP tuned by the MIGA achieved the best accuracy of 99.12% for breast cancer, 94.87% for Parkinson's disease, and 100% for chronic kidney disease. These results outperform those of other methods, such as grid search, random search, and Bayesian optimization. Compared with a standard genetic algorithm, kernel PCA revealed nonlinear relationships that improved classification, and the MIGA's parallel fitness evaluations reduced the tuning time by approximately 60%. The genetic algorithm incurs high computational cost from sequential fitness evaluations, but our multiprocessing interface GA (MIGA) parallelizes this step, slashing the tuning time and steering the MLP toward the best accuracy score of 99.12%, 94.87%, and 100% for breast cancer, Parkinson's disease, and CKD, respectively.
Life on Mars: Humans will live in huge 'space oases' on the Red Planet in just 15 years, European Space Agency predicts
Imagine a future where humans live in huge'space oases' on Mars – luxury indoor habitats made of heat-reflective material that grow their own food. Robots are sent into the vast Martian wilderness, where they explore without the risk of exhaustion, radiation poisoning or dust contamination. Enormous space stations and satellites are manufactured in orbit, AI is trusted to make critical decisions, and the whole solar system is connected by a vast internet network. While this sounds like science-fiction, the European Space Agency (ESA) hopes it will become a reality in just 15 years. In a new report, the agency – which represents more than 20 countries including the UK – outlines an ambitious vision for space exploration by 2040.
ViLLa: A Neuro-Symbolic approach for Animal Monitoring
Monitoring animal populations in natural environments requires systems that can interpret both visual data and human language queries. This work introduces ViLLa (Vision-Language-Logic Approach), a neuro-symbolic framework designed for interpretable animal monitoring. ViLLa integrates three core components: a visual detection module for identifying animals and their spatial locations in images, a language parser for understanding natural language queries, and a symbolic reasoning layer that applies logic-based inference to answer those queries. Given an image and a question such as "How many dogs are in the scene?" or "Where is the buffalo?", the system grounds visual detections into symbolic facts and uses predefined rules to compute accurate answers related to count, presence, and location. Unlike end-to-end black-box models, ViLLa separates perception, understanding, and reasoning, offering modularity and transparency. The system was evaluated on a range of animal imagery tasks and demonstrates the ability to bridge visual content with structured, human-interpretable queries.
The Hardness of Achieving Impact in AI for Social Impact Research: A Ground-Level View of Challenges & Opportunities
Majumdar, Aditya, Zhang, Wenbo, Prawal, Kashvi, Yadav, Amulya
In an attempt to tackle the UN SDGs, AI for Social Impact (AI4SI) projects focus on harnessing AI to address societal issues in areas such as healthcare, social justice, etc. Unfortunately, despite growing interest in AI4SI, achieving tangible, on-the-ground impact remains a significant challenge. For example, identifying and engaging motivated collaborators who are willing to co-design and deploy AI based solutions in real-world settings is often difficult. Even when such partnerships are established, many AI4SI projects "fail" to progress beyond the proof-of-concept stage, and hence, are unable to transition to at-scale production-level solutions. Furthermore, the unique challenges faced by AI4SI researchers are not always fully recognized within the broader AI community, where such work is sometimes viewed as primarily applied and not aligning with the traditional criteria for novelty emphasized in core AI venues. This paper attempts to shine a light on the diverse challenges faced in AI4SI research by diagnosing a multitude of factors that prevent AI4SI partnerships from achieving real-world impact on the ground. Drawing on semi-structured interviews with six leading AI4SI researchers - complemented by the authors' own lived experiences in conducting AI4SI research - this paper attempts to understand the day-to-day difficulties faced in developing and deploying socially impactful AI solutions. Through thematic analysis, we identify structural and organizational, communication, collaboration, and operational challenges as key barriers to deployment. While there are no easy fixes, we synthesize best practices and actionable strategies drawn from these interviews and our own work in this space. In doing so, we hope this paper serves as a practical reference guide for AI4SI researchers and partner organizations seeking to engage more effectively in socially impactful AI collaborations.
Winter Soldier: Backdooring Language Models at Pre-Training with Indirect Data Poisoning
Bouaziz, Wassim, Videau, Mathurin, Usunier, Nicolas, El-Mhamdi, El-Mahdi
The pre-training of large language models (LLMs) relies on massive text datasets sourced from diverse and difficult-to-curate origins. Although membership inference attacks and hidden canaries have been explored to trace data usage, such methods rely on memorization of training data, which LM providers try to limit. In this work, we demonstrate that indirect data poisoning (where the targeted behavior is absent from training data) is not only feasible but also allow to effectively protect a dataset and trace its use. Using gradient-based optimization prompt-tuning, we make a model learn arbitrary secret sequences: secret responses to secret prompts that are absent from the training corpus. We validate our approach on language models pre-trained from scratch and show that less than 0.005% of poisoned tokens are sufficient to covertly make a LM learn a secret and detect it with extremely high confidence ($p < 10^{-55}$) with a theoretically certifiable scheme. Crucially, this occurs without performance degradation (on LM benchmarks) and despite secrets never appearing in the training set.