Goto

Collaborating Authors

 Africa



Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization James Oldfield

Neural Information Processing Systems

An important corollary of successful task decomposition amongst experts is that layers are easier to debug and edit. Biased or unsafe behaviors can be better localized to specific experts' subcomputation, facilitating manual correction or surgery in a way that minimally affects the other functionality of the network. Addressing such behaviors is particularly crucial in the context of foundation models; being often fine-tuned as black boxes pre-trained on unknown, potentially imbalanced data distributions.


Eusebio: Between colonialism and football

Al Jazeera

Game Theory: What connects Eusebio, football, and colonialism? It's been 60 years since Eusebio lifted the Ballon d'Or, becoming the first Black player to win football's highest individual honour. But beyond the brilliance, his journey was shaped by colonial forces far bigger than football. Samantha Johnson explores the side to Eusebio's legacy that many may have missed out. Why does Israel play in European Football? What's behind bans on away fans?


EU moves to ease AI, privacy rules amid pressure from Big Tech, Trump

Al Jazeera

The reforms, which amend the AI Act and several other privacy and tech-related laws, would also cut back on website pop-ups asking permission to use cookies and reduce documentation requirements for small and medium-sized businesses. EU tech chief Henna Virkkunen said the changes, which need to be approved by representatives of the 27 EU member states, would boost European competitiveness by simplifying rules about AI, cybersecurity and data protection. "We have talent, infrastructure, a large internal single market. But our companies, especially our start-ups and small businesses, are often held back by layers of rigid rules," Virkkunen said. Lobby groups for tech giants in the United States, where President Donald Trump's administration has been a vocal critic of Europe's regulatory approach, welcomed the move, while lamenting that the measures did not go far enough.




A Dataset for Analyzing Streaming Media Performance over HTTP/3 Browsers

Neural Information Processing Systems

HTTP/3 is a new application layer protocol supported by most browsers. It uses QUIC as an underlying transport protocol. QUIC provides multiple benefits, like faster connection establishment, reduced latency, and improved connection migration.


FIND: A Function Description Benchmark for Evaluating Interpretability Methods Sarah Schwettmann

Neural Information Processing Systems

The central task of interpretability research is to explain the functions that AI systems learn from data. Investigating these functions requires experimentation with trained models, using tools that incorporate varying degrees of human input. Hand-tooled approaches that rely on close manual inspection [Zeiler and Fergus, 2014, Zhou et al., 2014, Mahendran and V edaldi, 2015, Olah et al., 2017, 2020, Elhage et al., 2021] or search for predefined phenomena [Wang et al., 2022, Nanda


Weather Maps as Tokens: Transformers for Renewable Energy Forecasting

arXiv.org Artificial Intelligence

Accurate renewable energy forecasting is essential to reduce dependence on fossil fuels and enabling grid decarbonization. However, current approaches fail to effectively integrate the rich spatial context of weather patterns with their temporal evolution. This work introduces a novel approach that treats weather maps as tokens in transformer sequences to predict renewable energy. Hourly weather maps are encoded as spatial tokens using a lightweight convolutional neural network, and then processed by a transformer to capture temporal dynamics across a 45-hour forecast horizon. Despite disadvantages in input initialization, evaluation against ENTSO-E operational forecasts shows a reduction in RMSE of about 60% and 20% for wind and solar respectively. A live dashboard showing daily forecasts is available at: https://www.sardiniaforecast.ifabfoundation.it.


The Learning Dynamics of Subword Segmentation for Morphologically Diverse Languages

arXiv.org Artificial Intelligence

Subword segmentation is typically applied in preprocessing and stays fixed during training. Alternatively, it can be learned during training to optimise the training objective. In this paper we study the learning dynamics of subword segmentation: if a language model can dynamically optimise tokenisation, how do its subwords evolve during pretraining and finetuning? To explore this, we extend the subword segmental language model (SSLM), a framework for learning subwords during training, to support pretraining and finetuning. We train models for three typologically diverse languages to study learning dynamics across the morphological spectrum: Isi-Xhosa is conjunctive (long word forms composed of many morphemes), Setswana is disjunctive (morphemes written as separate words), and English represents a typological middle ground. We analyse subword dynamics from a linguistic perspective, tracking morphology, productivity, and fertility. We identify four stages of subword learning, with the morphologically complex isi-Xhosa exhibiting greater instability. During finetuning, subword boundaries shift to become finer-grained. Lastly, we show that learnable subwords offers a promising approach to improve text generation and cross-lingual transfer for low-resource, morphologically complex languages.