Goto

Collaborating Authors

 Africa


WSM: Decay-Free Learning Rate Schedule via Checkpoint Merging for LLM Pre-training

arXiv.org Artificial Intelligence

Recent advances in learning rate (LR) scheduling have demonstrated the effectiveness of decay-free approaches that eliminate the traditional decay phase while maintaining competitive performance. Model merging techniques have emerged as particularly promising solutions in this domain. We present Warmup-Stable and Merge (WSM), a general framework that establishes a formal connection between learning rate decay and model merging. WSM provides a unified theoretical foundation for emulating various decay strategies-including cosine decay, linear decay and inverse square root decay-as principled model averaging schemes, while remaining fully compatible with diverse optimization methods. Through extensive experiments, we identify merge duration-the training window for checkpoint aggregation-as the most critical factor influencing model performance, surpassing the importance of both checkpoint interval and merge quantity. Our framework consistently outperforms the widely-adopted Warmup-Stable-Decay (WSD) approach across multiple benchmarks, achieving significant improvements of +3.5% on MATH, +2.9% on HumanEval, and +5.5% on MMLU-Pro. The performance advantages extend to supervised fine-tuning scenarios, highlighting WSM's potential for long-term model refinement.


Towards Human-AI Collaboration System for the Detection of Invasive Ductal Carcinoma in Histopathology Images

arXiv.org Artificial Intelligence

Invasive ductal carcinoma (IDC) is the most prevalent form of breast cancer, and early, accurate diagnosis is critical to improving patient survival rates by guiding treatment decisions. Combining medical expertise with artificial intelligence (AI) holds significant promise for enhancing the precision and efficiency of IDC detection. In this work, we propose a human-in-the-loop (HITL) deep learning system designed to detect IDC in histopathology images. The system begins with an initial diagnosis provided by a high-performance EfficientNetV2S model, offering feedback from AI to the human expert. Medical professionals then review the AI-generated results, correct any misclassified images, and integrate the revised labels into the training dataset, forming a feedback loop from the human back to the AI. This iterative process refines the model's performance over time. The EfficientNetV2S model itself achieves state-of-the-art performance compared to existing methods in the literature, with an overall accuracy of 93.65\%. Incorporating the human-in-the-loop system further improves the model's accuracy using four experimental groups with misclassified images. These results demonstrate the potential of this collaborative approach to enhance AI performance in diagnostic systems. This work contributes to advancing automated, efficient, and highly accurate methods for IDC detection through human-AI collaboration, offering a promising direction for future AI-assisted medical diagnostics.


Testing the Limits of Machine Translation from One Book

arXiv.org Artificial Intelligence

Current state-of-the-art models demonstrate capacity to leverage in-context learning to translate into previously unseen language contexts. Tanzer et al. [2024] utilize language materials (e.g. a grammar) to improve translation quality for Kalamang using large language models (LLMs). We focus on Kanuri, a language that, despite having substantial speaker population, has minimal digital resources. We design two datasets for evaluation: one focused on health and humanitarian terms, and another containing generalized terminology, investigating how domain-specific tasks impact LLM translation quality. By providing different combinations of language resources (grammar, dictionary, and parallel sentences), we measure LLM translation effectiveness, comparing results to native speaker translations and human linguist performance. We evaluate using both automatic metrics and native speaker assessments of fluency and accuracy. Results demonstrate that parallel sentences remain the most effective data source, outperforming other methods in human evaluations and automatic metrics. While incorporating grammar improves over zero-shot translation, it fails as an effective standalone data source. Human evaluations reveal that LLMs achieve accuracy (meaning) more effectively than fluency (grammaticality). These findings suggest LLM translation evaluation benefits from multidimensional assessment beyond simple accuracy metrics, and that grammar alone, without parallel sentences, does not provide sufficient context for effective domain-specific translation.


In 'Alien: Earth', the Future Is a Corporate Hellscape

WIRED

Seventeen years ago, Noah Hawley became a father during the Great Recession. If you look at everything he's written since having children--including the TV series Fargo and Legion--Hawley says it all revolves around the same question every parent faces: "How are we supposed to raise these people in the world that we're living in?" Hawley's new series, Alien: Earth, which premieres August 12 on Hulu and FX, explores this question even more directly than his previous work. Set two years before the original Alien in 2120, it imagines a future where the race for immortality has led to three competing technologies: synths (AI minds in synthetic bodies), cyborgs (humans with cybernetic enhancements), and hybrids (human minds downloaded into synthetic bodies). When a deep space research vessel, the USCSS Maginot, crashes into Earth carrying five captured alien species, a megacorporation called Prodigy sends six hybrids to investigate. The first-ever hybrid, Wendy, played by Sydney Chandler, was a terminally ill child before she was selected for the immortality experiment, just like the rest of Prodigy's hybrids, all six of whom wake up in super-strong, super-fast, synthetic adult bodies that will never age.


Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs

arXiv.org Artificial Intelligence

--This paper presents two effective approaches for Extractive Question Answering (QA) on the Qur'an. It addresses challenges related to complex language, unique terminology, and deep meaning in the text. The second uses few-shot prompting with instruction-tuned large language models such as Gemini and DeepSeek. A specialized Arabic prompt framework is developed for span extraction. A strong post-processing system integrates subword alignment, overlap suppression, and semantic filtering. This improves precision and reduces hallucinations. Evaluations show that large language models with Arabic instructions outperform traditional fine-tuned models. The best configuration achieves a pAP@10 score of 0.637. The results confirm that prompt-based instruction tuning is effective for low-resource, semantically rich QA tasks.



Teenager who lost his legs in crash will 'never forgive' driver

BBC News

Teenager who lost his legs in crash will'never forgive' driver 38 minutes agoShareSaveKen Banks and Louise HosieBBC Scotland NewsShareSaveBBC Adam Golebiewski had a double amputation after the crash last year A teenager who lost his lower legs in a crash says he "will never forgive" the drink-driver at the wheel. Young footballer Adam Golebiewski, 18, had been a passenger in Arran Paterson's car in Macduff, Aberdeenshire, in September last year. Paterson, 19, admitted dangerous driving, being over the drink-drive limit and driving without insurance at Aberdeen Sheriff Court. Adam walked into court unaided on prosthetic legs following intensive rehabilitation. He said: "I want to try to enjoy life again and stay positive."


Former Google executive issues bleak warning for next '15 years of dystopia' - and it won't be because of AI

Daily Mail - Science & tech

A terrifying societal collapse worthy of Hollywood can never be entirely ruled out. But according to one former Google executive, it may come a lot sooner than we expected. Mo Gawdat, a tech entrepreneur and author who spent 11 years at Google, has given a bleak warning about the near-future of society. Speaking with The Diary of a CEO podcast, Mr Gawdat said we'll be living in a dystopia in just two years' time. Sounding worthy of George Orwell's novel '1984', the dystopia will last up to 15 years, the expert said.


Anti-drone system boosts Greece's ambitious plans for defense drone industry

The Japan Times

It took just minutes for a new Greek-made anti-drone system to show what it is capable of. On its first test run with a European Union patrol in the Red Sea a year ago, the Centauros system detected and swiftly brought down two aerial drones launched by Yemen's Houthis, who have been attacking merchant vessels in the busy shipping lane. Another two drones swiftly retreated: Centauros had jammed their electronics, said Kyriakos Enotiadis, electronics director at state-run Hellenic Aerospace Industry (HAI), which produces the anti-drone system.


Bridging Robustness and Generalization Against Word Substitution Attacks in NLP via the Growth Bound Matrix Approach

arXiv.org Artificial Intelligence

Despite advancements in Natural Language Processing (NLP), models remain vulnerable to adversarial attacks, such as synonym substitutions. While prior work has focused on improving robustness for feed-forward and convolutional architectures, the robustness of recurrent networks and modern state space models (SSMs), such as S4, remains understudied. These architectures pose unique challenges due to their sequential processing and complex parameter dynamics. In this paper, we introduce a novel regularization technique based on Growth Bound Matrices (GBM) to improve NLP model robustness by reducing the impact of input perturbations on model outputs. We focus on computing the GBM for three architectures: Long Short-Term Memory (LSTM), State Space models (S4), and Convolutional Neural Networks (CNN). Our method aims to (1) enhance resilience against word substitution attacks, (2) improve generalization on clean text, and (3) providing the first systematic analysis of SSM (S4) robustness. Extensive experiments across multiple architectures and benchmark datasets demonstrate that our method improves adversarial robustness by up to 8.8% over existing baselines. These results highlight the effectiveness of our approach, outperforming several state-of-the-art methods in adversarial defense. Codes are available at https://github.com/BouriMohammed/GBM