nora
When No Paths Lead to Rome: Benchmarking Systematic Neural Relational Reasoning
Das, Anirban, Khalid, Irtaza, Peñaloza, Rafael, Schockaert, Steven
Designing models that can learn to reason in a systematic way is an important and long-standing challenge. In recent years, a wide range of solutions have been proposed for the specific case of systematic relational reasoning, including Neuro-Symbolic approaches, variants of the Transformer architecture, and specialised Graph Neural Networks. However, existing benchmarks for systematic relational reasoning focus on an overly simplified setting, based on the assumption that reasoning can be reduced to composing relational paths. In fact, this assumption is hard-baked into the architecture of several recent models, leading to approaches that can perform well on existing benchmarks but are difficult to generalise to other settings. To support further progress in the field of systematic relational reasoning with neural networks, we introduce NoRA, a new benchmark which adds several levels of difficulty and requires models to go beyond path-based reasoning.
Don't Forget the Nonlinearity: Unlocking Activation Functions in Efficient Fine-Tuning
Yin, Bo, Yang, Xingyi, Wang, Xinchao
Existing parameter-efficient fine-tuning (PEFT) methods primarily adapt weight matrices while keeping activation functions fixed. We introduce \textbf{NoRA}, the first PEFT framework that directly adapts nonlinear activation functions in pretrained transformer-based models. NoRA replaces fixed activations with learnable rational functions and applies structured low-rank updates to numerator and denominator coefficients, with a group-wise design that localizes adaptation and improves stability at minimal cost. On vision transformers trained on CIFAR-10 and CIFAR-100, NoRA matches or exceeds full fine-tuning while updating only 0.4\% of parameters (0.02M), achieving accuracy gains of +0.17\% and +0.27\%. When combined with LoRA (\textbf{NoRA++}), it outperforms LoRA and DoRA under matched training budgets by adding fewer trainable parameters. On LLaMA3-8B instruction tuning, NoRA++ consistently improves generation quality, yielding average MMLU gains of +0.3\%--0.8\%, including +1.6\% on STEM (Alpaca) and +1.3\% on OpenOrca. We further show that NoRA constrains adaptation to a low-dimensional functional subspace, implicitly regularizing update magnitude and direction. These results establish activation-space tuning as a complementary and highly parameter-efficient alternative to weight-based PEFT, positioning activation functions as first-class objects for model adaptation.
NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Tasks
Hung, Chia-Yu, Sun, Qi, Hong, Pengfei, Zadeh, Amir, Li, Chuan, Tan, U-Xuan, Majumder, Navonil, Poria, Soujanya
Existing Visual-Language-Action (VLA) models have shown promising performance in zero-shot scenarios, demonstrating impressive task execution and reasoning capabilities. However, a significant challenge arises from the limitations of visual encoding, which can result in failures during tasks such as object grasping. Moreover, these models typically suffer from high computational overhead due to their large sizes, often exceeding 7B parameters. While these models excel in reasoning and task planning, the substantial computational overhead they incur makes them impractical for real-time robotic environments, where speed and efficiency are paramount. To address the limitations of existing VLA models, we propose NORA, a 3B-parameter model designed to reduce computational overhead while maintaining strong task performance. NORA adopts the Qwen-2.5-VL-3B multimodal model as its backbone, leveraging its superior visual-semantic understanding to enhance visual reasoning and action grounding. Additionally, our \model{} is trained on 970k real-world robot demonstrations and equipped with the FAST+ tokenizer for efficient action sequence generation. Experimental results demonstrate that NORA outperforms existing large-scale VLA models, achieving better task performance with significantly reduced computational overhead, making it a more practical solution for real-time robotic autonomy.
NoRA: Nested Low-Rank Adaptation for Efficient Fine-Tuning Large Models
Lin, Cheng, Li, Lujun, Li, Dezhi, Zou, Jie, Xue, Wei, Guo, Yike
In this paper, we introduce Nested Low-Rank Adaptation (NoRA), a novel approach to parameter-efficient fine-tuning that extends the capabilities of Low-Rank Adaptation (LoRA) techniques. Vanilla LoRA overlooks pre-trained weight inheritance and still requires fine-tuning numerous parameters. To addresses these issues, our NoRA adopts a dual-layer nested structure with Singular Value Decomposition (SVD), effectively leveraging original matrix knowledge while reducing tunable parameters. Specifically, NoRA freezes the outer LoRA weights and utilizes an inner LoRA design, providing enhanced control over model optimization. This approach allows the model to more precisely adapt to specific tasks while maintaining a compact parameter space. By freezing outer LoRA weights and using an inner LoRA design, NoRA enables precise task adaptation with a compact parameter space. Evaluations on tasks including commonsense reasoning with large language models, fine-tuning vision-language models, and subject-driven generation demonstrate NoRA's superiority over LoRA and its variants. Code will be released upon acceptance.
Fast Inference of Removal-Based Node Influence
Li, Weikai, Xiao, Zhiping, Luo, Xiao, Sun, Yizhou
Graph neural networks (GNNs) are widely utilized to capture the information spreading patterns in graphs. While remarkable performance has been achieved, there is a new trending topic of evaluating node influence. We propose a new method of evaluating node influence, which measures the prediction change of a trained GNN model caused by removing a node. A real-world application is, "In the task of predicting Twitter accounts' polarity, had a particular account been removed, how would others' polarity change?". We use the GNN as a surrogate model whose prediction could simulate the change of nodes or edges caused by node removal. Our target is to obtain the influence score for every node, and a straightforward way is to alternately remove every node and apply the trained GNN on the modified graph to generate new predictions. It is reliable but time-consuming, so we need an efficient method. The related lines of work, such as graph adversarial attack and counterfactual explanation, cannot directly satisfy our needs, since their problem settings are different. We propose an efficient, intuitive, and effective method, NOde-Removal-based fAst GNN inference (NORA), which uses the gradient information to approximate the node-removal influence. It only costs one forward propagation and one backpropagation to approximate the influence score for all nodes. Extensive experiments on six datasets and six GNN models verify the effectiveness of NORA. Our code is available at https://github.com/weikai-li/NORA.git.
Reasons, Values, Stakeholders: A Philosophical Framework for Explainable Artificial Intelligence
The societal and ethical implications of the use of opaque artificial intelligence systems for consequential decisions, such as welfare allocation and criminal justice, have generated a lively debate among multiple stakeholder groups, including computer scientists, ethicists, social scientists, policy makers, and end users. However, the lack of a common language or a multi-dimensional framework to appropriately bridge the technical, epistemic, and normative aspects of this debate prevents the discussion from being as productive as it could be. Drawing on the philosophical literature on the nature and value of explanations, this paper offers a multi-faceted framework that brings more conceptual precision to the present debate by (1) identifying the types of explanations that are most pertinent to artificial intelligence predictions, (2) recognizing the relevance and importance of social and ethical values for the evaluation of these explanations, and (3) demonstrating the importance of these explanations for incorporating a diversified approach to improving the design of truthful algorithmic ecosystems. The proposed philosophical framework thus lays the groundwork for establishing a pertinent connection between the technical and ethical aspects of artificial intelligence systems.
Celebrating Valentine's Day during a pandemic with 6 awesome apps
Whether you're looking for love or ways to celebrate your loved one, technology is playing an increasingly important role – especially during a pandemic. After all, many of us are forced to remain socially distant for the time being. Valentine's Day might be celebrated at home this year, as opposed to dining in a restaurant, and florists may sell more bouquets to online customers instead of in-store shoppers. As the expression goes, there's an app for that. Interestingly, even online dating apps aren't just used to find a mate over the internet, but quite literally to date online – until it's safe to meet in person.
'Upload' Is a Clunky Parable About Class in a Digital Afterlife
In 2033, the Gordita Crunch is sold virtually by fast food goliath Nokia Taco Bell. Mega-airline corporation Frontier Spirit United offers 30-minute flights from New York to Los Angeles with the option of Economy Minus. The most popular reality show is Baby Botox--which is exactly what it sounds like. Vape lung is a chronic disease. Far East Movement's 2010 chart-topper "Like a G6" is considered classical dance coursework in schools.
Amazon's 'Upload' explores the digital afterlife in a world gone to hell
Take Black Mirror's dystopian tech commentary, The Good Place's philosophical exploration of the after-life, and the workplace antics of The Office, mash them together, and you have Amazon's Upload. It takes place in a world that could easily be 10 years from now -- self driving cars are commonplace, the Earth is polluted and over-crowded, and, oh yeah, you can also achieve digital immortality by uploading your consciousness to the cloud. Upload, which premieres today, is an entirely new territory for Greg Daniels, the genius writer behind The Office, and Parks and Rec (not to mention a long run on The Simpsons). But it's a world that's clearly been percolating in his mind for years. It's bold and raunchy in a way a network sitcom never could be, and it defies being classified into a single genre.
em Upload /em Is Like em The Good Place /em if It Were More Interested in Class Struggle
What if the next life were no better than this one? Not a heaven or a hell, or even a purgatorial waiting room, but a world that operates according to the same rules as the one that came before it, only tweaked enough that we don't just accept them as the way things have to be. In the near future of Upload, whose first season begins streaming on Amazon Prime on Friday, death is not the end, at least for those with the resources to survive it. But the hereafter in Greg Daniels' series isn't spiritual, it's digital, and everything, including entry and your continued existence, comes at a cost. It's less like heaven than a cruise ship on an infinite voyage, one where everything is marked up because the dead aren't in much of a position to comparison-shop. Nathan Brown (Robbie Amell) finds his way to Lakeview, as his particularly plush digital forever is called, after his self-driving car rear-ends a garbage truck.