retrieve
Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?
Retrieval-Augmented Generation (RAG) based on knowledge graphs (KGs) enhances large language models (LLMs) with structural and textual external knowledge. Yet, existing KG-based RAG methods struggle to retrieve accurate and diverse information when handling complex queries. By modeling KG-based retrieval as a multi-step decision process, Process Reward Models (PRMs) offer a promising solution to align the retrieval behavior with the query-specific knowledge requirements. However, PRMs heavily rely on process-level supervision signals that are expensive and hard to obtain on KGs. To address this challenge, we propose GraphFlow, a framework that efficiently retrieves accurate and diverse knowledge required for complex queries from text-rich KGs. GraphFlow employs a detailed balance objective with local exploration to jointly optimize a retrieval policy and a flow estimator.
Memory-Integrated Reconfigurable Adapters: A Unified Framework for Settings with Multiple Tasks
Organisms constantly pivot between tasks such as evading predators, foraging, traversing rugged terrain, and socializing, often within milliseconds. Remarkably, they preserve knowledge of once-learned environments sans catastrophic forgetting, a phenomenon neuroscientists hypothesize, is due to a singular neural circuitry dynamically overlayed by neuromodulatory agents such as dopamine and acetylcholine. In parallel, deep learning research addresses analogous challenges via domain generalization ($\textbf{DG}$) and continual learning ($\textbf{CL}$), yet these methods remain siloed, despite the brain's ability to perform them seamlessly. In particular, prior work has not explored architectures involving associative memories ($\textbf{AM}$s), which are an integral part of biological systems, to jointly address these tasks. We propose Memory-Integrated Reconfigurable Adapters ($\textbf{MIRA}$), a unified framework that integrates Hopfield-style associative memory modules atop a shared backbone. These memory modules store adapter-weight updates as values and retrieve them via learned keys. Associative memory keys are learned post-hoc to index and retrieve an affine combination of stored adapter updates for any given task or domain on a per-sample basis. By varying only the task-specific objectives, we demonstrate that $\textbf{MIRA}$ seamlessly accommodates domain shifts and sequential task exposures under one roof. Empirical evaluations on standard benchmarks confirm that our $\textbf{AM}$-augmented architecture significantly enhances adaptability and retention: in $\textbf{DG}$, $\textbf{MIRA}$ achieves SoTA out-of-distribution accuracy, and in incremental learning settings, it outperforms architectures explicitly designed to handle catastrophic forgetting using generic $\textbf{CL}$ algorithms.
completion
Algorithm 2 describes the prompt completion algorithm introduced in Section 2.2. It implicitly401 considers a single action, which takes the next sequence element.402 Algorithm 2 - Prompt completion Input: Grounded schema {T,C,Erb}with rebound CSCG emission matrix Erb, delimiter token x, prompt x(prompt) = (x1,...,xm) Output: A completed prompt x(prompt completed) = (x1,...,xm,xm+1,...,xm+p = x) 1: Run max-product for MAP inference and return zMAP = (z1,...,zm) = argmaxz p(z|x(prompt)). Algorithm 3 is a variant of the rebinding Algorithm 1 that does not use EM. Instead, it first searches404 for "surprising observations": a surprise has a low probability of being emitted by its decoded clone.405
derivation of Eqs . 3 and 5
A.1 Derivation of Eq. (3) By expanding Eq. (2) with the definition of ฮตli,t = xli,t ยตli,t, we have: Et = We note that each xli,t influences Et in two ways: (i) it occurs in Eq. (6) explicitly, but (ii) it also determines the values of ยตl 1k,t via Eq. Considering also the special cases of l = Land l = 0, we obtain Eq. (3). We note that ฮธl+1i,j affects the value of the function Et of Eq. (6) by influencing ยตli,t via Eq. Here, we provide further details about training PCNs, useful to reproduce them. Furthermore, we have applied a decay factor of 0.9 to ฮณ, applied each time the energy failed to decrease.
Retrieve, Reason,andRefine: AppendixofGenerating AccurateandFaithfulPatientInstructions
For the constructed knowledge graph, we use randomly initialized embeddingsH(0) = {v1,v2,...,vNKG} RNKG d to represent all node features. Table 2shows that all variants with different number ofretrieved instructionsNP can consistently outperform the baseline model, which proves the effectiveness of our approach in retrieving the working experience to boost the Patient Instruction generation. Asaresult, givenanewmale/female patient at61years old,wewillmatchmale/female patients in the age-group 55 <= Age < 70 in the training data to generate the PIs.
RETRIEVE: Coreset Selection for Efficient and Robust Semi-Supervised Learning
Semi-supervised learning (SSL) algorithms have had great success in recent years in limited labeled data regimes. However, the current state-of-the-art SSL algorithms are computationally expensive and entail significant compute time and energy requirements. This can prove to be a huge limitation for many smaller companies and academic groups. Our main insight is that training on a subset of unlabeled data instead of entire unlabeled data enables the current SSL algorithms to converge faster, significantly reducing computational costs. In this work, we propose RETRIEVE, a coreset selection framework for efficient and robust semi-supervised learning. RETRIEVE selects the coreset by solving a mixed discrete-continuous bi-level optimization problem such that the selected coreset minimizes the labeled set loss. We use a one-step gradient approximation and show that the discrete optimization problem is approximately submodular, enabling simple greedy algorithms to obtain the coreset. We empirically demonstrate on several real-world datasets that existing SSL algorithms like VAT, Mean-Teacher, FixMatch, when used with RETRIEVE, achieve a) faster training times, b) better performance when unlabeled data consists of Out-of-Distribution (OOD) data and imbalance. More specifically, we show that with minimal accuracy degradation, RETRIEVE achieves a speedup of around $3\times$ in the traditional SSL setting and achieves a speedup of $5\times$ compared to state-of-the-art (SOTA) robust SSL algorithms in the case of imbalance and OOD data.