Retrieval-augmented Large Language Models (LLMs) offer substantial benefits in enhancing performance across knowledge-intensive scenarios. However, these methods often face challenges with complex inputs and encounter difficulties due to noisy knowledge retrieval, notably hindering model effectiveness. To address this issue, we introduce BlendFilter, a novel approach that elevates retrieval-augmented LLMs by integrating query generation blending with knowledge filtering. BlendFilter proposes the blending process through its query generation method, which integrates both external and internal knowledge augmentation with the original query, ensuring comprehensive information gathering. Additionally, our distinctive knowledge filtering module capitalizes on the intrinsic capabilities of the LLM, effectively eliminating extraneous data. We conduct extensive experiments on three open-domain question answering benchmarks, and the findings clearly indicate that our innovative BlendFilter surpasses state-of-the-art baselines significantly.

Specifically, without knowing the future p -values, the analyst must irrevocably decide at each step whether to reject the null, such that with probability at least 1 α, there are no false rejections in the entire sequence. This paper unifies algorithm design concepts developed for offline FWER control and for online false discovery rate (FDR) control. Though Bonferroni, fallback procedures and Sidak's method can trivially be extended to the online setting, our main contribution is the design of new, adaptive online algorithms that control the FWER and per-family error rate (PFER) when the p -values are independent or locally dependent in time. Our experiments demonstrate substantial gains in power, also formally proved in an idealized Gaussian model. 1 Introduction Online multiple testing refers to the setting in which a potentially infinite stream of hypotheses H 1,H 2,... (respectively p -values P 1,P 2,...) is tested sequentially one at a time. At each step t N, one must decide whether to reject the current null hypothesis H t or not, without knowing the outcomes of all the future tests. Typically, we reject the null hypothesis when P t is smaller than some threshold α t. Let R represent the set of rejected null hypotheses, and H 0 be the unknown set of true null hypotheses; then, V R H 0 is the set of incorrectly rejected null hypotheses, also known as false discoveries. Denoting V V, some common error metrics are the false discovery rate (FDR), family wise error rate (FWER), per-family error rate (PFER) and power which are defined as FDR E null V R 1 null, FWER Pr{ V 1}, PFER E [V ], power E null H c 0 R H c 0 null .

Major internet companies routinely perform tens of thousands of A/B tests each year. Such large-scale sequential experimentation has resulted in a recent spurt of new algorithms that can provably control the false discovery rate (FDR) in a fully online fashion. However, current state-of-the-art adaptive algorithms can suffer from a significant loss in power if null p-values are conservative (stochastically larger than the uniform distribution), a situation that occurs frequently in practice. In this work, we introduce a new adaptive discarding method called ADDIS that provably controls the FDR and achieves the best of both worlds: it enjoys appreciable power increase over all existing methods if nulls are conservative (the practical case), and rarely loses power if nulls are exactly uniformly distributed (the ideal case). We provide several practical insights on robust choices of tuning parameters, and extend the idea to asynchronous and offline settings as well.