Modern systems for multi-hop question answering (QA) typically break questions into a sequence of reasoning steps, termed chain-of-thought (CoT), before arriving at a final answer. Often, multiple chains are sampled and aggregated through a voting mechanism over the final answers, but the intermediate steps themselves are discarded. While such approaches improve performance, they do not consider the relations between intermediate steps across chains and do not provide a unified explanation for the predicted answer. We introduce Multi-Chain Reasoning (MCR), an approach which prompts large language models to meta-reason over multiple chains of thought, rather than aggregating their answers. MCR examines different reasoning chains, mixes information between them and selects the most relevant facts in generating an explanation and predicting the answer. MCR outperforms strong baselines on 7 multi-hop QA datasets. Moreover, our analysis reveals that MCR explanations exhibit high quality, enabling humans to verify its answers.

We are seeking to characterize the performance and potential bottlenecks of the latest fast MCMC samplers. I see that Stan is currently using Intel TBB to parallelize the no-U-turn sampler (NUTS) across multiple chains. Do you know of any research attempted to parallelize each sampler itself within one chain. Our group at Google has been very interested in using parallel compute in HMC variants (including NUTS), particularly on accelerators (e.g., GPUs). We've been working in the deep-learning-oriented autodiff accelerator software frameworks TensorFlow and JAX, both of which are supported by our TensorFlow Probability library.