Conditional question answering (CQA) is an important task that aims to find probable answers and identify conditions that need to be satisfied to support the answer. Existing approaches struggle with CQA due to two main challenges: (1) precisely identifying conditions and their logical relationship, and (2) verifying and solving the conditions. To address these challenges, we propose Chain of Condition, a novel prompting approach by firstly identifying all conditions and constructing their logical relationships explicitly according to the document, then verifying whether these conditions are satisfied, finally solving the logical expression by tools to indicate any missing conditions and generating the answer based on the resolved conditions. The experiments on two benchmark conditional question answering datasets shows chain of condition outperforms existing prompting baselines, establishing a new state-of-the-art. Furthermore, with backbone models like GPT-3.5-Turbo or GPT-4, it surpasses all supervised baselines with only few-shot settings.

We introduce SHARCS for adaptive inference that takes into account the hardness of input samples. SHARCS can train a router on any transformer network, enabling the model to direct different samples to sub-networks with varying widths. Our experiments demonstrate that: (1) SHARCS outperforms or complements existing per-sample adaptive inference methods across various classification tasks in terms of accuracy vs. FLOPs; (2) SHARCS generalizes across different architectures and can be even applied to compressed and efficient transformer encoders to further improve their efficiency; (3) SHARCS can provide a 2 times inference speed up at an insignificant drop in accuracy.

Acknowledgments: The authors would like to acknowledge primary support from the National Science Foundation National Robotics Initiative which has made this research possible, additional support from NASA's PSTAR program, and in-kind support by the NOAA Ocean Exploration Cooperative Institute with ship and robotic vehicle operations during 2021 Pacific Ocean demonstrations in the San Pedro Basin. The authors would also like to thank the captain and crew of the R/V Nautilus, the NUI robotic vehicle operations team, and study participants who volunteered to assist with performance testing of the SHARC and conventional robotic manipulation systems. AP would like to acknowledge support from the National Science Foundation Graduate Research Fellowship under Grant No. 2141064 and the Link Foundation. Funding: National Science Foundation, National Robotics Initiative grant IIS-1830500 (RC) National Science Foundation, National Robotics Initiative grant IIS-1830660 (MW) National Aeronautics and Space Administration, Planetary Science and Technology from Analog Research grant NNX16AL08G (RC) Author contributions: Conceptualization: AFD, AP, GB, MRW, RC Methodology: AFD, AP, GB, MRW, RC Investigation: AFD, AP, GB, MRW, RC Visualization: AFD, AP, GB, RC Funding acquisition: MRW, RC Project administration: MRW, RC Supervision: MRW, RC Writing - original draft: AFD, AP, GB, MRW, RC Writing - review & editing: AFD, AP, GB, MRW, RC Competing interests: Authors declare that they have no competing interests. Data and materials availability: All data are available in the main text or the supplementary materials. NOTE: This is the author's version of the work. It is posted here by permission of the AAAS for personal use, not for redistribution. The definitive version was published in Science Robotics on 23 Aug 2023, DOI: 10.1126/scirobotics.adi5227.

Multimodal learning is an essential paradigm for addressing complex real-world problems, where individual data modalities are typically insufficient to accurately solve a given modelling task. While various deep learning approaches have successfully addressed these challenges, their reasoning process is often opaque; limiting the capabilities for a principled explainable cross-modal analysis and any domain-expert intervention. In this paper, we introduce SHARCS (SHARed Concept Space) -- a novel concept-based approach for explainable multimodal learning. SHARCS learns and maps interpretable concepts from different heterogeneous modalities into a single unified concept-manifold, which leads to an intuitive projection of semantically similar cross-modal concepts. We demonstrate that such an approach can lead to inherently explainable task predictions while also improving downstream predictive performance. Moreover, we show that SHARCS can operate and significantly outperform other approaches in practically significant scenarios, such as retrieval of missing modalities and cross-modal explanations. Our approach is model-agnostic and easily applicable to different types (and number) of modalities, thus advancing the development of effective, interpretable, and trustworthy multimodal approaches.

Document interpretation and dialog understanding are the two major challenges for conversational machine reading. In this work, we propose Discern, a discourse-aware entailment reasoning network to strengthen the connection and enhance the understanding for both document and dialog. Specifically, we split the document into clause-like elementary discourse units (EDU) using a pre-trained discourse segmentation model, and we train our model in a weakly-supervised manner to predict whether each EDU is entailed by the user feedback in a conversation. Based on the learned EDU and entailment representations, we either reply to the user our final decision "yes/no/irrelevant" of the initial question, or generate a follow-up question to inquiry more information. Our experiments on the ShARC benchmark (blind, held-out test set) show that Discern achieves state-of-the-art results of 78.3% macro-averaged accuracy on decision making and 64.0 BLEU1 on follow-up question generation. Code and models are released at https://github.com/Yifan-Gao/Discern.