Goto

Collaborating Authors

 Model-Based Reasoning


CausalVLBench: Benchmarking Visual Causal Reasoning in Large Vision-Language Models

Komanduri, Aneesh, Bhaila, Karuna, Wu, Xintao

arXiv.org Artificial Intelligence

Large language models (LLMs) have shown remarkable ability in various language tasks, especially with their emergent in-context learning capability. Extending LLMs to incorporate visual inputs, large vision-language models (LVLMs) have shown impressive performance in tasks such as recognition and visual question answering (VQA). Despite increasing interest in the utility of LLMs in causal reasoning tasks such as causal discovery and counterfactual reasoning, there has been relatively little work showcasing the abilities of LVLMs on visual causal reasoning tasks. We take this opportunity to formally introduce a comprehensive causal reasoning benchmark for multi-modal in-context learning from LVLMs. Our CausalVLBench encompasses three representative tasks: causal structure inference, intervention target prediction, and counterfactual prediction. We evaluate the ability of state-of-the-art open-source LVLMs on our causal reasoning tasks across three causal representation learning datasets and demonstrate their fundamental strengths and weaknesses. We hope that our benchmark elucidates the drawbacks of existing vision-language models and motivates new directions and paradigms in improving the visual causal reasoning abilities of LVLMs.



ChaosBench: A Multi-Channel, Physics-Based Benchmark for Subseasonal-to-Seasonal Climate Prediction Juan Nathaniel

Neural Information Processing Systems

Y et, forecasting beyond the weather timescale is challenging because it deals with problems other than initial condition, including boundary interaction, butterfly effect, and our inherent lack of physical understanding.




InsActor: Instruction-driven Physics-based Characters

Neural Information Processing Systems

Our framework empowers InsActor to capture complex relationships between high-level human instructions and character motions by employing diffusion policies for flexibly conditioned motion planning.



Bicriteria Multidimensional Mechanism Design with Side Information

Neural Information Processing Systems

Mechanism design is a high-impact branch of economics and computer science that studies the implementation of socially desirable outcomes among strategic self-interested agents. Major real-world use cases include combinatorial auctions ( e.g., strategic sourcing, radio spectrum auctions),


Bicriteria Multidimensional Mechanism Design with Side Information

Neural Information Processing Systems

Mechanism design is a high-impact branch of economics and computer science that studies the implementation of socially desirable outcomes among strategic self-interested agents. Major real-world use cases include combinatorial auctions ( e.g., strategic sourcing, radio spectrum auctions),