Individual Planning in Infinite-Horizon Multiagent Settings: Inference, Structure and Scalability