Natural Building Blocks for Structured World Models: Theory, Evidence, and Scaling