Learning a Multi-View Stereo Machine

Consider looking at a photograph of a chair. We humans have the remarkable capacity of inferring properties about the 3D shape of the chair from this single photograph even if we might not have seen such a chair ever before. A more representative example of our experience though is being in the same physical space as the chair and accumulating information from various viewpoints around it to build up our hypothesis of the chair's 3D shape. How do we solve this complex 2D to 3D inference task? What kind of cues do we use?