[D] Computer Vision as Inverse Computer Graphics?

Inverse Computer Graphics aims to solve the problem of computer vision end-to-end by having a model that can take an image (or sequence of images taken from different view points with known relative positioning), and output a 3D mesh of the world that generated these images. Many of the renowned researchers hold up Inverse Computer Graphics as one of the benchmarks for AI. Hinton is definitely the most prominent, and a lot of his recent research (like Capsule Networks) aims to address that. Karpathy got in the game as well, although I am not sure how exactly the research he cites relates to this problem. I personally find this area very interesting since I love both 3D graphics and AI.