nerf network
Asset2Vec: Turning 3D Objects into Vectors and Back
At Datagen, where I currently work as the Head of AI research, we create synthetic photorealistic images of common 3D environments, for the purpose of training computer vision algorithms. For example, if you want to teach a house robot to navigate through a messy bedroom like the one below, it will take you quite some time to collect real images for a large-enough training set [people don't usually like when outsiders enter their bedroom, and they definitely won't appreciate taking pictures of their mess]. We generated the above image using a graphic software. Once we build (in the software) the 3D models of the environment and everything in it, we use its ray-tracing renderer to generate an image of the scene from any camera view point we like, under any light conditions we want. Now if you think collecting real images is hard, wait till you try to label these images.
Photos to 3D Scenes in Milliseconds
As if taking a picture wasn't a challenging enough technological prowess, we are now doing the opposite: modeling the world from pictures. I've covered amazing AI-based models that could take images and turn them into high-quality scenes. A challenging task consists of taking a few images in the 2-dimensional picture world to create how the object or person would look in the real world. You can easily see how useful this technology is for many industries like video games, animation movies, or advertising. Take a few pictures and instantly have a realistic model to insert into your product.