In a chilly basement room at Carnegie Mellon University sits a giant dome that looks like part physics experiment, part that chamber Darth Vader kicks back in. But this space wasn't built for subatomic particles, and it wasn't built for space villains. It was built for the betterment of robots. This is the so-called Panoptic Studio. The wires and electronics are actually an elaborate system of 500 cameras, both 2-D and 3-D, that capture groups of people inside the dome--none of those silly ball-covered suits required.

A new computer code could provide robots the capabilities to better understand the humans around them, paving the way for more perceptive machines, from self-driving cars to surveillance. The new technique allows a computer to understand body poses and movements of multiple people, even tracking parts as minute as individual fingers. While humans naturally communicate using body language, computers are'more or less blind' to these interactions – but, by tracking the 2D human form and motion, the new code could vastly improve robots' abilities in social situations. Multi-person tracking presents several challenges to computers – and, hand detection is even more of an obstacle. While image datasets on the human hand are far more limited than those on the face or body, the Panoptic Studio has allowed for unprecedented views of hand movement.

Scientists at the Carnegie Mellon University's Robotics Institute have recently developed a method to which they enabled a computer that reads human body language and their movements on video in real-time. The computer is also able to read the pose of each individual's fingers. Scientists actually developed this method in collaboration with the Panoptic Studio. The Panoptic Studio is a two-story dome consist of 500 video cameras. They analyzed insights from experiments that now make it possible to detect the pose of a group of people using a single camera and a laptop computer. Currently, it is being used to improve body, face and hand detectors by jointly training them.