Kinect: The touch-free revolution in gaming

Today’s computer games depend on hand-held controllers that respond to physical touch. Kinect does away with any physical connection to the video game processors controlling the screen.

Instead, the players are lit up with an invisible infrared beam sent from the machine’s projector and their body movements, reflected in the infrared light, are captured by a 3D camera. The result is that body movements alone can be used to control the game: swing your right arm in your living room to return a ball on a screen version of Wimbledon, or kick your left leg and deliver a kung fu blow to the vitals of your mortal enemy.

Kinect, launched in the US and in Europe in November as an add-on peripheral for the Xbox 360 game console, marks a turning point in computer interaction. “This is a radically new way of controlling a computer,” says Andrew Blake, managing director of Microsoft Research in Cambridge. Blake, an expert in artificial intelligence and a former Oxford University professor of engineering science who leads the computer vision team involved in the development of Kinect, compares its motion-capture technology with the arrival of the touch-screen revolution more than a decade ago. “It is quite new both in terms of what’s out there in the market and the underlying technology. The technological achievement is something that the computer science community has been trying to do for a good twenty years,” he says. 

The no-touch technology involved two major breakthroughs: a 3D camera and sensors that could track the entire human skeleton; and algorithms and machine-learning techniques to interpret the images in real time – allowing programmers to build games based around those movements.

The potential for that technology goes far beyond games. Kinect’s underlying machine-learning and image analysis algorithms might  be deployed in a variety of settings where handling a physical object is dangerous or difficult, such as battlefields and sterile operating rooms. Image recognition algorithms similar to the ones developed for Kinect are also being harnessed to automatically interpret medical images.

Hollywood already works with expensive motion-capture technology, using computers that augment the movements of people in a digitally enhanced landscape. But the technique requires actors to wear tight-fitting body suits that mark all their limb joints. Kinect relies on “markerless” motion capture, Blake says.

Traditional motion capture relies on clever computer graphics, but this approach has inherent limitations. The Kinect team at Microsoft solved some of the fundamental problems with the help of advances in machine learning. It was marrying the two approaches, computer graphics and machine learning, that led to the breakthrough in Kinect’s development, Blake explains.

Despite 20 years of toil, the R&D investment in computer vision technology is coming to fruition more than 40 years before the 2054 dateline of Minority Report.  And the payback has only just begun.

Article by Steve Connor, Science|Business