Speak Sign Language to Your Computer with Microsoft Kinect

  • July 23, 2013
teaching-Kinect-sign-language
Listen to this page using ReadSpeaker

Researchers in China are working on technology that will allow sign language speakers to communicate better with Microsoft Kinect. The research project has the potential to make interacting with computers more accessible–and cheaper–for sign language speakers.  The technology works by identifying a user’s movements and matching them to the word the movements represent. For example, if a person were to complete the movement shown in the diagram below, the Kinect would pick up the movement and match it to “yes” and interpret it as a command.

The image is of a sign language diagram. The background is white. In black letters, the word "yes" is written in lowercase font. Below the word, there is a closed fist with arrows pointing above and below it, indicating vertical motion

For those who primarily use sign language to communicate, this technology opens a new door. It ensures that they can use their preferred method of communication to interact with computers, programs, and other technology more easily. It also makes use of an existing technology, meaning that the sign language recognition technology will be cheap for those who already have a Kinect. Furthermore, the sign language recognition is rapid and accurate, making it a practical way for sign language users to communicate.

Another exciting feature of this technology is the translate feature. A user can sign to the Kinect and it will translate the signs into text. A person who does not understand sign language can then communicate with the person using sign language and vice versa. This opens a new plane of communication for two people who do not speak the same language.

While the technology is still in the research and development stage, researchers hope that it will be an available feature in “the near future”. Guobin Wu, a researcher working on the project summarizes the mission of the project by saying, “We believe that information technology should be used to improve daily life for all persons.”  Take a look at the video below to get a feel for how this new technology will work:

Support Life Labs through a donation Donate Today!