SIGN LANGUAGE RECOGNITION SYSTEM AND METHOD

• US 20130051614A1
• Filed: 10/31/2011
• Published: 02/28/2013
• Est. Priority Date: 08/23/2011
• Status: Active Grant

• Current Assignee: Hon Hai Precision Industry Co., Ltd.
• Original Assignee: Hon Hai Precision Industry Co., Ltd.
• Inventors: LEE, HOU-HSIEN, LEE, CHANG-JUNG, LO, CHIH-PING
• Granted Patent: US 8,503,731 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 40/28 Recognition of hand or arm movements, e.g. recognition of deaf sign language static hand signs G06V40/113
• CPC Class Codes: G10L 13/04 Details of speech synthesis systems, e.g. synthesiser structure or memory management

Abstract

A sign language recognition method includes a depth-sensing camera capturing an image of a gesture of a signer and gathering data about distances between a number of points on the signer and the depth-sensing camera, building a three dimension (3D) model of the gesture, comparing the 3D model of the gesture with a number of 3D models of different gestures to find out the representations of the 3D model of the gesture, and displaying or vocalizing the representations of the 3D model of the gesture.

Claims

1. A sign language recognition system comprising:

a depth-sensing camera to capture an image of a gesture of a signer in a line of sight of the depth-sensing camera, and gather data as to distances between a plurality of points on the signer and the depth-sensing camera;

a processing unit;

a storage unit connected to the processing unit and the depth-sensing camera, and storing a plurality of programs to be executed by the processing unit, wherein the storage unit comprises;

a three dimension (3D) model building module to build a 3D model of the gesture according to the image of the gesture and the data about distances between the plurality of points in the scene and the depth-sensing camera;

a gesture storing module storing a plurality of 3D models of gestures and representations for each 3D model; and

a recognition module to compare the 3D model of the gesture obtained with the plurality of 3D models of gestures, to recognize the representations of the 3D model; and

an output unit connected to the processing unit, to output the representations of the 3D model of the gesture.

2. The system of claim 1, wherein the plurality of 3D models of gestures stored in the gesture storing module comprises a plurality of types of gestures, each type of gestures corresponds to a work mode, and comprises a plurality of 3D models of gestures;

the storage unit further comprises a sign language system setting module to manually set the work mode, and the recognition module compares the 3D model of the gesture obtained by the 3D model building module with the plurality of 3D models of gestures belonged to a type of gestures corresponding to the work mode, to recognize the representations of the 3D model of the gesture.

3. The system of claim 1, wherein the plurality of 3D models of gestures stored in the gesture storing module comprises a plurality of types of gestures, each type of gestures corresponds to a work mode, and comprises a plurality of gestures;

the storage unit further comprises a sign language identification module to compare the 3D model of the gesture obtained by the 3D model building module with the plurality of 3D models of gestures, to determine which type the 3D module of the gesture belongs to, and correspondingly set the work mode;

the recognition module compares the 3D module of the gesture obtained by the 3D model building module with the plurality of 3D models of gestures belonged to one type of gestures corresponding to the work mode, to find out the representations of the 3D model of the gesture.

4. The system of claim 3, wherein if a 3D model of gesture of the signer obtained by the 3D model building module belongs to two or more types of gestures, the sign language identification module then compares a 3D model of a next gesture of the signer obtained by the 3D model building module with the plurality of 3D models of different types of gestures, to determine which type the 3D model of the gesture belongs to.

5. The system of claim 1, wherein the output unit is a screen to display the representations of the 3D model of the gesture obtained by the 3D model building module.

6. The system of claim 1, wherein the storage unit further comprises a voice conversion module to convert the representations of the 3D model of the gesture obtained by the 3D model building module into audible sounds;

the output unit is a speaker to play the representations of the 3D model of the gesture obtained by the 3D model building module.

7. The system of claim 1, wherein the storage unit further comprises a voice conversion module to convert the representations of the 3D model of the gesture obtained by the 3D model building module into audible sounds;

the output unit is an earphone to play the representations of the 3D model of the gesture obtained by the 3D model building module.

8. A sign language recognition method comprising:

capturing an image of a gesture of a signer and gathering data about distances between a plurality of points on the signer and a depth-sensing camera by the depth-sensing camera;

building a three dimension (3D) model of the gesture according to the image and the data about distances;

comparing the 3D model of the gesture with a plurality of 3D models of gestures to find out the representations of the 3D model of the gesture; and

outputting the representations of the 3D model of the gesture.

9. The method of claim 8, wherein between the step “

building a three dimension (3D) model of the gesture according to the image and the data about distances” and

the step “

comparing the 3D model of the gesture with a plurality of 3D models of gestures to find out the representations of the 3D model of the gesture”

, further comprising;

determining whether to set a work mode manually; and

setting the work mode manually upon the condition that the work mode needs to be set manually.

10. The method of claim 9, further comprising:

comparing the 3D model of the gesture of the signer obtained with a plurality of 3D models of different types of gestures, to determine which type the 3D model of the gesture of the signer belongs to, and correspondingly set the work mode, upon the condition that the work mode does not need to be set manually.

11. The method of claim 9, wherein the step “

outputting the representations of the 3D model of the gesture”

comprises;

displaying the representations of the 3D model of the gesture by a screen.

12. The method of claim 9, wherein the step “

outputting the representations of the 3D model of the gesture”

comprises;

outputting the representations of the 3D model of the gesture by a speaker or an earphone.