Sign language recognition system and method

• US 8,428,643 B2
• Filed: 04/01/2010
• Issued: 04/23/2013
• Est. Priority Date: 12/17/2009
• Status: Active Grant

• Current Assignee: Chi Mei Communication Systems Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Original Assignee: Chi Mei Communication Systems Incorporated (Hon Hai Precision Industry Co., Ltd.)
• Inventors: Lin, Yung-Chuan Oneal
• Primary Examiner(s): Safaipour, Bobbak
• Application Number:
  US12/752,154
• Publication Number:
  US 20110151846A1
• Time in Patent Office: 1,118 Days
• Field of Search: 455/556.1, 455/550.1, 379/52, 379/93.15, 704/235, 704/260, 704/271, 704/3, 704/2
• US Class Current: 455/550.1
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: H04M 1/72478 for hearing-impaired users
• CPC Class Codes: H04M 2250/70 methods for entering alphabetical characters, e.g. multi-tap or dictionary disambiguation methods for entering alphabetical characters per se G06F3/0237

Abstract

A sign language recognition system includes a menu generating module, an image processing module, and text processing module. The image processing module controls an image capturing unit to capture a gesture image of a target person when an image recognition icon is selected, extracts sign language data from the gesture image, and transmits the sign data packet to a server for analyzing the sign language data to generate sign recognition information. The text processing module packs text data to be sent to a reception device, and a reception language and a phone number of the reception device to generate a text data packet when a text recognition icon is selected, and sends the text data packet to the server for analyzing the text data to generate a sign recognition image recognizable by the reception device.

Claims

1. A sign language recognition system, the system being implemented by a mobile device connected to a server through a network, the mobile device comprising an image capturing unit and a display screen, the sign language recognition system comprising:

a menu generating module configured to generate and display a main menu interface on the display screen, the main menu interface comprising an image recognition icon and a text recognition icon, and further generate and display a submenu interface on the display screen when the text recognition icon of the main menu interface is selected, the submenu interface comprising a language selection icon;

an image processing module configured to control the image capturing unit to capture a gesture image of a target person when the image recognition icon is selected, extract sign language data from the gesture image according to body language of the target person, pack the sign language data and a preset language of the mobile device to generate a sign data packet, and transmit the sign data packet to the server for analyzing the sign language data to generate sign recognition information; and

a text processing module configured to pack text data to be sent to a reception device, and a reception language and a phone number of the reception device to generate a text data packet when the text recognition icon is selected, and send the text data packet to the server for analyzing the text data to generate a sign recognition image recognizable by the reception device, wherein the reception language is selected from the language selection icon of the submenu interface according to a language used in the reception device.

2. The sign language recognition system according to claim 1, wherein the server connects to a data storage device that stores a sign language standard and a text analysis standard.

3. The sign language recognition system according to claim 2, wherein the server comprises:

a sign language analysis unit configured to analyze the sign data packet to generate a sign analysis result according to the sign language standard when the server receives the sign data packet from the mobile device, convert the sign analysis result into the sign recognition information according to the preset language of the mobile device, and send the sign recognition information to the mobile device through the network.

4. The sign language recognition system according to claim 2, wherein the server comprises:

a text analysis unit configured to analyze the text data packet to generate a text analysis result according to the text analysis standard when the server receives the text data packet from the mobile device, convert the text analysis result into the sign recognition image according to the sign language standard, and transmit the sign recognition image to the reception device according to the reception phone number.

5. The sign language recognition system according to claim 1, wherein the submenu interface further comprises a text input box, a phone number input icon, a text analysis icon, and an interface return icon.

6. The sign language recognition system according to claim 5, wherein text data is input from text input box, and the phone number of the reception device is input from the phone number input icon.

7. A sign language recognition method for a mobile device, the mobile device being connected to a server through a network and comprising an image capturing unit and a display screen, the method comprising:

generating and displaying a main menu interface on the display screen, the main menu interface comprising an image recognition icon and a text recognition icon;

determining whether the image recognition icon or the text recognition icon is selected;

in response to the image recognition icon being selected, executing block (a1) to block (a5);

(a1) controlling the image capturing unit to capture a gesture image of a target person;

(a2) extracting sign language data from the gesture image according to body language of the target person;

(a3) packing the sign language data and a preset language of the mobile device to generate a sign data packet;

(a4) transmitting the sign data packet to the server for analyzing the sign language data to generate sign recognition information;

(a5) receiving the sign recognition information from the server, and displaying the sign recognition information in format of the preset language on the display screen;

in response to the text recognition icon being selected, executing block (b1) to block (b4);

(b1) generating and displaying a submenu interface on the display screen, the submenu interface comprising a language selection icon;

(b2) packing text data to be sent to a reception device, and a reception language and a phone number of the reception device to generate a text data packet, wherein the reception language is selected from the language selection icon of the submenu interface according to a language used in the reception device;

(b3) transmitting the text data packet to the server for analyzing the text data to generate a sign recognition image recognizable by the reception device; and

(b4) receiving the sign recognition image from the server, and displaying the sign recognition image on the reception device.

8. The method according to claim 7, wherein the server connects to a data storage device that stores a sign language standard and a text analysis standard.

9. The method according to claim 8, wherein the block (a4) comprises:

analyzing the sign data packet to generate a sign analysis result according to the sign language standard when the server receives the sign data packet from the mobile device;

converting the sign analysis result into the sign recognition information according to the preset language of the mobile device; and

sending the sign recognition information to the mobile device through the network.

10. The method according to claim 8, wherein the block (b3) comprises:

analyzing the text data packet to generate a text analysis result according to the text analysis standard when the server receives the text data packet from the mobile device;

converting the text analysis result into the sign recognition image according to the sign language standard; and

transmitting the sign recognition image to the reception device according to the reception phone number.

11. The method according to claim 7, wherein the submenu interface further comprises a text input box for inputting the text data to be sent to the reception device, and a phone number input icon for inputting the phone number of the reception device.

12. A non-transitory storage medium having stored thereon instructions that, when executed by a processor of a mobile device, cause the mobile device to perform a method for recognizing sign language, the method comprising:

generating and displaying a main menu interface on the display screen, the main menu interface comprising an image recognition icon and a text recognition icon;

determining whether the image recognition icon or the text recognition icon is selected;

in response to the image recognition icon being selected, executing block (a1) to block (a5);

(a1) controlling the image capturing unit to capture a gesture image of a target person;

(a2) extracting sign language data from the gesture image according to body language of the target person;

(a3) packing the sign language data and a preset language of the mobile device to generate a sign data packet;

(a4) transmitting the sign data packet to a server for analyzing the sign language data to generate sign recognition information; and

(a5) receiving the sign recognition information from the server, and displaying the sign recognition information in format of the preset language on the display screen;

in response to the text recognition icon being selected, executing block (b1) to block (b4);

(b1) generating and displaying a submenu interface on the display screen, the submenu interface comprising a language selection icon;

(b2) packing text data to be sent to a reception device, and a reception language and a phone number of the reception device to generate a text data packet, wherein the reception language is selected from the language selection icon of the submenu interface according to a language used in the reception device;

(b3) transmitting the text data packet to the server for analyzing the text data to generate a sign recognition image recognizable by the reception device; and

(b4) receiving the sign recognition image from the server, and displaying the sign recognition image on the reception device.

13. The non-transitory storage medium according to claim 12, wherein the server connects to a data storage device that stores a sign language standard and a text analysis standard.

14. The non-transitory storage medium according to claim 13, wherein the block (a4) comprises:

analyzing the sign data packet to generate a sign analysis result according to the sign language standard when the server receives the sign data packet from the mobile device;

converting the sign analysis result into the sign recognition information according to the preset language of the mobile device; and

sending the sign recognition information to the mobile device through the network.

15. The non-transitory storage medium according to claim 13, wherein the block (b3) comprises:

analyzing the text data packet to generate a text analysis result according to the text analysis standard when the server receives the text data packet from the mobile device;

converting the text analysis result into the sign recognition image according to the sign language standard; and

transmitting the sign recognition image to the reception device according to the reception phone number.

16. The non-transitory storage medium according to claim 12, wherein the submenu interface comprises a text input box for inputting the text data to be sent to the reception device, and a phone number input icon for inputting the phone number of the reception device.