Sign language/word translation system

• US 5,481,454 A
• Filed: 10/27/1993
• Issued: 01/02/1996
• Est. Priority Date: 10/29/1992
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Abe, Masahiro, Inoue, Kiyoshi, Sagawa, Hirohiko
• Primary Examiner(s): Hayes, Gail O.
• Assistant Examiner(s): Tkacs, Stephen R.
• Application Number:
  US08/141,646
• Time in Patent Office: 797 Days
• Field of Search: 364/419.02, 364/419.13, 364/419.11, 364/419.12, 364/419.03, 364/419.07, 382/1, 379/52, 379/93, 381/36, 341/20, 434/229
• US Class Current: 704/3
• CPC Class Codes: G06F 3/017 Gesture based interaction, e.g. based on a set of recognized hand gestures interaction based on gestures traced on a digitiser G06F3/04883
• CPC Class Codes: G09B 21/009 Teaching or communicating with deaf persons G09B21/04 takes precedence; deaf and dumb persons G09B21/00

Abstract

A sign language/word translation system can cope with the case where no word corresponding to a hand operation is found. The system includes a configuration for taking a countermeasure against the finding of a plurality of words corresponding to a hand operation. When a word corresponding to a hand operation is not found in a sign language word dictionary, a finger operation is input, a character corresponding to the finger operation is searched for in a manual alphabet pattern dictionary, and the character train thus obtained is found from a character word dictionary, thereby producing an appropriate word. Also, a plurality of words found from the sign language word dictionary are displayed as translation candidates, and the user inputs by a hand operation as to whether the translation candidate is appropriate or not. The translation candidate that has been decided to be appropriate is determined as the word resulting from the translation.

Claims

1. A sign language/word translation computer system for translating a sign language motion into a verbal language representation thereof, comprising:

hand operation input means for converting a sign language hand operation into a signal representative thereof;

means including a sign language word dictionary memory for storing words corresponding to sign language hand operations;

means for pattern matching contents of said sign language word dictionary memory to recognize a word corresponding to a hand operation signal from said hand operation input means;

finger operation input means for converting a sign language finger operation into a finger operation signal when the word corresponding to the hand operation signal is not found in the sign language word dictionary memory;

means including a manual alphabet pattern dictionary memory for storing characters corresponding to sign language finger operations;

manual alphabet pattern recognition means for pattern matching contents of said manual alphabet pattern dictionary memory to recognize a character corresponding to each finger operation signal from said finger operation input means;

means including a character word dictionary memory for storing words corresponding to trains of said characters of said manual alphabet pattern dictionary memory; and

character train word pattern recognition means for pattern matching contents of said character word dictionary memory for a word based on a train of characters searched by said manual alphabet pattern recognition means.

2. A sign language/word translation computer system according to claim 1, further comprising learning means for receiving the word found in said character word dictionary memory by said character train word pattern recognition means, and for storing the found word in said sign language word dictionary memory in correspondence with a corresponding hand operation.

3. A sign language/word translation computer system according to claim 1, wherein said character train word pattern recognition means includes means for outputting the character train as it is when the word corresponding to the character train is not found in said character word dictionary memory.

4. A sign language/word translation computer system according to claim 1, wherein said finger operation input means includes means for inputting a sign language finger operation of writing a character in the air by a finger when the word corresponding to the hand operation is not found in the sign language word dictionary memory.

5. A sign language/word translation computer system according to claim 1, further comprising:

translation candidate presentation means for displaying one of a plurality of words found in said sign language word dictionary memory as a translation candidate and inputting by a hand operation whether the translation candidate is appropriate;

means for determining the translation candidate as a word resulting from translation when a hand operation indicating the appropriateness of the translation candidate is input through said hand operation input means; and

means for deleting a translation candidate and re-energizing the translation candidate presentation means when a hand operation indicating the inappropriateness of the translation candidate is input through said hand operation input means.

6. A sign language/word translation computer system according to claim 1, further comprising:

translation candidate presentation means for displaying a plurality of words as a list of translation candidates and having the user select an appropriate one of the translation candidates by hand operation when said words are found in said sign language word dictionary memory; and

means for determining an appropriate translation candidate as a word resulting from translation when the hand operation for selecting said translation candidate is input through the hand operation input means.

7. An automated machine-implemented sign language/word translation method for translating a sign language motion into a verbal language representation thereof, comprising the steps of:

converting a sign language hand operation into an electrical signal representative thereof;

storing words corresponding to sign language hand operations in a sign language word dictionary memory;

pattern matching contents of said sign language word dictionary memory to recognize a word corresponding to a hand operation electrical signal generated by said step of converting a sign language hand operation into an electrical signal;

converting a sign language finger operation into a finger operation electrical signal when the word corresponding to the hand operation electrical signal is not found in the step of pattern matching contents of the sign language word dictionary memory;

storing characters corresponding to sign language finger operations in a manual alphabet pattern dictionary memory;

pattern matching contents of said manual alphabet pattern dictionary memory to recognize a character corresponding to each finger operation electrical signal generated by said step of converting a sign language finger operation into a finger operation electrical signal;

storing words corresponding to trains of said characters of said manual alphabet pattern dictionary memory in a character word dictionary memory; and

pattern matching contents of said character word dictionary memory to recognize a word based on a train of characters determined by said step of pattern matching contents of said manual alphabet pattern dictionary memory.

8. An automated machine-implemented sign language/word translation method as claimed in claim 7, further comprising the steps of receiving the word found in said character word dictionary memory, and storing the found word in said sign language word dictionary memory in correspondence with a corresponding sign language hand operation.

9. An automated machine-implemented sign language/word translation method as claimed in claim 7, wherein said step of pattern matching contents of said character word dictionary memory includes a step of outputting the train of characters pattern-matched in said step of pattern matching contents of said manual alphabet pattern dictionary memory, when the word corresponding to the character train is not found in said character word dictionary memory.

10. An automated machine-implemented sign language/word translation method as claimed in claim 7, wherein said step of converting a sign language finger operation into a finger operation electrical signal includes the step of inputting a sign language finger operation of writing a character in the air by a finger when the word corresponding to the hand operation is not found in the sign language word dictionary memory.

11. An automated machine-implemented sign language/word translation method as claimed in claim 7, further comprising the steps of:

displaying one of a plurality of words found in said sign language word dictionary memory as a translation candidate and inputting by a hand operation whether the translation candidate is appropriate;

determining the translation candidate as a word resulting from translation when a hand operation indicating the appropriateness of the translation candidate is converted in said step of converting a sign language hand operation into an electrical signal representative thereof; and

deleting a translation candidate and repeating the step of displaying one of a plurality of words when a hand operation indicating the inappropriateness of the translation candidate is converted in said step of converting a sign language hand operation into an electrical a signal representative thereof.

12. An automated machine-implemented sign language/word translation method as claimed in claim 7, further comprising the steps of:

displaying a plurality of words as a list of translation candidates and having a user select an appropriate one of the translation candidates by a hand operation when said words are found in said sign language word dictionary memory; and

determining an appropriate translation candidate as a word resulting from translation when the hand operation for selecting said translation candidate is converted in said step of converting a sign language hand operation into an electrical signal representative thereof.