Apparatus capable of processing sign language information

• US 6,549,887 B1
• Filed: 01/20/2000
• Issued: 04/15/2003
• Est. Priority Date: 01/22/1999
• Status: Expired due to Term

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Takeuchi, Masaru, Ando, Haru, Sagawa, Hirohiko
• Primary Examiner(s): SMITS, TALIVALDIS IVARS
• Application Number:
  US09/488,076
• Time in Patent Office: 1,181 Days
• Field of Search: 704/271, 345/723, 345/474
• US Class Current: 704/271
• CPC Class Codes: G06T 13/20 3D [Three Dimensional] animation
• CPC Class Codes: G09B 21/00 Teaching, or communicating with, the blind, deaf or mute audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip

Abstract

Inputted sign language word labels and editing items such as speeds and positions of moving portions for specifying manual signs and/or sign gestures corresponding to the respective sign language word labels are displayed on an editing screen. These editing items are modified by the user to add non-language information such as emphasis/feeling information to the contents of communication, thereby generating modified sign language animation information data including the inputted sign language word label string having the added non-language information. For communication or interaction, the non-language information is extracted from the modified sign language animation information data and stored into a memory with the inputted sign language word label string. When a hearing impaired person communicates or interacts with another person through text, the user can emphasize the contents of communication or show the user'"'"'s feeling for the contents of communication to the other person.

Claims

1. A sign language information processing apparatus capable of sign language mail communication, comprising:

information input means capable of inputting at least text information;

animation information generation means for generating sign language animation information by use of sign language word label information;

information storage means coupled with said information input means and said animation information generation means;

information display means coupled with said information input means, said animation information generation means and said information storage means;

a memory for storing non-language information correspondence data represented by a medium of each of sign language information, text information and speech information, said non-language information including at least one of emphasis representation and feeling representation;

communication means for sending and receiving said sign language word label information and said non-language information through a communication line;

non-language information extraction means for extracting said at least one emphasis representation and feeling representation from modified information, wherein said modified information includes original language animation information generated by said animation information generation means from sign language word label information inputted by said information input means and said at least one of emphasis representation and feeling representation added to the original sign language animation information, said non-language information extracting means being operative to determine a difference between said original sign language animation information and said modified information by use of the non-language information correspondence data stored in said memory and to extract said at least one of emphasis representation and feeling representation on the basis of the determined difference; and

sign-language data translation means for translating the sent sign language word label information and non-language information into output data in the form of text data or speech data.

2. A sign language information processing apparatus according to claim 1, wherein said non-language information extraction means includes characteristics extraction means for detecting at least one of a pose point, an acceleration value between pose points, the stay duration of a pose point, the duration of a word, the ratio of the duration of a word to the duration of a text, and a syntax as characteristics from said modified information data, and difference value determination means for determining a difference value between the characteristics detected by said characteristics extraction and characteristics having already been entered as basic data.

3. A sign language information processing apparatus according to claim 1, further comprising:

morphological analysis means for dividing character string information of text information inputted by said information input means into word information which includes information of parts of speech and conjugations;

adjacent-word coupling means for coupling adjacent ones of divisional words obtained by said morphological analysis means;

in-word division means for further dividing the divisional word;

word division means for dividing natural language words in natural language word to sign language word labels stored in said information storage means; and

natural language word to sign language word label correspondence data collation means for collating the coupled words or the divisional words with said natural language word to sign language word label correspondence data to extract sign language word labels.

4. A sign language information processing apparatus according to claim 1, further comprising:

sign language animation data editing means including means for designating information of a position on a screen of said information display means, said sign language animation data editing means specifying at least one of an emphasis portion and a feeling added portion indicative of non-language information on the text information inputted by said information input means;

means for storing the specified portion as the at least one of an emphasis portion and a feeling added portion; and

means for adding non-language information to sign language animation data corresponding to said at least one of an emphasis portion and a feeling added portion.

5. A sign language information processing apparatus according to claim 1, wherein said information input means includes speech input means for inputting vocalization information, A/D translation means for A/D-translating said vocalization information, and speech recognition means for acquiring vocal sound information, sound pressure information, sound pitch information and time information of the inputted vocalization information from the A/D-translated information, and said speech recognition means extracts non-language information from a word string, the duration of silence between words and the duration of a vocal sound in the vocalization information and includes speech to sign language non-language information translation means for displaying said non-language information as a sign language representing animation.

6. A sign language information processing apparatus according to claim 1, wherein said output data form is selected on an information receiving side or on an information sending side.

7. A sign language information processing apparatus according to claim 1, further comprising speech synthesis means for synthesizing a speech from character string information, said speech synthesis means including sign language animation to speech translation means for translating sign animation information into speech information through the speech synthesis by controlling the duration of a word, the duration of silence between words and the duration of each vocal sound in a word by use of the non-language information extracted by said non-language information extraction means.

8. A sign language information processing apparatus comprising:

a display unit;

an information input section;

means for generating sign language animation information from a sign language word label string inputted into said information input section;

a first memory for storing non-language information correspondence data represented by a medium of each of sign language information, text information, and speech information, said non-language information including at least one of emphasis representation and feeling representation;

means for driving said display unit to display a sign language animation corresponding to said sign language animation information, each sign language word label of said sign language word label string defining said sign language animation, for editing items for specifying manual signs or sign gestures corresponding to the respective sign language word labels on said display unit, and for changing said editing item to add said non-language information inclusive of said at least one of emphasis information and feeling information to said sign language animation information, thereby generating modified sign language animation information;

means for extracting said at least one of emphasis representation and feeling representation from said modified sign language animation information, said extracting means being operative to determine a difference between said generated sign language animation information and said modified information by use of the non-language information correspondence data stored in said first memory and to extract said at least one of emphasis representation and feeling representation on the basis of the determined difference;

a second memory for storing the inputted sign language word label string and the extracted non-language information;

means for reading said sign language word label string and said non-language information from said second memory and coupling the read non-language information with the read sign language word label string; and

an information output section for receiving an output from said coupling means.

9. A sign language information processing apparatus according to claim 8, further comprising:

means for translating the read sign language word label string into a natural language text;

means for translating said natural language text into a phonetic alphabet string; and

means for coupling the read non-language information with a sound source data string corresponding to said phonetic alphabet string.

10. A sign language information processing apparatus comprising:

an information input section;

means for recognizing speech information inputted to said information input section and including non-language information including at least one of emphasis information and feeling information to generate character strings corresponding to said speech information;

a first memory for storing non-language information correspondence data represented by a medium of each of sign language information, text information, and speech information, said non-language information including said at least one of emphasis representation and feeling representation;

means for extracting said non-language information from the generated character strings, said extracting means being operative to distinguish said non-language information in the generated character strings and said modified information by use of the non-language information correspondence data stored in said first memory and-to extract said at least one of emphasis representation and feeling representation on the basis of the distinction;

a second memory for storing the generated character strings and the extracted non-language information;

means for reading said character strings from said memory and translating the read character strings into a sign language word label string;

means for reading said non-language information from said memory and coupling the read non-language information with said sign language word label string; and

an information output section for receiving an output from said coupling means.

11. A sign language information processing apparatus comprising:

a display unit;

an information input section;

means for generating sign language animation information from a sign language word label string inputted into said information input section;

a first memory for storing non-language information correspondence data represented by a medium of each of sign language information, text information, and speech information, said non-language information including at least one of emphasis representation and feeling representation;

means for driving said display unit to display a sign language animation corresponding to said sign language animation information, each sign language word label of said sign language word label string defining said sign language animation, and editing items for specifying manual signs or sign gestures corresponding to the respective sign language word labels on said display unit and changing said editing item to add non-language information inclusive of said at least one of emphasis information and feeling information to said sign language animation information, thereby generating modified sign language animation information;

means for extracting said non-language information from said modified sign language animation information, said extracting means being operative to determine a difference between said generated sign language animation information and said modified information by use of the non-language information correspondence data stored in said first memory and to extract said at least one of emphasis representation and feeling representation on the basis of the determined difference;

a second memory for storing the inputted sign language word label string and the extracted non-language information;

means for reading said sign language word label string from said memory and translating the read sign language word label string into a natural language text;

means for translating said natural language text into a phonetic alphabet string;

means for reading said non-language information from said memory and coupling the read non-language information with a sound source data string corresponding to said phonetic alphabet string; and

an information output section for receiving an output from said coupling means.