Apparatus for interactively editing and outputting sign language information using graphical user interface

• US 5,734,923 A
• Filed: 06/17/1996
• Issued: 03/31/1998
• Est. Priority Date: 09/22/1993
• Status: Expired due to Fees

• Current Assignee: Hitachi, Ltd.
• Original Assignee: Hitachi, Ltd.
• Inventors: Kaneko, Youichi, Sagawa, Hirohiko, Ohki, Masaru, Ohira, Eiji, Sakiyama, Tomoko
• Primary Examiner(s): Herndon, Heather R.
• Assistant Examiner(s): HONG, STEPHEN S
• Application Number:
  US08/664,803
• Time in Patent Office: 652 Days
• Field of Search: 395/806, 395/807, 395/173, 395/174, 395/175, 395/328, 395/751, 395/2.44, 395/2.69, 395/2.8, 434/185, 434/169, 434/167
• US Class Current: 715/204
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text

Abstract

An information output apparatus which outputs guide information in combination with image, text, voice and sign-language so that hearing impaired, handicapped people can easily recognize the guide information. The sign-language is automatically generated by computer graphics on the basis of text information inputted to the apparatus. A user can edit the generated sign-language. When it is desired to perform a generating operation over additional sign-language information, a transitional picture is automatically inserted between the sign-language words to provide clear discrimination between the words. The inserting manner can be specified by the user.

Claims

1. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall comprising:

image input means for inputting an image;

text input means for inputting a text;

voice input means for inputting voice;

sign-language input means for inputting sign-language action pattern data representing action patterns of sign-language;

first image memory means for storing the image inputted by said image input means;

first text memory means for storing the text inputted by said text input means;

first voice memory means for storing the voice inputted by said voice input means;

first sign-language memory means for storing the sign-language inputted by said sign-language input means;

image editing means for editing the images stored in said first image memory means;

text editing means for editing the text stored in said first text memory means;

voice editing means for editing the voice stored in said first voice memory means;

sign-language editing means for editing the sign-language stored in said first sign-language memory means;

second image memory means for storing the images edited by said image editing means;

second text memory means for storing the text edited by said text editing means;

second voice memory means for storing the voice edited by said voice editing means;

second sign-language memory means for storing the sign-language edited by said sign-language editing means;

guide information editing means for generating guide information including at least one of the images stored in said second image memory means, the text stored in said second text memory means, the voice stored in said second voice memory means, and the sign-language stored in said second sign-language memory means;

guide information memory means for storing said guide information;

guide information output means for outputting the guide information stored in said guide information memory means;

wherein said sign-language editing means further includes;

converting means for converting said text into a word string, where each word in the word string corresponds to a sign-language expression which is a fundamental unit of meaning in sign-language;

input means for permitting a user to input a word string;

adding means for permitting the user to add information such as a facial expression, gesture or direction of motion which are varied according to context for each word, plural words or a part of a word in a word string; and

generation means for generating a series of sign-language animation automatically by utilizing sign-language pattern data of each word according to word string and information added by user.

2. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, wherein said sign-language editing means outputs to said output means at least one of said sign-language;

the text corresponding to said sign-language;

a word string, a gesture size of sign-language expression corresponding to a word;

a direction of sign-language expression corresponding to a word;

a display position of the sign-language expression corresponding to a word;

a time interval between adjacent words;

facial expression;

gesture;

mouth movement;

a plurality of sign-language expression corresponding to a word and display sequence of sign-language expression corresponding to a word when generating sign-language animation.

3. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, wherein said sign-language editing means edits at least one of said sign-language;

the text corresponding to said sign-language;

a word string, a gesture size of sign-language expression corresponding to a word;

a direction of sign-language expression corresponding to a word;

a display speed of gesture of sign-language expression corresponding to a word;

a display position of sign-language expression corresponding to word;

a time interval between adjacent words;

facial expression;

gesture;

mouth movement;

a plurality of sign-language expression corresponding to a word and display sequence of sign-language expression corresponding to a word when generating sign-language animation.

4. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, wherein said sign-language editing means outputs to said output means said sign-language and the text or word string corresponding to said sign-language synchronously, and indicates one of the words contained in said text or in said word string, corresponding to the sign-language being outputted so as to distinguish said indicated word from the other words.

5. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, wherein said output means includes a converter means for converting the text stored in said first text memory means or the text stored in said second text memory means to voice and for outputting the voice.

6. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, wherein said voice editing means edits at least one of the text corresponding to said voice to be outputted, an accent, tone and inter-word time interval of the words of the text.

7. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, wherein said voice editing means outputs to said voice output means said voice and the text corresponding to said voice synchronously, and wherein said voice editing means indicates one of the words of said text being outputted to said output means and corresponding to said voice from the other word.

8. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1, further comprising means for selecting at least one of the image, text, voice and sign-language animation stored in said guide information memory means and means for providing the selected information to said output means.

9. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 1:

wherein said sign-language editing means produces a sign-language animation of a transitional part which is the hand motion between adjacent words on the basis of said sign-language pattern data corresponding to a word adjacent to said transitional part and located ahead of the transitional part as well as said sign-language pattern data corresponding to a word adjacent to said transitional part and located behind the transitional part, and a model of sign-language speed obtained by analysis of sign-language motion characteristics and inserts said transitional sign-language pattern into between sign-language animation corresponding to the adjacent words.

10. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 9, wherein said sign-language editing means controls a speed of the hand motion of the sign-language information to be displayed in said motion image of the transitional part so as to start the hand motion of the sign-language displayed in the motion image of the transitional part at a final speed of the hand motion of the sign-language corresponding to a word adjacent to the transitional part and located ahead of the transitional part, to increase a speed of the hand motion of the sign-language information up to a predetermined value which is determined by analyzing the feature of hand motion in sign-language, and after said hand motion speed reaches said predetermined value, and to decrease said hand motion speed down to an initial speed of hand motion of the sign-language corresponding to a word adjacent to the transitional part and located behind the transitional part.

11. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 10, wherein said sign-language editing means controls said predetermined value to be substantially larger than both of said final and initial speeds.

12. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 9, wherein said sign-language editing means determines a speed of hand motion of the sign-language and a time length of said transitional part to be displayed in said motion image of the transitional part on the basis of a distance between a final hand position of the sign-language word located ahead the transitional part and an initial hand position of the sign-language word adjacent to said transitional part.

13. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 12, wherein said sign-language editing means determines the time length of said transitional part to be a multiple of a predetermined value which is determined by analyzing the feature of hand motion in sign-language.

14. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 9, wherein said sign-language editing means generates the position and direction of hands and the bent angle of finger joints to be displayed in the motion image of the transitional part by a linear interpolation method with use of the final hand position and direction of hands and the final bent angle of finger joints in the sign-language pattern data corresponding to a word adjacent to said transitional part and located ahead of the transitional part and of the initial hand position and direction of hands and the final bent angle of finger joints in the sign-language pattern data corresponding to a word adjacent to said transitional part and located after the transitional part.

15. A sign language guide information output apparatus for providing guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall as set forth in claim 9, further comprising means for storing a correlation between a distance from a final hand position of the sign-language words located ahead the transition part, to an initial hand position of the sign-language words adjacent to the transitional part, and a highest hand speed in the transition part to be inserted in between the adjacent sign-language words, and wherein said highest hand speed in the transition part to be inserted in between the adjacent sign-language words is determined, referencing said correlation, on the basis of said distance between the final hand position of the former sign-language word and the initial hand position of the latter sign-language word, and the time length of said transition part is determined on the basis of the final hand speed of the former sign-language word and the initial speed of the later sign-language word.

16. A method of outputting sign language guide information such as an information menu in an automatic teller machine or an exhibit guidance in an exhibit hall comprising the steps of:

inputting image;

inputting text;

inputting voice;

inputting sign-language;

storing said inputted image in a first memory;

storing said inputted text in a second memory;

storing said inputted voice in a third memory;

storing said inputted sign-language in a fourth memory;

editing the image stored in said first memory;

editing the text stored in said second memory;

editing the voice stored in said third memory;

editing the sign-language stored in said fourth memory;

storing said edited image in a fifth memory;

storing said edited text in a sixth memory;

storing said edited voice in a seventh memory;

storing said edited sign-language in an eighth memory;

producing guide information including at least one of the image stored in said fifth memory, the text stored in said sixth memory, the voice stored in said seventh memory, and the sign-language stored in said eighth memory;

storing said synthesized guide information in a ninth memory;

outputting said guide information stored in said ninth memory;

wherein said step of editing the sign-language includes;

converting said text into word string, where each word in the word string corresponds to a sign-language expression which is a fundamental unit of meaning in sign-language;

inputting a word string;

adding information such as a facial expression, gesture or direction of motion which are varied according to context for each word, plural words or a part of a word in word string by the user; and

generating successive sign-language animation automatically utilizing sign-language pattern data corresponding to each word according to the word string and said information added by the user.