Human image dialogue device and a recording medium storing a human image dialogue device

• US 6,434,525 B1
• Filed: 05/26/1999
• Issued: 08/13/2002
• Est. Priority Date: 05/27/1998
• Status: Expired due to Fees

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Nagisa, Izumi, Kusui, Dai
• Primary Examiner(s): Dorvil, Richemond
• Assistant Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/318,806
• Time in Patent Office: 1,175 Days
• Field of Search: 704/270, 704/275, 704/276, 704/260, 704/251, 345/326, 345/333, 345/729
• US Class Current: 704/260
• CPC Class Codes: G10L 21/06 Transformation of speech into a non-audible representation, e.g. speech visualisation or speech processing for tactile aids G10L15/26 takes precedence

Abstract

A device is provided that generates the gestures and expressions of a human image on a computer without expending a great amount of labor. The words for the system response to the input of a user and the state of the dialogue are described in a dialogue flow memory unit, a dialogue flow analysis unit analyzes the spoken text of the flow, extracts the key words associated with a movement pattern by referring to a text movement association table, and the movement expression generation unit generates the movements corresponding to the movement pattern. In the generation of the movement, movement patterns determined in advance are selected according to the state of the dialogue written in the dialogue flow, and the movement pattern is determined or modified by the key words. In addition, in a text output control unit, words are displayed by switching between the display of a “conversation balloon” or the display of a “message board” according to the state of the dialogue written in the dialogue flow.

Claims

1. A human image dialogue device that is a system that carries out a dialogue with a user by producing a human image on a computer and generating this human image, characterized in providing:

a dialogue flow memory unit that stores a dialogue flow describing in a flow format the spoken text for the responses of said system and the state of the dialogue during this response;

a dialogue control unit that advances the responses of the user and the system by referring to the dialogue flow stored in said dialogue flow memory unit;

a dialogue flow analysis unit that carries out analysis of said dialogue flow to which said dialogue control unit refers; and

a human image generation unit that accepts the results of the analysis of said dialogue flow from said dialogue analysis unit, and generates the movements of said human image from said spoken text in said dialogue flow and the state of said dialogue.

2. A human image dialogue device according to claim 1, wherein said human image generation unit characterized in providing for generating the movements of said human image, at least:

a text movement association memory unit that stores the correspondences between the key words that may be contained in said dialogue text and the movement patterns;

a movement data memory unit that stores the movement data corresponding to said movement pattern; and

a movement-expression generation unit that generates movements of said human image from said spoken text in said dialogue flow and the state of said dialogue by referring to said text movement association memory unit and said movement data memory unit.

3. A human image dialogue device according to claim 2 characterized in further providing a text output control unit that carries out control in order that said human image generation unit represents said spoken text disclosed in said dialogue flow.

4. A human image dialogue device according to claim 3 characterized in that said text output control unit switches the display format of said dialogue text depending on the state of the dialogue disclosed in said dialogue flow.

5. A human image dialogue device according to claim 3 characterized in further providing a voice synthesis unit wherein said human image generation unit carries out and outputs the voice synthesis of said dialogue text disclosed in said dialogue flow.

6. A human image dialogue device according to claim 5 characterized in providing a synchronization unit that synchronizes the movements of the human image generated in said movement expression generation unit, the spoken text displayed by said text output control unit, and the voice synthesis output by said voice synthesis unit, and wherein the timing of the commencement of the movements of said human image, the spoken text, and the voice output are synchronized.

7. A human image dialogue device according to claim 4 characterized in further providing a voice synthesis unit wherein said human image generation unit carries out and outputs the voice synthesis of said dialogue text disclosed in said dialogue flow.

8. A recording medium that records a human image dialogue program that makes possible the generation of the human image by a computer in a system that carries out a dialogue with a user by providing a human image on the computer, and this computer making possible the generation of:

a dialogue flow memory function that stores a dialogue flow describing in a flow format the spoken text for the response of said system and the state of the dialogue at the time of the response;

a dialogue control function that advances the responses of the user and the system by referring to the dialogue flow stored by said dialogue flow memory function;

a dialogue flow analysis function that carries out analysis of said dialogue flow to which said dialogue control function refers; and

a human image generation function that receives the results of the analysis of said dialogue flow from said dialogue flow analysis function and generates the movements of said human image from said spoken text in said dialogue flow and the state of said dialogue.

9. A recording medium according to claim 8 that records a human image dialogue program wherein said human image generation function characterized in providing for generating the movements of said human image, at least:

a text movement association function unit that stores the correspondences between the key words that may be contained in said dialogue text and the movement patterns;

a movement data memory function that stores the movement data corresponding to said movement pattern; and

a movement-expression generation function that generates movements of said human image from said spoken text in said dialogue flow and the state of said dialogue by referring to said text movement association memory unit and said movement data memory unit.

10. A recording medium according to claim 9 that records a human image dialogue program characterized in further providing a text output control function that carries out control in order that said human image generation unit represents said spoken text disclosed in said dialogue flow.

11. A recording medium according to claim 10 that records a human image dialogue program characterized in that said text output control function switches the display format of said dialogue text depending on the state of the dialogue disclosed in said dialogue flow.

12. A recording medium according to claim 11 characterized in further providing a voice synthesis function wherein said human image generation function carries out and outputs the voice synthesis of said dialogue text disclosed in said dialogue flow.

13. A recording medium according to claim 10 characterized in further providing a voice synthesis function wherein said human image generation function carries out and outputs the voice synthesis of said dialogue text disclosed in said dialogue flow.

14. A recording medium according to claim 13 that records a human image dialogue program characterized in providing a synchronization function that synchronizes the movements of the human image generated in said movement expression generation function, the spoken text displayed by said text output control function, and the voice synthesis output by said voice synthesis function, and wherein the timing of the commencement of the movements of said human image, the spoken text, and the voice output are synchronized.