Multi modal interactive system, method, and medium

• US 6,513,011 B1
• Filed: 06/02/2000
• Issued: 01/28/2003
• Est. Priority Date: 06/04/1999
• Status: Expired due to Term

• Current Assignee: Rakuten, Inc. (Rakuten Group, Inc.)
• Original Assignee: NEC Corporation
• Inventors: Uwakubo, Shinichi
• Primary Examiner(s): MCFADDEN, SUSAN IRIS
• Application Number:
  US09/585,288
• Time in Patent Office: 970 Days
• Field of Search: 704/270,27 S,272,257,9 707/1
• US Class Current: 704/275
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue

Abstract

A multi modal interactive device which can supply a conversation between the device and user with proper length of a pause which is determined according to types of the user.

Claims

1. A multi modal interactive device comprising:

an input unit which inputs information related to a user;

a recognition unit which recognizes the information obtained by the input unit;

an integrate process unit which determines an intention of the user from the recognition result from the recognition unit;

a reaction generating unit which generates a reaction which corresponds to the intention of the user;

a storing device which stores a timing for each user state;

a conversation managing unit which determines a timing on the basis of the user state with reference to the storing device; and

an output unit which outputs the reaction to the user based on the determined timing.

2. The multi modal interactive device of claim 1, wherein the timing is defined as a time period from a time instant when the input unit inputs the information to a time instant when the output unit outputs the corresponding reaction.

3. The multi modal interactive device of claim 1, wherein the user state includes one of sex, age, the number of use times of the multi modal interactive device, and personality of the user.

4. The multi modal interactive device of claim 3, wherein the user state is identified from the information obtained by the input unit.

5. The multi modal interactive device of claim 3, wherein the user state is identified from the ID card which is presented to the multi modal interactive device.

6. The multi modal interactive device of claim 1 further comprising:

a conversation analyze unit which analyzes a transition of a conversation between the multi modal interactive device and the user,

wherein the timing determine unit determines the timing on the basis of the user state and the analyzed results.

7. The multi modal interactive device of claim 1 further comprising:

one or more additional reaction generating units; and

a reaction select unit which selects a reaction to be sent to the output unit among a group of reactions from the plurality of reaction generating units,

wherein the reaction select unit selects the reaction to be sent on a timing so that the reaction is presented by the output unit based on the determined timing.

8. The multi modal interactive device of claim 7, wherein each reaction in the group of reactions is generated until a predetermined time interval before the determined timing.

9. The multi modal interactive device of claim 1 further comprising:

one or more pairs of the input unit and the recognition unit, the recognition unit is related to the input unit to recognize the information from the corresponding input unit.

10. A method of providing with a multi modal conversation, the method comprising the steps of:

receiving information related to a user;

recognizing the information;

determining an intention of the user from the recognition result;

generating a reaction which corresponds to the intention of the user;

preparing a timing for each user state;

determining a timing on the basis of the user state; and

supplying the user with the reaction based on the determined timing.

11. The method of claim 10, wherein two or more reactions are generated by the step of generating a reaction, and a reaction to be presented to the user is selected among the generated reactions so that the selected reaction is presented based on the determined timing.

12. A recording medium readable by a computer, tangibly embodying a program of instructions executable by the computer to perform a method of providing with a multi modal conversation, the method comprising the steps of:

receiving information related to a user;

recognizing the information;

determining an intention of the user from the recognition result;

generating a reaction which corresponds to the intention of the user;

preparing a timing for each user state;

determining a timing on the basis of the user state; and

supplying the user with the reaction based on the determined timing.