Device, method, and program for performing interaction between user and machine

• US 8,352,273 B2
• Filed: 07/26/2006
• Issued: 01/08/2013
• Est. Priority Date: 07/26/2005
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Nakano, Mikio
• Primary Examiner(s): Armstrong, Angela A
• Application Number:
  US11/989,409
• Publication Number:
  US 20100131277A1
• Time in Patent Office: 2,358 Days
• Field of Search: 704/231, 704/251, 704/270, 704/270.1, 704/275
• US Class Current: 704/270
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/228 of application context

Abstract

There is provided a device for performing interaction between a user and a machine. The device includes a plurality of domains corresponding to a plurality of stages in the interaction. Each of the domains has voice comprehension means which understands the content of the user'"'"'s voice. The device includes: means for recognizing the user'"'"'s voice; means for selecting a domain enabling the best voice comprehension results as ht domain; means for referencing task knowledge of the domain and extracting a task correlated to the voice comprehension result; means for obtaining a sub task sequence correlated to the extracted task; means for setting the first sub task of the sub task sequence as the sub task and updating the domain to which the sub task belongs as the domain; means for extracting a behavior or sub task end flag correlated to the voice comprehension result and the subtask; and means for causing the machine to execute the extracted behavior.

Claims

1. A device for performing interaction between a user and a machine, the device having a plurality of domains corresponding to multiple phases of the interaction, each domain including speech understanding means for understanding content of a speech of the user and outputting a speech understanding result, the device comprising:

means for recognizing the speech of the user from a signal detected by a microphone;

means for delivering the speech of the user to the respective speech understanding means, receiving the speech understanding result from each of the speech understanding means and selecting, as a relevant domain, the domain having the speech understanding means that produced an optimum speech understanding result;

means for referring to a task knowledge of the relevant domain out of task knowledges included in the respective multiple domains, and extracting a task associated with the speech understanding result;

means for referring to a subtask knowledge including a plurality of subtasks associated with the kind of task, and determining a sequence of subtasks associated with the extracted task;

means for determining a first subtask of the sequence of subtasks as a relevant subtask and updating, as the relevant domain, the domain to which the relevant subtask belongs;

means for referring to an action knowledge of the relevant domain out of action knowledges included in the respective multiple domains, and extracting a subtask completion flag or an action associated with the speech understanding result and the subtask; and

means for causing the machine to execute the extracted action.

2. The device according to claim 1, wherein the subtask knowledge includes a knowledge about at least one subtask associated with the task and a knowledge about the domain associated with the subtask.

3. The device according to claim 1, wherein each of the speech understanding means refers to a speech knowledge including a plurality of sentence patterns relevant to the corresponding domain, calculates a degree of adaptation between the speech and each of the plurality of sentence patterns, selects the sentence pattern having a highest degree of adaptation, and outputs the selected sentence pattern and the degree of adaptation of the sentence pattern as a speech understanding result.

4. The device according to claim 3, wherein the selecting means calculates a degree of reliability by multiplying the degree of adaptation by a weight set for each of the plurality of domains, and selects the domain having a highest degree of reliability as the relevant domain.

5. The device according to claim 1, wherein when the means for extracting an action or the subtask completion flag extracts the subtask completion flag, the updating means updates a subsequent subtask of the relevant subtask in the sequence of subtasks as another relevant subtask and updates, as the relevant domain, the domain to which the another relevant subtask belongs.

6. A method for performing interaction between a user and a machine, comprising the steps of:

recognizing a speech of the user from a signal detected by a microphone;

delivering the speech of the user to a plurality of domains corresponding to multiple phases of the interaction;

understanding content of the speech in each of the plurality of domains and outputting a speech understanding result;

receiving the speech understanding results respectively from the plurality of domains;

selecting, as a relevant domain, the domain that produced an optimum speech understanding result out of the plurality of speech understanding results;

referring to a task knowledge of the relevant domain out of task knowledges included in the respective multiple domains, and extracting a task associated with the speech understanding result;

referring to a subtask knowledge including a plurality of subtasks associated with the kind of task, and determining a sequence of subtasks associated with the extracted task;

determining a first subtask of the sequence of subtasks as a relevant subtask and updating, as the relevant domain, the domain to which the relevant subtask belongs;

referring to an action knowledge of the relevant domain out of action knowledges included in the respective multiple domains, and extracting a subtask completion flag or an action associated with the speech understanding result and the subtask; and

causing the machine to execute the extracted action.

7. The method according to claim 6, wherein the subtask knowledge includes a knowledge about at least one subtask associated with the task and a knowledge about the domain associated with the subtask.

8. The method according to claim 6, wherein the outputting step includes the steps of:

referring to a speech knowledge including a plurality of sentence patterns relevant to the corresponding domain, and calculating a degree of adaptation between the speech and each of the plurality of sentence patterns; and

selecting the sentence pattern having a highest degree of adaptation, and outputting the selected sentence pattern and the degree of adaptation of the sentence pattern as a speech understanding result.

9. The method according to claim 8, wherein the selecting step includes the step of calculating a degree of reliability by multiplying the degree of adaptation by a weight set for each of the plurality of domains, and selecting the domain having a highest degree of reliability as the relevant domain.

10. The method according to claim 6, wherein the updating step includes the step of, when the subtask completion flag is extracted in the step of extracting an action or the subtask completion flag, updating a subsequent subtask of the relevant subtask in the sequence of subtasks as another relevant subtask and updating, as the relevant domain, the domain to which the another relevant subtask belongs.

11. A non-transitory computer readable medium storing a computer program for performing interaction between a user and a machine, the program causing a computer to perform the functions of:

recognizing a speech of the user from a signal detected by a microphone;

delivering the speech of the user to a plurality of domains corresponding to multiple steps of the interaction with the user;

understanding content of the speech in each of the plurality of domains and outputting a speech understanding result;

receiving the speech understanding results respectively from the plurality of domains;

selecting, as a relevant domain, the domain having an optimum speech understanding result out of the plurality of speech understanding results;

referring to a task knowledge of the relevant domain out of task knowledges included in the respective multiple domains, and extracting a task associated with the speech understanding result;

referring to a subtask knowledge including a plurality of subtasks associated with the kind of task, and determining a sequence of subtasks associated with the extracted task;

determining a first subtask of the sequence of subtasks as a relevant subtask and updating, as the relevant domain, the domain to which the relevant subtask belongs;

referring to an action knowledge of the relevant domain out of action knowledges included in the respective multiple domains, and extracting a subtask completion flag or an action associated with the speech understanding result and the subtask; and

causing the machine to execute the extracted action,

the program being recorded on a recording medium readable by the computer.

12. The non-transitory computer readable medium according to claim 11, wherein the subtask knowledge includes a knowledge about at least one subtask associated with the task and a knowledge about the domain associated with the subtask.

13. The non-transitory computer readable medium according to claim 11, wherein the outputting function includes the steps of:

referring to a speech knowledge including a plurality of sentence patterns relevant to the corresponding domain, calculating a degree of adaptation between the speech and each of the plurality of sentence patterns; and

selecting the sentence pattern having a highest degree of adaptation, and outputting the selected sentence pattern and the degree of adaptation of the sentence pattern as a speech understanding result.

14. The non-transitory computer readable medium according to claim 13, wherein the selecting function includes the function of calculating a degree of reliability by multiplying the degree of adaptation by a weight set for each of the plurality of domains, and selecting the domain having a highest degree of reliability as the relevant domain.

15. The non-transitory computer readable medium according to claim 11, wherein the updating function includes the function of, when the subtask completion flag is extracted in the function of extracting an action or the subtask completion flag, updating a subsequent subtask of the relevant subtask in the sequence of subtasks as another relevant subtask, and updating, as the relevant domain, the domain to which the another relevant subtask belongs.