System and method for a cooperative conversational voice user interface

• US 10,515,628 B2
• Filed: 05/20/2019
• Issued: 12/24/2019
• Est. Priority Date: 10/16/2006
• Status: Active Grant

• Current Assignee: VB Assets, LLC
• Original Assignee: VB Assets, LLC
• Inventors: Baldwin, Larry, Freeman, Tom, Tjalve, Michael, Ebersold, Blane, Weider, Chris
• Primary Examiner(s): Yen, Eric
• Application Number:
  US16/417,173
• Publication Number:
  US 20190272822A1
• Time in Patent Office: 218 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/183 using context dependencies, e.g. language models
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 17/22 Interactive procedures; Man-machine interfaces
• CPC Class Codes: G10L 2015/0631 Creating reference templates; Clustering
• CPC Class Codes: G10L 2015/225 Feedback of the input speech
• CPC Class Codes: G10L 2015/228 of application context
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/63 for estimating an emotional state

Abstract

A cooperative conversational voice user interface is provided. The cooperative conversational voice user interface may build upon short-term and long-term shared knowledge to generate one or more explicit and/or implicit hypotheses about an intent of a user utterance. The hypotheses may be ranked based on varying degrees of certainty, and an adaptive response may be generated for the user. Responses may be worded based on the degrees of certainty and to frame an appropriate domain for a subsequent utterance. In one implementation, misrecognitions may be tolerated, and conversational course may be corrected based on subsequent utterances and/or responses.

Claims

1. A computer-implemented method of facilitating natural language system responses utilizing accumulated short-term and long-term knowledge, the method being implemented by a computer system that includes one or more physical processors executing one or more computer program instructions which, when executed, perform the method, the method comprising:

accumulating, by the computer system, short-term knowledge based on one or more natural language utterances received during a predetermined time period;

expiring, by the computer system, one or more items of short-term knowledge that are based on one or more natural language utterances received prior to the predetermined time period;

accumulating, by the computer system, long-term knowledge based on one or more natural language utterances received prior to the predetermined time period, wherein the long-term knowledge includes at least one of the one or more expired items of short-term knowledge, first long-term knowledge associated with a first user based on one or more natural language utterances received from the first user, and second long-term knowledge associated with a second user based on one or more natural language utterances received from the second user;

receiving, at the computer system, a first natural language utterance via an input device;

determining, by the computer system, based on the short-term knowledge and the long-term knowledge, a first context for the first natural language utterance;

determining, by the computer system, based on the first context, an interpretation of the first natural language utterance; and

generating, by the computer system, a first response to the first natural language utterance based on the interpretation.

2. The method of claim 1, wherein the one or more natural language utterances received during the predetermined time period are related to a single conversation between a user and the computer system.

3. The method of claim 1, wherein the long-term knowledge comprises one or more of user preferences, an indication of utilized domain agents, contexts of prior natural language utterances, requests of prior natural language utterances, and tasks performed in response to prior natural language utterances.

4. The method of claim 1, the method further comprising:

receiving, by the computer system, a second natural language utterance in response to the first response;

identifying, by the computer system, one or more user preferences based on the second natural language utterance; and

updating, by the computer system, the long-term knowledge based on the identified user preferences.

5. The method of claim 1, wherein the long-term knowledge is associated with a first user, the method further comprising:

generating, by the computer system, a profile associated with the first user based on the long-term knowledge, wherein the first context for the first natural language utterance is further determined based on the profile associated with the first user.

6. The method of claim 1, the method further comprising:

generating, by the computer system, one or more of a user profile, an environmental profile, a historical profile, and/or a cognitive profile based on the long-term knowledge, wherein subsequent natural language utterances are processed based at least on the one or more of the user profile, the environmental profile, the historical profile, and the cognitive profile.

7. The method of claim 1, the method further comprising:

receiving, by the computer system, information from a remote data source, wherein the long-term knowledge is based further on the information obtained from the remote data source.

8. The method of claim 1, wherein determining the interpretation of the first natural language utterance comprises:

determining, by the computer system, an interpretation of one or more recognized words of the first natural language utterance based on the first context.

9. The method of claim 1, the method further comprising:

causing, by the computer system, the response to the first natural language utterance to be provided to a user.

10. A system for facilitating natural language system responses utilizing accumulated short-term and long-term knowledge, the system comprising:

one or more physical processors programmed with one or more computer program instructions which, when executed, configure the one or more physical processors to;

accumulate short-term knowledge based on one or more natural language utterances received during a predetermined time period;

expire one or more items of short-term knowledge that are based on one or more natural language utterances received prior to the predetermined time period;

accumulate long-term knowledge based on one or more natural language utterances received prior to the predetermined time period, wherein the long-term knowledge includes at least one of the one or more expired items of short-term knowledge, first long-term knowledge associated with a first user based on one or more natural language utterances received from the first user, and second long-term knowledge associated with a second user based on one or more natural language utterances received from the second user;

receive a first natural language utterance via an input device;

determine a first context for the first natural language utterance based on the short-term knowledge and the long-term knowledge;

determine an interpretation of the first natural language utterance based on the first context; and

generate a first response to the first natural language utterance based on the interpretation.

11. The system of claim 10, wherein the one or more natural language utterances received during the predetermined time period are related to a single conversation between a user and the computer system.

12. The system of claim 10, wherein the long-term knowledge comprises one or more of user preferences, an indication of utilized domain agents, contexts of prior natural language utterances, requests of prior natural language utterances, and tasks performed in response to prior natural language utterances.

13. The system of claim 10, wherein the one or more physical processors are further configured to:

receive a second natural language utterance in response to the first response;

identify one or more user preferences based on the second natural language utterance; and

update the long-term knowledge based on the identified user preferences.

14. The system of claim 10, wherein the long-term knowledge is associated with a first user, and wherein the one or more physical processors are further configured to:

generate a profile associated with the first user based on the long-term knowledge, wherein the first context for the first natural language utterance is further determined based on the profile associated with the first user.

15. The system of claim 10, wherein the one or more physical processors are further configured to:

generate one or more of a user profile, an environmental profile, a historical profile, and/or a cognitive profile based on the long-term knowledge, wherein subsequent natural language utterances are processed based at least on the one or more of the user profile, the environmental profile, the historical profile, and the cognitive profile.

16. The system of claim 10, wherein the one or more physical processors are further configured to:

receive information from a remote data source, wherein the long-term knowledge is based further on the information obtained from the remote data source.

17. The system of claim 10, wherein to determine the interpretation of the first natural language utterance, the one or more physical processors are configured to:

determine an interpretation of one or more recognized words of the first natural language utterance based on the first context.

18. The system of claim 10, wherein the one or more physical processors are further configured to:

cause the response to the first natural language utterance to be provided to a user.