Voice and connection platform

• US 10,235,996 B2
• Filed: 09/30/2015
• Issued: 03/19/2019
• Est. Priority Date: 10/01/2014
• Status: Active Grant

• Current Assignee: XBrain, Inc.
• Original Assignee: XBrain, Inc.
• Inventors: Renard, Gregory, Herbaux, Mathias
• Primary Examiner(s): Yen, Eric
• Application Number:
  US14/871,272
• Publication Number:
  US 20160098992A1
• Time in Patent Office: 1,266 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A system and method for providing a voice assistant including receiving, at a first device, a first audio input from a user requesting a first action; performing automatic speech recognition on the first audio input; obtaining a context of user; performing natural language understanding based on the speech recognition of the first audio input; and taking the first action based on the context of the user and the natural language understanding.

Claims

1. A computer-implemented method comprising:

detecting an event;

responsive to detecting the event, proactively initiating a dialogue of a voice assistant on a first user device with a user;

responsive to initiating the dialogue with the user, receiving, at the first user device, a first audio input associated with the dialogue from the user requesting a first action;

performing automatic speech recognition on the first audio input;

determining, at the first user device, a first context of the user;

determining a first tuple describing user intent, the first tuple including the first action and an actor associated with the first action, the first tuple determined by performing natural language understanding based on the automatic speech recognition of the first audio input;

initiating the first action on the first user device based on the first tuple;

subsequent to initiating the first action, receiving a second audio input from the user requesting a second action unrelated to the first action;

initiating the second action;

subsequent to initiating the second action, receiving, at a second user device distinct from the first user device, a third audio input from the user continuing the dialogue and requesting a third action related to the first action, the third audio input missing information for completing a third tuple, the third tuple for initiating the third action;

obtaining the missing information using the first context to complete the third tuple associated with the third action; and

initiating the third action on the second user device based on the third tuple.

2. The computer-implemented method of claim 1, wherein the event is an internal event.

3. The computer-implemented method of claim 1, further comprising:

initiating the voice assistant without a user input and receiving the first audio input from the user subsequent to an initiation of the voice assistant.

4. The computer-implemented method of claim 1, wherein the first context includes one or more of a context history, a dialogue history, a user profile, a user history, a location, and a current context domain.

5. The computer-implemented method of claim 1, wherein the missing information is one or more of the third action, an actor associated with the third action, and an entity associated with the third action.

6. The computer-implemented method of claim 1, further comprising:

determining that the first context and the first audio input are missing first information used to initiate the first action;

determining what information is the missing first information; and

prompting the user to provide an audio input supplying the missing first information.

7. The computer-implemented method of claim 1, further comprising:

determining that first information used to initiate the first action is unable to be obtained from the first audio input;

determining what information is the missing first information; and

prompting the user to provide an audio input supplying the missing first information unable to be obtained from the first audio input.

8. The computer-implemented method of claim 1, further comprising:

determining that first information used to initiate the first action is unable to be obtained from the first audio input;

determining what information is the missing first information;

providing for selection by the user a plurality of options, an option supplying potential information for completing the first action; and

receiving an audio input selecting a first option from the plurality of options.

9. The computer-implemented method of claim 1, wherein the second action unrelated to the first action is associated with a second context, and the first action and the third action are associated with the first context.

10. A system comprising:

one or more processors; and

a memory storing instructions that when executed by the one or more processors, cause the system to perform steps including;

detect an event;

responsive to detecting the event, proactively initiate a dialogue of a voice assistant on a first user device with a user;

responsive to initiating the dialogue with the user, receive, at the first user device, a first audio input associated with the dialogue from the user requesting a first action;

perform automatic speech recognition on the first audio input;

determine, at the first user device, a first context of the user;

determine a first tuple describing user intent, the first tuple including the first action and an actor associated with the first action, the first tuple determined by performing natural language understanding based on the automatic speech recognition of the first audio input;

initiate the first action on the first user device based on the first tuple;

subsequent to initiating the first action, receive a second audio input from the user requesting a second action unrelated to the first action;

initiate the second action;

subsequent to initiating the second action, receive, at a second user device distinct from the first user device a third audio input from the user continuing the dialogue and requesting a third action related to the first action, the third audio input missing information for completing a third tuple, the third tuple for initiating the third action;

obtaining the missing information using the first context to complete the third tuple associated with the third action; and

initiating the third action on the second user device based on the third tuple.

11. The system of claim 10, wherein the event is an internal event.

12. The system of claim 10, wherein the instructions, when executed by the one or more processors, cause the system to:

initiate the voice assistant without a user input and receiving the first audio input from the user subsequent to an initiation of the voice assistant.

13. The system of claim 10, wherein the first context includes one or more of a context history, a dialogue history, a user profile, a user history, a location, and a current context domain.

14. The system of claim 10, wherein the missing information is one or more of the third action, an actor associated with the third action, and an entity associated with the third action.

15. The system of claim 10, wherein the instructions, when executed by the one or more processors cause the system to:

determine that the first context and the first audio input are missing first information used to initiate the first action;

determine what information is the missing first information; and

prompt the user to provide an audio input supplying the missing first information.

16. The system of claim 10, wherein the instructions, when executed by the one or more processors, cause the system to:

determine that first information used to initiate the first action is unable to be obtained from the first audio input;

determine what information is the missing first information; and

prompt the user to provide an audio input supplying the missing first information unable to be obtained from the first audio input.

17. The system of claim 10, wherein the instructions, when executed by the one or more processors, cause the system to:

determine that first information used to initiate the first action is unable to be obtained from the first audio input;

determine what information is the missing first information;

provide for selection by the user a plurality of options, an option supplying potential information for completing the first action; and

receive an audio input selecting a first option from the plurality of options.

18. The system of claim 10, wherein the second action unrelated to the first action is associated with a second context, and the first action and the third action are associated with the first context.