Voice and connection platform

• US 10,789,953 B2
• Filed: 02/15/2019
• Issued: 09/29/2020
• Est. Priority Date: 10/01/2014
• Status: Active Grant

• Current Assignee: XBrain, Inc.
• Original Assignee: XBrain, Inc.
• Inventors: Renard, Gregory, Herbaux, Mathias
• Primary Examiner(s): Yen, Eric
• Application Number:
  US16/277,844
• Publication Number:
  US 20190180750A1
• Time in Patent Office: 592 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/18 using natural language modelling
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 15/30 Distributed recognition, e.g. in client-server systems, for mobile phones or network applications
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 2015/226 using non-speech characteristics
• CPC Class Codes: G10L 2015/227 of the speaker; Human-factor methodology
• CPC Class Codes: G10L 2015/228 of application context

Abstract

A system and method for providing a voice assistant including receiving, at a first device, a first audio input from a user requesting a first action; performing automatic speech recognition on the first audio input; obtaining a context of user; performing natural language understanding based on the speech recognition of the first audio input; and taking the first action based on the context of the user and the natural language understanding.

Claims

1. A computer-implemented method comprising:

receiving, at a first user device, a first audio input from a user, the first audio input associated with a dialogue and requesting a first action;

determining, at the first user device, a first context of the user;

initiating the first action on the first user device;

subsequent to initiating the first action, receiving, at a second user device distinct from the first user device, a second audio input from the user continuing the dialogue and requesting a second action related to the first action;

determining a tuple describing user intent associated with the second action based on the second audio input, the tuple missing an element;

determining the missing element of the tuple using the first context to complete the tuple;

initiating the second action on the second user device based on the completed tuple;

determining, using a machine learning model, an action pattern of the user based on the first context of the user and one or more past contexts of the user;

predicting a third action of the user based on the action pattern of the user; and

proactively providing to the user a recommendation based on the predicted third action of the user.

2. The computer-implemented method of claim 1, wherein the first context includes one or more of a context history, a dialogue history, a user profile, a user history, a location, and a current context domain.

3. The computer-implemented method of claim 1, further comprising:

determining a dictionary based on the first context of the user; and

determining a first tuple describing user intent associated with the first action by performing Natural Language Understanding (NLU) on the first audio input using the dictionary; and

initiating the first action on the first user device based on the first tuple.

4. The computer-implemented method of claim 1, further comprising:

determining a first tuple describing user intent associated with the first action based on the first context of the user and one or more past contexts of the user; and

initiating the first action on the first user device based on the first tuple.

5. The computer-implemented method of claim 1, further comprising:

detecting an event; and

responsive to detecting the event, proactively initiating the dialogue, wherein the dialogue is between a voice assistant and the user and is performed using the first user device.

6. The computer-implemented method of claim 5, wherein the event includes one or more of an internal event associated with the first user device, an external event associated with the user, an input of a wake-up phrase, an input of a user input, and an expiration of a timer.

7. The computer-implemented method of claim 1, further comprising:

proactively initiating the dialogue based on a second action pattern.

8. The computer-implemented method of claim 1, further comprising:

subsequent to initiating the first action, receiving a third audio input from the user requesting the third action, wherein the third action is unrelated to the first action;

initiating the third action;

subsequent to initiating the third action, receiving a fourth audio input from the user continuing the dialogue and requesting a fourth action related to the first action;

determining a fourth tuple describing user intent associated with the fourth action based on the fourth audio input, the fourth tuple missing another element;

determining the another element of the fourth tuple using the first context to complete the fourth tuple; and

initiating the fourth action based on the completed fourth tuple.

9. The computer-implemented method of claim 1, further comprising:

creating an input shortcut based on the second audio input and the completed tuple.

10. A system comprising:

one or more processors; and

a memory storing instructions that when executed by the one or more processors, cause the system to perform steps including;

receive, at a first user device, a first audio input from a user, the first audio input associated with a dialogue and requesting a first action;

determine, at the first user device, a first context of the user;

initiate the first action on the first user device;

subsequent to initiating the first action, receive, at a second user device distinct from the first user device, a second audio input from the user continuing the dialogue and requesting a second action related to the first action;

determine a tuple describing user intent associated with the second action based on the second audio input, the tuple missing an element;

determine the missing element of the tuple using the first context to complete the tuple;

initiate the second action on the second user device based on the completed tuple;

determine, using a machine learning model, an action pattern of the user based on the first context of the user and one or more past contexts of the user;

predict a third action of the user based on the action pattern of the user; and

proactively provide to the user a recommendation based on the predicted third action of the user.

11. The system of claim 10, wherein the first context includes one or more of a context history, a dialogue history, a user profile, a user history, a location, and a current context domain.

12. The system of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to:

determine a dictionary based on the first context of the user; and

determine a first tuple describing user intent associated with the first action by performing Natural Language Understanding (NLU) on the first audio input using the dictionary; and

initiate the first action on the first user device based on the first tuple.

13. The system of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to:

determine a first tuple describing user intent associated with the first action based on the first context of the user and one or more past contexts of the user; and

initiate the first action on the first user device based on the first tuple.

14. The system of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to:

detect an event; and

responsive to detecting the event, proactively initiate the dialogue, wherein the dialogue is between a voice assistant and the user and is performed using the first user device.

15. The system of claim 14, wherein the event includes one or more of an internal event associated with the first user device, an external event associated with the user, an input of a wake-up phrase, an input of a user input, and an expiration of a timer.

16. The system of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to:

proactively initiate the dialogue based on a second action pattern.

17. The system of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to:

subsequent to initiating the first action, receive a third audio input from the user requesting the third action, wherein the third action is unrelated to the first action;

initiate the third action;

subsequent to initiating the third action, receive a fourth audio input from the user continuing the dialogue and requesting a fourth action related to the first action;

determine a fourth tuple describing user intent associated with the fourth action based on the fourth audio input, the fourth tuple missing another element;

determine the another element of the fourth tuple using the first context to complete the fourth tuple; and

initiate the fourth action based on the completed fourth tuple.

18. The system of claim 10, comprising instructions that, when executed by the one or more processors, cause the system to:

create an input shortcut based on the second audio input and the completed tuple.