System and method for performing tasks based on user inputs using natural language processing

• US 10,747,954 B2
• Filed: 10/31/2017
• Issued: 08/18/2020
• Est. Priority Date: 10/31/2017
• Status: Active Grant

• Current Assignee: Baidu USA LLC (Baidu Incorporated)
• Original Assignee: Baidu USA LLC (Baidu Incorporated)
• Inventors: Yao, Xuchen
• Primary Examiner(s): Sirjani, Fariba
• Application Number:
  US15/799,934
• Publication Number:
  US 20190129938A1
• Time in Patent Office: 1,022 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06F 40/211 Syntactic parsing, e.g. based on context-free grammar [CFG] or unification grammars
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 40/40 Processing or translation of natural language natural language analysis G06F40/20; semantic analysis G06F40/30
• CPC Class Codes: H04L 12/282 based on user interaction within the home receiver circuitry for displaying additional information being controlled by a remote control apparatus H04N21/42204

Abstract

Embodiments of the invention provide a natural language understanding (NLU) system that performs relatively complex task automations via verbal or voice instructions. Normally, task automations have a trigger condition, and a series of one or more actions that would require a user'"'"'s selection of an option via manual input. This is because the underlying machine logic requires knowledge and classification capabilities from multiple domains that conventional personal assistants are not capable of. Embodiments of the inventive system and methods provide a solution to this complex task by analyzing trigger and action domains, pinpointing appropriate APIs, extracting corresponding API parameters, and automatically fulfilling API calls for the user. The system is configured to parse natural language commands into API calls. It analyzes both the trigger and action APIs and prompts a user for any missing information if necessary.

Claims

1. A computer-implemented method for automatically performing tasks based on user inputs, the method comprising:

in response to an input text originated from a user, performing a natural language processing (NLP) operation on the input text to determine a task to be performed in response to a condition relating to an event that may take place at a later time, wherein the condition is satisfied if and when the event takes place at the later time;

determining a first domain associated with the condition based on the NLP operation;

determining a second domain associated with the task to be performed based on the NLP operation;

determining a first application programming interface (API) associated with the first domain and a second API associated with the second domain;

parsing the input text to determine one or more parameters of the first API and the second API;

receiving a notification from a first application via the first API;

examining the notification received from the first API to determine whether the condition has been satisfied; and

in response to determining at the later time, that the condition has been satisfied, transmitting a control command to a second application via the second API, the control command requesting the second application to perform the task.

2. The method of claim 1, wherein determining a first domain associated with the condition and a second domain associated with the task comprises predicting the first domain and the second domain using a predetermined domain predictive model.

3. The method of claim 1, wherein determining the second API comprises:

searching in a domain-to-API mapping table based on the second domain to locate a mapping entry that matches the second domain; and

obtaining the second API from the matching mapping entry.

4. The method of claim 3, wherein the domain-to-API mapping table comprises a plurality of mapping entries, wherein each mapping entry maps a domain to one or more APIs.

5. The method of claim 3, further comprising:

determining whether there are multiple APIs corresponding to the second domain listed in the matching mapping entry; and

prompting the user to select one of the APIs associated with the second domain, wherein the control command is transmitted via the selected API.

6. The method of claim 3, further comprising:

determining whether a parameter of the control command is missing based on the NLP operation of the text input; and

in response to determining that there is a parameter missing from the control command, prompting the user to provide the missing parameter.

7. The method of claim 1, further comprising:

receiving a voice input having a voice command originated from the user; and

performing a speech recognition on the voice stream to transcribe the voice input to generate the input text.

8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

in response to an input text originated from a user, performing a natural language processing (NLP) operation on the input text to determine a task to be performed in response to a condition relating to an event that may take place at a later time, wherein the condition is satisfied if and when the event takes place at the later time;

determining a first domain associated with the condition based on the NLP operation;

determining a second domain associated with the task to be performed based on the NLP operation;

determining a first application programming interface (API) associated with the first domain and a second API associated with the second domain;

parsing the input text to determine one or more parameters of the first API and the second API;

receiving a notification from a first application via the first API;

examining the notification received from the first API to determine whether the condition has been satisfied; and

in response to determining, at the later time, that the condition has been satisfied, transmitting a control command to a second application via the second API, the control command requesting the second application to perform the task.

9. The machine-readable medium of claim 8, wherein determining a first domain associated with the condition and a second domain associated with the task comprises predicting the first domain and the second domain using a predetermined domain predictive model.

10. The machine-readable medium of claim 8, wherein determining the second API comprises:

searching in a domain-to-API mapping table based on the second domain to locate a mapping entry that matches the second domain; and

obtaining the second API from the matching mapping entry.

11. The machine-readable medium of claim 10, wherein the domain-to-API mapping table comprises a plurality of mapping entries, wherein each mapping entry maps a domain to one or more APIs.

12. The machine-readable medium of claim 10, wherein the operations further comprise:

determining whether there are multiple APIs corresponding to the second domain listed in the matching mapping entry; and

prompting the user to select one of the APIs associated with the second domain, wherein the control command is transmitted via the selected API.

13. The machine-readable medium of claim 10, wherein the operations further comprise:

determining whether a parameter of the control command is missing based on the NLP operation of the text input; and

in response to determining that there is a parameter missing from the control command, prompting the user to provide the missing parameter.

14. A data processing system, comprising:

a processor; and

a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including;

in response to an input text originated from a user, performing a natural language processing (NLP) operation on the input text to determine a task to be performed in response to a condition relating to an event that may take place at a later time, wherein the condition is satisfied if and when the event takes place at the later time,

determining a first domain associated with the condition based on the NLP operation,

determining a second domain associated with the task to be performed based on the NLP operation,

determining a first application programming interface (API) associated with the first domain and a second API associated with the second domain,

parsing the input text to determine one or more parameters of the first API and the second API,

receiving a notification from a first application via the first API;

examining the notification received from the first API to determine whether the condition has been satisfied, and

in response to determining, at the later time, that the condition has been satisfied, transmitting a control command to a second application via the second API, the control command requesting the second application to perform the task.

15. The system of claim 14, wherein determining a first domain associated with the condition and a second domain associated with the task comprises predicting the first domain and the second domain using a predetermined domain predictive model.

16. The system of claim 14, wherein determining the second API comprises:

searching in a domain-to-API mapping table based on the second domain to locate a mapping entry that matches the second domain; and

obtaining the second API from the matching mapping entry.

17. The system of claim 16, wherein the domain-to-API mapping table comprises a plurality of mapping entries, wherein each mapping entry maps a domain to one or more APIs.

18. The system of claim 16, wherein the operations further comprise:

determining whether there are multiple APIs corresponding to the second domain listed in the matching mapping entry; and

prompting the user to select one of the APIs associated with the second domain, wherein the control command is transmitted via the selected API.

19. The system of claim 16, wherein the operations further comprise:

determining whether a parameter of the control command is missing based on the NLP operation of the text input; and

in response to determining that there is a parameter missing from the control command, prompting the user to provide the missing parameter.