Method and apparatus for managing natural language queries of customers

• US 10,572,516 B2
• Filed: 12/13/2016
• Issued: 02/25/2020
• Est. Priority Date: 12/15/2015
• Status: Active Grant

• Current Assignee: [24]7.ai, Inc.
• Original Assignee: [24]7.ai, Inc.
• Inventors: Walia, Anmol, Zuverink, David J.
• Primary Examiner(s): Willoughby, Alicia M
• Application Number:
  US15/377,038
• Publication Number:
  US 20170169101A1
• Time in Patent Office: 1,169 Days
• CPC Class Codes: G06F 16/3329 Natural language query formulation or dialogue systems
• CPC Class Codes: G06F 16/3344 using natural language analysis
• CPC Class Codes: G06Q 30/016 After-sales

Abstract

A computer-implemented method and an apparatus manage natural language queries of customers. A natural language query provided by a customer on an enterprise interaction channel is received. The natural language query is analyzed to determine if an answer to the natural language query exists in at least one question-answer (QA) domain from among a plurality of QA domains by analyzing each QA domain from among the plurality of QA domains using a multi-level framework of natural language models. An answer to the natural language query is provided to the customer on the enterprise interaction channel if such an answer in available in the plurality of QA domains. If an answer is not available, then an appropriate response is provided to the customer to assist the customer.

Claims

1. A computer-implemented method, comprising:

receiving, by a processor, a natural language query provided by a customer on an enterprise interaction channel;

performing a multi-step search by;

in a first step of the multi-step search, determining, by the processor, when a question-answer (QA) domain is relevant to the natural language query, the relevancy of the QA domain determined from among a plurality of QA domains by analyzing each QA domain from among the plurality of QA domains using a top-level natural language model associated with a multi-level framework of natural language models, wherein each QA domain is associated with one or more sub-domains and each sub-domain from among the one or more sub-domains is associated with at least one respective lower-level natural language model;

in a second step subsequent to the first step of the multi-step search, for the QA domain determined to be relevant to the natural language query, identifying, by the processor, at least one sub-domain comprising one or more questions substantially similar to the natural language query, the at least one sub-domain identified by analyzing questions associated with the each sub-domain of the QA domain in relation to the natural language query, the questions analyzed using the at least one respective lower-level natural language model;

in a third step subsequent to the second step, determining, by the processor, when at least one question from among the one or more questions of the at least one sub-domain is associated with a query matching metric of greater than a predefined threshold value, wherein the query matching metric is generated by comparing individual words and a sequence of words of the natural language query with the one or more questions; and

effecting, by the processor, a provisioning of a response to the customer on the enterprise interaction channel when a determination is made that the at least one question from among the one or more questions is associated with the query matching metric of greater than the predefined threshold value.

2. The method of claim 1, further comprising:

effecting provisioning of at least one answer as the response to the customer when the at least one question is associated with the query matching metric of greater than the predefined threshold value, the at least one answer corresponding to the at least one question.

3. The method of claim 1, further comprising:

effecting provisioning of a link to an enterprise webpage as the response to the customer query when it is determined that there is no QA domain among the plurality of QA domains that is relevant to the natural language query.

4. The method of claim 1, further comprising:

effecting provisioning of an offer for agent assistance as the response to the customer query when it is determined that there is no QA domain among the plurality of QA domains that is relevant to the natural language query.

5. The method of claim 4, further comprising:

facilitating, by the processor, an interaction between the customer and an agent subsequent to receiving an acceptance of the offer for agent assistance from the customer.

6. The method of claim 5, wherein the agent is any of an automated agent, a smart virtual assistant, and a human agent.

7. The method of claim 5, further comprising:

effecting, by the processor, a provisioning of a request to the customer to provide a rating for the agent subsequent to the customer'"'"'s interaction with the agent.

8. The method of claim 4, further comprising:

effecting, by the processor, a provisioning of a message indicative of an unavailability of an answer to the natural language query when an acceptance of the offer for agent assistance is not received from the customer.

9. The method of claim 1, further comprising:

effecting provisioning of a follow-up question when no question from among the one or more questions is associated with the query matching metric of greater than the predefined threshold value, the follow-up question configured to seek clarification on the natural language query from the customer.

10. The method of claim 9, further comprising:

receiving, by the processor, a reply from the customer as a response to the follow-up question; and

repeating, by the processor, the steps of;

determining a relevant QA domain from among the plurality of QA domains;

identifying a sub-domain from the relevant QA domain comprising the one or more questions substantially similar to the natural language query;

determining when the at least one question from among the one or more questions is associated with the query matching metric of greater than the predefined threshold value; and

effecting a provisioning of a response to the customer.

11. The method of claim 1, wherein the provisioning of the response is effected based on one or more predefined business rules.

25. The method of claim 1, wherein the relevancy of the QA domain is determined by:

parsing the natural language query to generate n-grams;

employing the top-level natural language model to calculate a correspondence of the natural language query with the plurality of QA domains using one or more of the n-grams;

employing a lower-level natural language model to calculate a correspondence of the natural language query with a plurality of sub-domains in the QA domain using one or more of the n-grams;

determining a relevance of questions in the at least one sub-domain to the natural language query by computing the query matching metric for the questions in the at least one sub-domain to determine if any of the query matching metrics exceed the predefined threshold value; and

upon determining that at least one of the query matching metrics exceeds the predefined threshold value, designating the QA domain and sub-domain as relevant.

12. An apparatus, comprising:

at least one processor; and

a memory having stored therein machine executable instructions, that when executed by the at least one processor, cause the apparatus to;

receive a natural language query provided by a customer on an enterprise interaction channel;

perform a multi-step search by;

in a first step of the multi-step search, determine when a question-answer (QA) domain is relevant to the natural language query, the relevancy of the QA domain determined from among a plurality of QA domains by analyzing each QA domain from among the plurality of QA domains using a top-level natural language model associated with a multi-level framework of natural language models, wherein each QA domain is associated with one or more sub-domains and each sub-domain from among the one or more sub-domains is associated with at least one respective lower-level natural language model;

in a second step subsequent to the first step of the multi-step search, for the QA domain determined to be relevant to the natural language query, identify at least one sub-domain comprising one or more questions substantially similar to the natural language query, the at least one sub-domain identified by analyzing questions associated with the each sub-domain of the QA domain in relation to the natural language query, the questions analyzed using the at least one respective lower level natural language model;

in a third step subsequent to the second step of the multi-step search, determine when at least one question from among the one or more questions of the at least one sub-domain is associated with a query matching metric of greater than a predefined threshold value, wherein the query matching metric is generated by comparing individual words and a sequence of words of the natural language query with the one or more questions; and

effect a provisioning of a response to the customer on the enterprise interaction channel when a determination is made that the at least one question from among the one or more questions is associated with the query matching metric of greater than the predefined threshold value.

13. The apparatus of claim 12, wherein the apparatus is further caused to:

effect provisioning of at least one answer as the response to the customer when the at least one question is associated with the query matching metric of greater than the predefined threshold value, the at least one answer corresponding to the at least one question.

14. The apparatus of claim 12, wherein the apparatus is further caused to:

effect provisioning of a link to an enterprise webpage as the response to the customer when it is determined that there is no QA domain from among the plurality of QA domains that is relevant to the natural language query.

15. The apparatus of claim 12, wherein the apparatus is further caused to:

effect provisioning of an offer for agent assistance as the response to the customer when it is determined that there is no QA domain from among the plurality of QA domains that is relevant to the natural language query; and

facilitate an interaction with an agent subsequent to receiving an acceptance of the offer for agent assistance from the customer.

16. The apparatus of claim 15, wherein the apparatus is further caused to:

effect a provisioning of a message indicative of an unavailability of an answer to the natural language query when the acceptance of the offer for agent assistance is not received from the customer.

17. The apparatus of claim 12, wherein the apparatus is further caused to:

effect provisioning of a follow-up question when it is determined that there is no question from among the one or more questions that is associated with the query matching metric of greater than the predefined threshold value, the follow-up question configured to seek clarification on the natural language query from the customer.

18. The apparatus of claim 17, wherein the apparatus is further caused to:

receive a reply from the customer as a response to the follow-up question; and

repeat the steps of;

determining a relevant QA domain from among the plurality of QA domains;

identifying a sub-domain from the relevant QA domain comprising the one or more questions substantially similar to the natural language query;

determining when the at least one question from among the one or more questions is associated with a matching metric of greater than the predefined threshold value; and

effecting a provisioning of a response to the customer.

19. A computer-implemented method, comprising:

causing, by a processor, display of a chat widget offering chat assistance on one or more webpages of an enterprise website;

in response to customer input corresponding to the chat widget on a webpage of the enterprise website, causing display of a dialog screen, by the processor, on the webpage;

receiving, by the processor, a natural language query provided as an input by a customer in the dialog screen;

performing a multi-step search by;

in a first step of the multi-step search, determining, by the processor, when an answer to the natural language query exists in at least one question-answer (QA) domain from among a plurality of QA domains by analyzing each QA domain from among the plurality of QA domains using a hierarchical framework of natural language models in a second step subsequent to the first step, determining, by the processor, when at least one question from among one or more questions of the at least one QA domain is associated with a query matching metric of greater than a predefined threshold value, wherein the query matching metric is generated by comparing individual words and a sequence of words of the natural language query with the one or more questions; and

effecting, by the processor, a provisioning of a response to the customer in the dialog screen when a determination is made that the at least one question from among the one or more questions is associated with the query matching metric of greater than the predefined threshold value.

20. The method of claim 19, wherein determining whether the answer to the natural language query exists in the plurality of QA domains comprises:

determining when the QA domain is relevant to the natural language query, the relevancy of the QA domain determined from among the plurality of QA domains by analyzing each QA domain from among the plurality of QA domains using a top-level natural language model associated with the hierarchical framework of natural language models, wherein the each QA domain is associated with one or more sub-domains and each sub-domain from among the one or more sub-domains is associated with at least one respective lower-level natural language model; and

for the QA domain determined to be relevant to the natural language query, identifying, at least one sub-domain comprising one or more questions substantially similar to the natural language query, the at least one sub-domain identified by analyzing questions associated with the each sub-domain of the QA domain in relation to the natural language query, the questions analyzed using the at least one respective lower-level natural language model.

21. The method of claim 20, further comprising:

effecting provisioning of a follow-up question when it is determined that there no question from among the one or more questions that is associated with the query matching metric of greater than the predefined threshold value, the follow-up question configured to seek clarification on the natural language query from the customer.

22. The method of claim 21, further comprising:

receiving, by the processor, a reply from the customer as a response to the follow-up question; and

repeating, by the processor, the steps of;

determining a relevant QA domain from among the plurality of QA domains;

identifying a sub-domain from the relevant QA domain comprising the one or more questions substantially similar to the natural language query;

determining when the at least one question from among the one or more questions is associated with a matching metric of greater than the predefined threshold value; and

effecting a provisioning of a response to the customer.

23. The method of claim 20, further comprising:

effecting provisioning of an offer to the customer to interact with a live agent as the response to the customer when it is determined that the answer to the natural language query does not exist in the plurality of QA domains.

24. The method of claim 23, further comprising:

effecting, by the processor, a provisioning of a message indicative of an unavailability of the answer to the natural language query when the customer does not accept the offer to interact with the live agent.