Method and system for performing a probabilistic topic analysis of search queries for a customer support system

• US 10,242,093 B2
• Filed: 10/29/2015
• Issued: 03/26/2019
• Est. Priority Date: 10/29/2015
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Podgorny, Igor A., Bartolome, Warren, Patterson, Nolan Richard
• Primary Examiner(s): Mackes, Kris E
• Assistant Examiner(s): Bui, Tiffany Thuy
• Application Number:
  US14/926,232
• Publication Number:
  US 20170124184A1
• Time in Patent Office: 1,244 Days
• CPC Class Codes: G06F 16/3322 using system suggestions G06F16/3325 takes precedence
• CPC Class Codes: G06F 16/338 Presentation of query results
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/016 After-sales

Abstract

A method and system uses a probabilistic topic model to identify topics that are relevant search query terms received by a customer support system from a user, according to one embodiment. The probabilistic topic model identifies topics that are relevant to the search query terms at least partially based on the context around the receipt of the search query terms, according to one embodiment. By identifying relevant topics at least partially based on the context around the receipt of the search query terms, a likelihood of inaccurately identifying a relevant topic is reduced, according to one embodiment.

Claims

1. A method for improving a likelihood of user satisfaction with a customer support response that is provided by a customer support system in response to receiving search query terms, by using context information to reduce a likelihood of inaccurately identifying a relevant topic for the search query terms with a probabilistic topic model, the method comprising:

receiving search query terms data representing one or more search query terms received by a customer support system with one or more computing systems, from a current user;

applying the search query terms data to a probabilistic topic model to identify topics data representing one or more topics that are relevant to the one or more search query terms, and to determine topic relevance scores data for the topics data representing the one or more topics, the topic relevance scores data representing one or more topic relevance scores that quantify a likelihood of relevance between the one or more topics and the one or more search query terms received from the current user;

generating length data for the search query terms data, the length data for the search query terms data representing a combined length of the one or more search query terms;

comparing the length data for the search query terms data to search query length threshold data representing a search query length threshold, below which a likelihood of inaccuracy increases for the probabilistic topic model;

upon determining that the combined length of the one or more search query terms is less than the search query length threshold, updating the topic relevance scores data representing the one or more topic relevance scores with context characteristics probabilities data to reduce the likelihood of inaccuracy for the probabilistic topic model, the context characteristics probabilities data representing one or more context characteristics probabilities that quantify a likelihood that a question about the one or more topics occurs while one or more context characteristics for the search query terms exist, wherein updating the topic relevance scores data includes;

identifying context characteristics data representing the context characteristics for the search query terms, the context characteristics data at least including data representing identification of user experience displays visited by the current user, the user experience displays being provided by one or more service provider systems associated with the customer support system;

training the probabilistic topic model using a Latent Dirichlet algorithm, wherein training the probabilistic topic model includes;

retrieving existing customer support content data;

applying the Latent Dirichlet algorithm to existing customer support content data representing existing customer support content, to generate a predetermined number of topics, to generate a plurality of topic terms that are relevant to each of the predetermined number of topics, and to generate a plurality of topic term probabilities associated with the plurality of topic terms; and

storing predetermined number of topics data representing the predetermined number of topics, plurality of topic terms data representing the plurality of topic terms, and plurality of topic term probabilities data representing the plurality of topic term probabilities, in a topics data structure for the customer support system;

applying the context characteristics data to the probabilistic topic model to generate the context characteristics probabilities data representing the one or more context characteristics probabilities; and

combining the context characteristics probabilities data with the topic relevance scores data to update the topic relevance scores data to reflect a combination of the context characteristics probabilities data and the topic relevance scores data;

selecting a relevant topic from the one or more topics that is likely most relevant to the one or more search query terms, at least partially based on a highest one of the one or more topic relevance scores represented by the topic relevance scores data; and

providing a personalized customer support response to the current user for the search query terms, at least partially based on the relevant topic, to increase a likelihood of customer satisfaction of the current user with a user experience within the customer support system, by reducing a likelihood of inaccurately identifying the relevant topic for the search query terms received by the customer support system.

2. The method of claim 1, wherein the length of the one or more search query terms is one of a character length and a word length.

3. The method of claim 1, wherein the probabilistic topic model includes a Latent Dirichlet algorithm.

4. The method of claim 1, wherein the topics data structure for the customer support system is part of the probabilistic topic model.

5. The method of claim 1, wherein applying the search query terms data to the probabilistic topic model to identify the topics data representing the one or more topics that are relevant to the one or more search query terms, and to determine topic relevance scores data for the topics data representing the one or more topics, includes:

searching the topics data structure to identify ones of the plurality of topic terms that match at least some of the one or more search query terms; and

for each of the predetermined number of topics, calculating a sum of ones of the plurality of topic term probabilities associated with the ones of the plurality of topic terms that match at least some of the one or more search query terms,

wherein for each of the predetermined number of topics, the sum of the ones of the topic term probabilities is a topic relevance score for the each of the predetermined number of topics.

6. The method of claim 1, wherein identifying the context characteristics data representing the context characteristics for the search query terms, includes at least one of:

retrieving the data representing the user characteristics from existing user characteristics data stored by the customer support system or stored by the one or more service provider systems associated with the customer support system;

receiving the data representing the user characteristics from the current user through a user experience display of the customer support system;

receiving the data representing identification of user experience displays visited by the current user from browsing history for the current user maintained by the one or more service provider systems associated with the customer support system;

identifying the data representing identification of user experience displays visited by the current user by monitoring the browsing history of the current user within the customer support system; and

receiving the data representing identification of the one or more service provider systems from the one or more service provider systems associated with the customer support system.

7. The method of claim 1, wherein the data representing user characteristics are selected from a group of data representing user characteristics, consisting of:

data indicating an age of the current user;

data indicating an age of a spouse of the current user;

data indicating a zip code;

data indicating a tax return filing status;

data indicating state income;

data indicating a home ownership status;

data indicating a home rental status;

data indicating a retirement status;

data indicating a student status;

data indicating an occupation of the current user;

data indicating an occupation of a spouse of the current user;

data indicating whether the user is claimed as a dependent;

data indicating whether a spouse of the user is claimed as a dependent;

data indicating whether another taxpayer is capable of claiming the user as a dependent;

data indicating whether a spouse of the current user is capable of being claimed as a dependent;

data indicating salary and wages;

data indicating taxable interest income;

data indicating ordinary dividend income;

data indicating qualified dividend income;

data indicating business income;

data indicating farm income;

data indicating capital gains income;

data indicating taxable pension income;

data indicating pension income amount;

data indicating IRA distributions;

data indicating unemployment compensation;

data indicating taxable IRA;

data indicating taxable Social Security income;

data indicating amount of Social Security income;

data indicating amount of local state taxes paid;

data indicating whether the user filed a previous years'"'"' federal itemized deduction;

data indicating whether the user filed a previous years'"'"' state itemized deduction; and

data indicating whether the user is a returning user to a tax return preparation system.

8. The method of claim 1, wherein the personalized customer support response includes suggested user actions selected from a group of suggested user actions, consisting of:

select a hyperlink or button to receive a telephone call from customer support personnel;

select a hyperlink or button to initiate a webchat with the customer support personnel;

select a hyperlink or button to access reference materials and/or a topic index for the relevant topic;

select a hyperlink or button to receive a telephone number to dial to reach the customer support personnel; and

select a hyperlink or button to receive information related to available services from a service provider.

9. A system for identifying a relevant topic for search query terms based on context characteristics for the search query terms, to provide customer support responses to the search query terms based on the relevant topic for the search query terms, the system comprising:

a customer support engine that receives search query terms data and context characteristics data for a customer support system hosted by one or more computing systems, the search query terms data representing one or more search query terms, the context characteristics data representing one or more context characteristics, wherein the context characteristics data at least including data representing identification of user experience displays visited by the current user, the user experience displays being provided by one or more service provider systems associated with the customer support system;

an analytics module that identifies one of a plurality of topics as being a relevant topic for the one or more search query terms, at least partially based on the search query terms data and at least partially based on the context characteristics data, wherein the analytics module identifies one of the plurality of topics as being the relevant topic by applying the search query terms data and the context characteristics data to a probabilistic topic model that generates topic relevance scores data representing a plurality of topic relevance scores for the plurality of topics, wherein the analytics module identifies the relevant topic from the plurality of topics by selecting a highest one of the plurality of topic relevance scores for the plurality of topics, wherein the analytics module trains the probabilistic topic model using a Latent Dirichlet algorithm, wherein the analytics module trains the probabilistic topic model with an application of existing customer support content data to a Latent Dirichlet algorithm to generate a predetermined number of topics used to identify the relevant topic, and to generate a plurality of topic term probabilities used to generate the plurality of topic relevance scores;

wherein the customer support engine provides a customer support response that is responsive to receipt of the search query terms data, at least partially based on the relevant topic, to provide the customer support response at least partially based on the context characteristics data to reduce a likelihood of inaccurately identifying the relevant topic and to increase a likelihood of correctly addressing the search query terms with the customer support response.

10. The system of claim 9, wherein the probabilistic topic model includes a Latent Dirichlet algorithm.

11. The system of claim 9, wherein the customer support response includes suggested user actions selected from a group of suggested user actions, consisting of:

select a hyperlink or button to receive a telephone call from customer support personnel;

select a hyperlink or button to initiate a webchat with the customer support personnel;

select a hyperlink or button to access reference materials and/or a topic index for the relevant topic;

select a hyperlink or button to receive a telephone number to dial to reach the customer support personnel; and

select a hyperlink or button to receive information related to available services from a service provider.

12. A system for improving a likelihood of user satisfaction with a customer support response that is provided by a customer support system in response to receiving search query terms, by using context information to reduce a likelihood of inaccurately identifying a relevant topic for the search query terms with a probabilistic topic model, the system comprising:

at least one processor; and

at least one memory coupled to the at least one processor, the at least one memory having stored therein instructions which, when executed by the at least one processors, perform a process including;

training the probabilistic topic model using a Latent Dirichlet algorithm, wherein training the probabilistic topic model includes;

retrieving existing customer support content data;

applying the Latent Dirichlet algorithm to existing customer support content data representing existing customer support content, to generate a predetermined number of topics, to generate a plurality of topic terms that are relevant to each of the predetermined number of topics, and to generate a plurality of topic term probabilities associated with the plurality of topic terms; and

storing predetermined number of topics data representing the predetermined number of topics, plurality of topic terms data representing the plurality of topic terms, and plurality of topic term probabilities data representing the plurality of topic term probabilities, in a topics data structure for the customer support system;

receiving search query terms data representing one or more search query terms received by a customer support system with one or more computing systems, from a current user;

applying the search query terms data to a probabilistic topic model to identify topics data representing one or more topics that are relevant to the one or more search query terms, and to determine topic relevance scores data for the topics data representing the one or more topics, the topic relevance scores data representing one or more topic relevance scores that quantify a likelihood of relevance between the one or more topics and the one or more search query terms received from the current user;

generating length data for the search query terms data, the length data for the search query terms data representing a combined length of the one or more search query terms;

comparing the length data for the search query terms data to search query length threshold data representing a search query length threshold, below which a likelihood of inaccuracy increases for the probabilistic topic model;

upon determining that the combined length of the one or more search query terms is less than the search query length threshold, updating the topic relevance scores data representing the one or more topic relevance scores with context characteristics probabilities data to reduce the likelihood of inaccuracy for the probabilistic topic model, the context characteristics probabilities data representing one or more context characteristics probabilities that quantify a likelihood that a question about the one or more topics occurs while one or more context characteristics for the search query terms exist, wherein updating the topic relevance scores data includes;

identifying context characteristics data representing the context characteristics for the search query terms, the context characteristics data at least including data representing identification of user experience displays visited by the current user, the user experience displays being provided by one or more service provider systems associated with the customer support system;

applying the context characteristics data to the probabilistic topic model to generate the context characteristics probabilities data representing the one or more context characteristics probabilities; and

combining the context characteristics probabilities data with the topic relevance scores data to update the topic relevance scores data to reflect a combination of the context characteristics probabilities data and the topic relevance scores data;

selecting a relevant topic from the one or more topics that is likely most relevant to the one or more search query terms, at least partially based on a highest one of the one or more topic relevance scores represented by the topic relevance scores data; and

providing a personalized customer support response to the current user for the search query terms, at least partially based on the relevant topic, to increase a likelihood of customer satisfaction of the current user with a user experience within the customer support system, by reducing a likelihood of inaccurately identifying the relevant topic for the search query terms received by the customer support system.

13. The system of claim 12, wherein the length of the one or more search query terms is one of a character length and a word length.

14. The system of claim 12, wherein the probabilistic topic model includes a Latent Dirichlet algorithm.

15. The method of claim 12, wherein applying the search query terms data to the probabilistic topic model to identify the topics data representing the one or more topics that are relevant to the one or more search query terms, and to determine topic relevance scores data for the topics data representing the one or more topics, includes:

searching the topics data structure to identify ones of the plurality of topic terms that match at least some of the one or more search query terms; and

for each of the predetermined number of topics, calculating a sum of ones of the plurality of topic term probabilities associated with the ones of the plurality of topic terms that match at least some of the one or more search query terms,

wherein for each of the predetermined number of topics, the sum of the ones of the topic term probabilities is a topic relevance score for the each of the predetermined number of topics.

16. The system of claim 12, wherein identifying the context characteristics data representing the context characteristics for the search query terms, includes at least one of:

retrieving the data representing the user characteristics from existing user characteristics data stored by the customer support system or stored by the one or more service provider systems associated with the customer support system;

receiving the data representing the user characteristics from the current user through a user experience display of the customer support system;

receiving the data representing identification of user experience displays visited by the current user from browsing history for the current user maintained by the one or more service provider systems associated with the customer support system;

identifying the data representing identification of user experience displays visited by the current user by monitoring the browsing history of the current user within the customer support system; and

receiving the data representing identification of the one or more service provider systems from the one or more service provider systems associated with the customer support system.

17. The system of claim 12, wherein the data representing user characteristics are selected from a group of data representing user characteristics, consisting of:

data indicating an age of the current user;

data indicating an age of a spouse of the current user;

data indicating a zip code;

data indicating a tax return filing status;

data indicating state income;

data indicating a home ownership status;

data indicating a home rental status;

data indicating a retirement status;

data indicating a student status;

data indicating an occupation of the current user;

data indicating an occupation of a spouse of the current user;

data indicating whether the user is claimed as a dependent;

data indicating whether a spouse of the user is claimed as a dependent;

data indicating whether another taxpayer is capable of claiming the user as a dependent;

data indicating whether a spouse of the current user is capable of being claimed as a dependent;

data indicating salary and wages;

data indicating taxable interest income;

data indicating ordinary dividend income;

data indicating qualified dividend income;

data indicating business income;

data indicating farm income;

data indicating capital gains income;

data indicating taxable pension income;

data indicating pension income amount;

data indicating IRA distributions;

data indicating unemployment compensation;

data indicating taxable IRA;

data indicating taxable Social Security income;

data indicating amount of Social Security income;

data indicating amount of local state taxes paid;

data indicating whether the user filed a previous years'"'"' federal itemized deduction;

data indicating whether the user filed a previous years'"'"' state itemized deduction; and

data indicating whether the user is a returning user to a tax return preparation system.

18. The system of claim 12, wherein the personalized customer support response includes suggested user actions selected from a group of suggested user actions, consisting of:

select a hyperlink or button to receive a telephone call from customer support personnel;

select a hyperlink or button to initiate a webchat with the customer support personnel;

select a hyperlink or button to access reference materials and/or a topic index for the relevant topic;

select a hyperlink or button to receive a telephone number to dial to reach the customer support personnel; and

select a hyperlink or button to receive information related to available services from a service provider.