Method and system for applying probabilistic topic models to content in a tax environment to improve user satisfaction with a question and answer customer support system

• US 10,268,956 B2
• Filed: 07/31/2015
• Issued: 04/23/2019
• Est. Priority Date: 07/31/2015
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Podgorny, Igor A., Koonse, Benjamin John
• Primary Examiner(s): Vincent, David R
• Application Number:
  US14/814,765
• Publication Number:
  US 20170032251A1
• Time in Patent Office: 1,362 Days
• Field of Search: 706 12, 706 15, 706 45
• CPC Class Codes: G06N 5/04 Inference or reasoning models
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06Q 30/016 After-sales
• CPC Class Codes: G06Q 40/123 Tax preparation or submission

Abstract

A method and system applies a probabilistic topic model to content in a tax environment to improve user satisfaction with a question and answer customer support system, according to one embodiment. The probabilistic topic model may be the Latent Dirichlet allocation algorithm or other implementations of probabilistic topic models, according to various embodiments. The method and system include receiving submission content from a user, according to one embodiment. The method and system include applying a probabilistic topic model to the submission content to determine submission content topics and submission content statistics, according to one embodiment. The method and system include generating and providing customer support content at least partially based on the submission content topics and at least partially based on the submission content statistics, to facilitate use of the question and answer customer support system by the user, according to one embodiment.

Claims

1. A computer-implemented method for applying probabilistic topic models to content in a tax environment to improve user satisfaction with a question and answer customer support system, the method comprising:

receiving, with a computing system, submission content from a user through a user interface for the question and answer customer support system;

applying, in real-time while portions of the submission content are being received, a probabilistic topic model to the submission content to determine submission content topics and submission content statistics;

generating customer support content at least partially based on the submission content topics and at least partially based on the submission content statistics, to facilitate use of the question and answer customer support system by the user, the generated customer support content including recommendations for modifying question content that is received in the submission content and recommendations for modifying answer content that is received in the submission content; and

providing the customer support content to the user through the user interface, in response to receiving the submission content from the user through the user interface, the provided customer support content including populated new customer support content including real-time recommendations for improving the quality and/or the popularity of the question content, the populated content further including a real-time question quality indicator and a real-time question popularity indicator.

2. The method of claim 1, wherein the probabilistic topic model includes a Latent Dirichlet allocation algorithm.

3. The method of claim 1, wherein the submission content includes content selected from a group consisting of:

question content from an asking user,

response content from a responding user, and

query content from a searching user.

4. The method of claim 1, wherein the submission content topics are discrete portions of the submission content that provide quantifiable summaries of the submission content.

5. The method of claim 1, wherein the customer support content includes content selected from a group consisting of:

indicators for strength of question content that is received in the submission content;

indicators for strength of response content that is received in the submission content; and

search results for query content that is received in the submission content.

6. The method of claim 1, wherein applying the probabilistic topic model includes applying the probabilistic topic model to the submission content to determine submission content topics and submission content statistics, without training the probabilistic topic model with existing content from the question and answer customer support system.

7. The method of claim 1, wherein the submission content topics include individual terms that are discrete portions of the submission content, wherein each of the terms are unique to each of other ones of the terms.

8. The method of claim 7, wherein each term is an individual word.

9. The method of claim 1, further comprising:

categorizing the submission content as product-related content or tax-related content, at least partially based on one or more of the submission content topics and the submission content statistics; and

identifying the submission content as product-related content or tax-related content.

10. The method of claim 9, further comprising:

updating a question and answer customer support database to include the submission content that has been identified as product-related content or tax-related content,

wherein updating the question and answer customer support database includes associating a product-related content identifier or a tax-related content identifier with the submission content, in the question and answer customer support database.

11. The method of claim 9, further comprising:

if the submission content is identified as product-related content, routing the submission content to a first responding user to generate response content for the submission content; and

if the submission content is identified as tax-related content, routing the submission content to a second responding user to generate the response content for the submission content.

12. The method of claim 11, wherein the first responding user is a customer service representative for the question and answer customer support system.

13. The method of claim 1, wherein the submission content includes query content from a searching user, the method further comprising:

determining, with the probabilistic topic model, whether the query content is more relevant to product-related content or tax-related content; and

searching product-related content or tax-related content in a question and answer customer support database for response content that satisfies search criteria of the query content, to increase a likelihood of returning relevant response criteria in response to the query content.

14. The method of claim 1, wherein the submission content includes query content from a searching user, the method further comprising:

determining existing submission content topics of existing submission content, at least partially based on the probabilistic topic model;

determining relevant ones of the existing submission content topics; and

providing weblinks to the relevant ones of the existing submission content topics, to improve relevant navigation of search results for the user.

15. The method of claim 14, wherein the weblinks of the relevant ones of the existing submission content topics are sorted by at least one of:

popularity, quality, and relevance to the query content.

16. The method of claim 1, further comprising:

applying the probabilistic topic model to content of a question and answer customer support database to determine submission content topics for existing submission content in the question and answer customer support database; and

correcting mis-categorized submission content topics for the existing submission content, at least partially based on model output from the probabilistic topic model, to improve an accuracy of queries to the question and answer customer support database.

17. The method of claim 1, further comprising:

applying the probabilistic topic model to content of a question and answer customer support database to determine submission content topics for existing submission content in the question and answer customer support database; and

removing portions of the existing submission content, at least partially based on model output from the probabilistic topic model, to improve user satisfaction with queries to the question and answer customer support database,

wherein the portions of the existing submission content include one or more of low-quality content, low-popularity content, and redundant content.

18. A non-transitory computer-readable medium having instructions which, when executed by one or more computer processors, perform a method for applying probabilistic topic models to content in a tax environment to improve user satisfaction with a question and answer customer support system, the instructions including:

a customer support content database configured to maintain existing submission content to support operations for a question and answer customer support system;

an analytics module configured to apply, in real-time while portions of the submission content are being received, a probabilistic topic model to new submission content to generate new customer support content, the generated customer support content including recommendations for modifying question content that is received in the submission content and recommendations for modifying answer content that is received in the submission content,

wherein the new customer support content is at least partially based on submission content topics and submission content topics statistics that are generated by the probabilistic topic model from the new submission content; and

a customer support engine configured to receive new submission content from a user, to update the existing submission content in the customer support content database with the new submission content, to provide the new submission content to the analytics module, and to receive customer support content from the analytics module that is at least partially based on the new submission content provided to the analytics module, the received customer support content including populated new customer support content including real-time recommendations for improving the quality and/or the popularity of the question content, the populated content further including a real-time question quality indicator and a real-time question popularity indicator.

19. The computer-readable medium of claim 18, wherein the probabilistic topic model includes a Latent Dirichlet allocation algorithm.

20. The computer-readable medium of claim 18, wherein the new submission content includes content selected from a group consisting of:

question content from an asking user,

response content from a responding user, and

query content from a searching user.

21. The computer-readable medium of claim 18, wherein the submission content topics are discrete portions of the new submission content that provide quantifiable summaries of the submission content.

22. The computer-readable medium of claim 18, wherein the new customer support content includes content selected from a group consisting of:

indicators for strength of question content that is received in the submission content;

indicators for strength of response content that is received in the submission content; and

search results for query content that is received in the submission content.

23. The computer-readable medium of claim 18, wherein the analytics module is configured to categorized the new submission content as product-related content or tax-related content, at least partially based on one or more of the submission content topics and the submission content statistics, and to tag the new submission content as product-related content or tax-related content for identification in the customer support content database.

24. A system for applying probabilistic topic models to content in a tax environment to improve user satisfaction with a question and answer customer support system, the system comprising:

at least one processor; and

at least one memory coupled to the at least one processor, the at least one memory having stored therein instructions which, when executed by any set of the one or more processors, perform a process for applying probabilistic topic models to content in a tax environment, the process including;

receiving, with a computing system, submission content from a user through a user interface for the question and answer customer support system;

applying, in real-time while portions of the submission content are being received, a probabilistic topic model to the submission content to determine submission content topics and submission content statistics;

generating customer support content at least partially based on the submission content topics and at least partially based on the submission content statistics, to facilitate use of the question and answer customer support system by the user, the generated customer support content including recommendations for modifying question content that is received in the submission content and recommendations for modifying answer content that is received in the submission content; and

providing the customer support content to the user through the user interface, in response to receiving the submission content from the user through the user interface, the provided customer support content including populated new customer support content including real-time recommendations for improving the quality and/or the popularity of the question content, the populated content further including a real-time question quality indicator and a real-time question popularity indicator.

25. The system of claim 24, wherein the probabilistic topic model includes a Latent Dirichlet allocation algorithm.

26. The system of claim 24, wherein the submission content includes content selected from a group consisting of:

question content from an asking user,

response content from a responding user, and

query content from a searching user.

27. The system of claim 24, wherein the submission content topics are discrete portions of the submission content that provide quantifiable summaries of the submission content.

28. The system of claim 24, wherein the customer support content includes content selected from a group consisting of:

indicators for strength of question content that is received in the submission content;

indicators for strength of response content that is received in the submission content; and

search results for query content that is received in the submission content.

29. The system of claim 24, wherein applying the probabilistic topic model includes applying the probabilistic topic model to the submission content to determine submission content topics and submission content statistics, without training the probabilistic topic model with existing content from the question and answer customer support system.

30. The system of claim 24, wherein the submission content topics include individual terms that are discrete portions of the submission content, wherein each of the terms are unique to each of other ones of the terms.

31. The system of claim 30, wherein each term is an individual word.

32. The system of claim 24, wherein the process further comprises:

categorizing the submission content as product-related content or tax-related content, at least partially based on one or more of the submission content topics and the submission content statistics; and

identifying the submission content as product-related content or tax-related content.

33. The system of claim 32, wherein the process further comprises:

updating a question and answer customer support database to include the submission content that has been identified as product-related content or tax-related content,

wherein updating the question and answer customer support database includes associating a product-related content identifier or a tax-related content identifier with the submission content, in the question and answer customer support database.

34. The system of claim 32, wherein the process further comprises:

if the submission content is identified as product-related content, routing the submission content to a first responding user to generate response content for the submission content; and

if the submission content is identified as tax-related content, routing the submission content to a second responding user to generate the response content for the submission content.

35. The system of claim 34, wherein the first responding user is a customer service representative for the question and answer customer support system.

36. The system of claim 24, wherein the submission content includes query content from a searching user, wherein the process further comprises:

determining, with the probabilistic topic model, whether the query content is more relevant to product-related content or tax-related content; and

searching product-related content or tax-related content in a question and answer customer support database for response content that satisfies search criteria of the query content, to increase a likelihood of returning relevant response criteria in response to the query content.

37. The system of claim 24, wherein the submission content includes query content from a searching user, wherein the process further comprises:

determining existing submission content topics of existing submission content, at least partially based on the probabilistic topic model;

determining relevant ones of the existing submission content topics; and

providing weblinks to the relevant ones of the existing submission content topics, to improve relevant navigation of search results for the user.

38. The system of claim 37, wherein the weblinks of the relevant ones of the existing submission content topics are sorted by at least one of:

popularity, quality, and relevance to the query content.

39. The system of claim 24, wherein the process further comprises:

applying the probabilistic topic model to content of a question and answer customer support database to determine submission content topics for existing submission content in the question and answer customer support database; and

correcting mis-categorized submission content topics for the existing submission content, at least partially based on model output from the probabilistic topic model, to improve an accuracy of queries to the question and answer customer support database.

40. The system of claim 24, wherein the process further comprises:

applying the probabilistic topic model to content of a question and answer customer support database to determine submission content topics for existing submission content in the question and answer customer support database; and

removing portions of the existing submission content, at least partially based on model output from the probabilistic topic model, to improve user satisfaction with queries to the question and answer customer support database,

wherein the portions of the existing submission content include one or more of low-quality content, low-popularity content, and redundant content.