Determining application experience based on paralinguistic information

• US 10,049,664 B1
• Filed: 10/27/2016
• Issued: 08/14/2018
• Est. Priority Date: 10/27/2016
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Indyk, Benjamin, Podgorny, Igor A., Chan, Raymond
• Primary Examiner(s): Azad, Abul
• Application Number:
  US15/336,215
• Time in Patent Office: 656 Days
• Field of Search: 704275
• CPC Class Codes: G10L 15/02 Feature extraction for speech recognition; Selection of recognition unit
• CPC Class Codes: G10L 15/1815 Semantic context, e.g. disambiguation of the recognition hypotheses based on word meaning
• CPC Class Codes: G10L 15/22 Procedures used during a speech recognition process, e.g. man-machine dialogue
• CPC Class Codes: G10L 2015/223 Execution procedure of a spoken command
• CPC Class Codes: G10L 25/63 for estimating an emotional state
• CPC Class Codes: H04L 67/53 using third party service providers
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34
• CPC Class Codes: H04L 67/63 Routing a service request depending on the request content or context

Abstract

Techniques are disclosed for determining application experience of a user based on paralinguistic information determined for the user. One embodiment presented herein includes a computer-implemented method, which includes receiving, at a computing device, one or more audio streams. Each audio stream includes audio from a user interacting with at least one page of an application. The computer-implemented method also includes generating paralinguistic information from at least one of the audio streams. The paralinguistic information characterizes acoustic aspects of the audio stream that are distinct from verbal content of the audio stream. The computer-implemented method further includes determining, from the paralinguistic information, at least a current user sentiment regarding the at least one page, determining, based on evaluating the current user sentiment, that the user is predicted to discontinue use of the application, and taking action to reduce the likelihood of the user discontinuing use of the application.

Claims

1. A computer-implemented method for determining application experience of a user, comprising:

receiving, at a computing device, one or more audio streams, each audio stream comprising audio of a user interacting with at least a first page of an application;

generating paralinguistic information from at least one of the audio streams, wherein the paralinguistic information characterizes acoustic aspects of the audio stream that are distinct from verbal content of the audio stream;

determining, from the paralinguistic information, at least a current user sentiment regarding the first page;

determining, based on evaluating the current user sentiment with a propensity model, that the user is predicted to discontinue use of the application, wherein determining that the user is predicted to discontinue use of the application comprises;

generating, from the evaluation of the current user sentiment with the propensity model, a score indicating a likelihood of the user to discontinue use of the application; and

determining that the score satisfies a predetermined condition for the determination that the user is predicted to discontinue use of the application;

determining, based on evaluating at least the current user sentiment with an uplift model, an intervention that reduces the likelihood of the user discontinuing use of the application; and

modifying a user interface according to the intervention while the user is interacting with the application.

2. The computer-implemented method of claim 1, wherein the intervention is determined by using the uplift model to evaluate the current user sentiment for the at least first page and the score indicating the likelihood of the user to discontinue use of the application.

3. The computer-implemented method of claim 1, wherein the intervention comprises at least one of:

offering a discount, offering assisted support, offering self-support content, and providing a list of content items.

4. The computer-implemented method of claim 1, wherein modifying the user interface according to the intervention comprises at least one of:

presenting content items on the user interface, and altering at least one content item of the user interface.

5. The computer-implemented method of claim 1, wherein the current user sentiment for the first page comprises an array of measures of emotional states predicted for the user based on the paralinguistic information.

6. The computer-implemented method of claim 1, further comprising:

determining at least one of a count of user clicks for the first page, a total amount of time spent by the user on the first page, age of the user, gender of the user, occupation of the user, and location of the user; and

evaluating the at least one of the count of user clicks, total amount of time, age of the user, gender of the user, occupation of the user, and location of the user in addition to the current user sentiment with the propensity model to determine the score indicating the likelihood of the user to discontinue use of the application.

7. A computing device for determining application experience of a user, the computing device comprising:

a memory; and

a processor configured to;

receive one or more audio streams, each audio stream comprising audio of a user interacting with at least a first page of an application;

generate paralinguistic information from at least one of the audio streams, wherein the paralinguistic information characterizes acoustic aspects of the audio stream that are distinct from verbal content of the audio stream;

determine, from the paralinguistic information, at least a current user sentiment regarding the first page;

determine, based on evaluating the current user sentiment with a propensity model, that the user is predicted to discontinue use of the application, wherein the processor is configured to determine that the user is predicted to discontinue use of the application by;

generating, from the evaluation of the current user sentiment with the propensity model, a score indicating a likelihood of the user to discontinue use of the application; and

determining that the score satisfies a predetermined condition for the determination that the user is predicted to discontinue use of the application;

determine, based on evaluating at least the current user sentiment with an uplift model, an intervention that reduces the likelihood of the user discontinuing use of the application; and

modify a user interface according to the intervention while the user is interacting with the application.

8. The computing device of claim 7, wherein the processor is configured to determine the intervention by using the uplift model to evaluate the current user sentiment for the at least first page and the score indicating the likelihood of the user to discontinue use of the application.

9. The computing device of claim 7, wherein the intervention comprises at least one of:

offering a discount, offering assisted support, offering self-support content, and providing a list of content items.

10. The computing device of claim 7, wherein the processor is configured to modify the user interface according to the intervention by at least one of presenting content items on the user interface, and altering at least one content item of the user interface.

11. The computing device of claim 7, wherein the current user sentiment for the first page comprises an array of measures of emotional states predicted for the user based on the paralinguistic information.

12. The computing device of claim 7, wherein the processor is further configured to:

determine at least one of a count of user clicks for the first page, a total amount of time spent by the user on the first page, age of the user, gender of the user, occupation of the user, and location of the user; and

evaluate the at least one of the count of user clicks, total amount of time, age of the user, gender of the user, occupation of the user, and location of the user in addition to the current user sentiment with the propensity model to determine the score indicating the likelihood of the user to discontinue use of the application.

13. A non-transitory computer-readable medium comprising instructions that when executed by a computing device cause the computing device to perform a method for determining application experience of a user, the method comprising:

receiving one or more audio streams, each audio stream comprising audio of a user interacting with at least a first page of an application;

generating paralinguistic information from at least one of the audio streams, wherein the paralinguistic information characterizes acoustic aspects of the audio stream that are distinct from verbal content of the audio stream;

determining, from the paralinguistic information, at least a current user sentiment regarding the first page;

determining, based on evaluating the current user sentiment with a propensity model, that the user is predicted to discontinue use of the application,

wherein determining that the user Is predicted to discontinue use of the application comprises;

generating from the evaluation of the current user sentiment with the propensity model, a score indicating a likelihood of the user to discontinue use of the application; and

determining that the score satisfies a predetermined condition for the determination that the user is predicted to discontinue use of the application;

determining, based on evaluating at least the current user sentiment with an uplift model, an intervention that reduces the likelihood of the user discontinuing use of the application; and

modifying a user interface according to the intervention while the user is interacting with the application.

14. The non-transitory computer-readable medium of claim 13, wherein the intervention is determined by using the uplift model to evaluate the current user sentiment for the at least first page and the score indicating the likelihood of the user to discontinue use of the application.

15. The non-transitory computer-readable medium of claim 13, wherein the intervention comprises at least one of:

offering a discount, offering assisted support, offering self-support content, and providing a list of content items.

16. The non-transitory computer-readable medium of claim 13, wherein modifying the user interface according to the intervention comprises at least one of:

presenting content items on the user interface, and altering at least one content item of the user interface.

17. The non-transitory computer-readable medium of claim 13, wherein the current user sentiment for the first page comprises an array of measures of emotional states predicted for the user based on the paralinguistic information.

18. The non-transitory computer-readable medium of claim 13, the method further comprising:

determining at least one of a count of user clicks for the first page, a total amount of time spent by the user on the first page, age of the user, gender of the user, occupation of the user, and location of the user; and

evaluating the at least one of the count of user clicks, total amount of time, age of the user, gender of the user, occupation of the user, and location of the user in addition to the current user sentiment with the propensity model to determine the score indicating the likelihood of the user to discontinue use of the application.