Method and system for using emotional state data to tailor the user experience of an interactive software system

• US 10,387,173 B1
• Filed: 02/22/2018
• Issued: 08/20/2019
• Est. Priority Date: 03/27/2015
• Status: Active Grant

• Current Assignee: Intuit, Inc.
• Original Assignee: Intuit, Inc.
• Inventors: Paulus, Wolfgang, Cabrera, Luis Felipe, Graves, Mike
• Primary Examiner(s): Sax, Steven P
• Application Number:
  US15/902,832
• Time in Patent Office: 544 Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06Q 10/0635 Risk analysis of enterprise or organisation activities
• CPC Class Codes: G06Q 30/04 Billing or invoicing
• CPC Class Codes: G06Q 40/123 Tax preparation or submission
• CPC Class Codes: G06Q 40/125 Finance or payroll
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

Emotional state data is used to tailor the user experience of an interactive software system, by monitoring and obtaining data about a user'"'"'s emotional state. Invoice data is further used, and organized in a particular manner for presentation to the user based on a predicted future emotional state of the user. Emotional state data is analyzed and used to dynamically modify the user'"'"'s experience by selecting user experience components and invoice presentation characteristics based on the analysis of the user'"'"'s emotional state data. Different types of user experience components can be utilized to adjust the user experience to adapt to the user'"'"'s new emotional state, prevent the user from entering an undesirable emotional state, and/or encourage the user to enter into a desirable emotional state.

Claims

1. A method comprising:

defining user experience component data representing one or more user experience components capable of combination to provide a user experience through an interactive software system, the defined user experience components at least including images provided to the user;

storing the user experience component data in a partitioned user experience data section of a memory device and/or system;

generating emotional state threshold parameter data representing emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed;

storing the emotional state threshold parameter data in a partitioned emotional state threshold parameter data section of a memory device and/or system;

analyzing, using a currently installed analytics module implementing one or more user emotional state adaptation algorithms, historical and/or current operational data associated with a business employing the interactive software system, the user profile data associated with the user, the user attempting to fill out financial invoices, and current invoice content data to predict an emotional effect on the user of the current invoices represented by the current invoice content data;

analyzing obtained emotional state data and emotional state parameter data associated with the obtained emotional state data;

predicting, based at least in part on the analysis of the obtained emotional state data and the emotional state parameter data associated with the one or more types of obtained emotional state data, a future emotional state of the user;

selecting, based at least in part on the predicted future emotional state of the user, a portion of the user experience component data, the selected portion of the user experience component data representing one or more selected user experience components of the defined one or more user experience components and further determining, based at least in part on the predicted future emotional state of the user, in what manner, and/or at what time, and/or in what order or grouping to present the invoices represented by the current invoice content data to the user;

transforming, using the portion of the user experience component data representing one or more selected user experience components of the defined one or more user experience components, a user experience provided through the interactive software system into a user experience adapted to the predicted future emotional state of the user; and

providing the user experience adapted to the predicted future emotional state of the user to the user including invoices represented by the current invoice content data in the manner, and/or at the time, and/or in the grouping determined, based at least in part, on the predicted emotional effect on the user of the invoices represented by the current invoice content data, wherein the user experience components include at least one user experience component selected from the group of user experience components consisting of;

a sequence with which interview questions are presented to the user;

content or topics of the interview questions that are presented to the user;

user recommendations presented to the user.

2. The method of claim 1, wherein the interactive software system is selected from the group of interactive software systems consisting of:

a computing system implemented tax preparation software system;

a network accessed tax preparation software system;

a web-based tax preparation software system;

a cloud-based tax preparation software system;

a computing system implemented business management software system;

a network accessed business management software system;

a web-based business management software system;

a cloud-based business management software system;

a computing system implemented accounting software system;

a network accessed accounting software system;

a web-based accounting software system;

a cloud-based accounting software system;

a computing system implemented financial management system;

a network accessed financial management system;

a web-based financial management system; and

a cloud-based financial management system.

3. The method of claim 1, wherein the user experience is presented to the user via at least one computing system selected from the group of computing systems consisting of:

a server computing system;

a workstation;

a desktop computing system;

a mobile computing system;

a database system or storage cluster;

a virtual asset;

a switching system;

a router;

a hardware system;

a communications system;

a proxy system;

a gateway system;

a firewall system; and

a load balancing system.

4. The method of claim 1, wherein the user experience components further include at least one user experience component selected from the group of user experience components consisting of:

font sizes used while presenting information to the user;

themes presented to the user;

types of icons displayed to the user;

types of interface formats presented to the user;

interface displays presented to the user;

assistance resources listed and/or recommended to the user;

backgrounds presented to the user;

background audio presented to the user;

avatars and/or icons presented to the user;

avatars and/or icons presented to the user;

highlighting mechanisms used and highlighted components presented to the user;

support mechanisms presented to the user; and

supplemental actions and recommendations presented to the user.

5. The method of claim 1, wherein the emotional state data includes at least one of the emotional state data selected from the group of emotional state data consisting of:

data acquired from the user'"'"'s own characterization of his or her emotional state;

data acquired from historical user data;

data acquired from a segment of users having characteristics comparable to the user;

data acquired from measuring the user'"'"'s heart beat;

data acquired from measuring the user'"'"'s eye rotation;

data acquired from measuring the user'"'"'s perspiration;

data acquired from measuring the user'"'"'s respiration;

data acquired from measuring the user'"'"'s oxygen saturation;

data acquired from measuring the user'"'"'s blood pressure

data acquired from measuring the user'"'"'s skin temperature;

data acquired from measuring the user'"'"'s neural activity;

data acquired from measuring the user'"'"'s eye blinking;

data acquired from measuring the user'"'"'s facial expression;

data acquired from measuring the user'"'"'s voice and/or speech; and

data acquired from measuring the user'"'"'s interactions with hardware associated with an interactive software system.

6. The method of claim 1, wherein the emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed are defined based on emotional state data associated with a general population.

7. The method of claim 6, wherein the emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed include at least one of emotional state threshold parameter selected from the group of emotional state threshold parameters consisting of:

a heart rate emotional state threshold parameter associated with a user'"'"'s heart beat;

a heart rate variability level emotional state threshold parameter associated with a user'"'"'s heart beat;

a scan path emotional state threshold parameter associated with a user'"'"'s eye rotation;

a fixation period emotional state threshold parameter associated with a user'"'"'s eye rotation;

a skin conductance level emotional state threshold parameter associated with a user'"'"'s perspiration;

a respiration rate emotional state threshold parameter associated with a user'"'"'s respiration;

an oxygen saturation level emotional state threshold parameter associated with a user'"'"'s oxygen saturation;

a blood pressure level emotional state threshold parameter associated with a user'"'"'s blood pressure;

a skin temperature emotional state threshold parameter associated with a user'"'"'s skin temperature;

a muscle tension level emotional state threshold parameter associated with a user'"'"'s muscle tension;

a neural activity level emotional state threshold parameter associated with a user'"'"'s neural activity;

an eye blink rate emotional state threshold parameter associated with a user'"'"'s eye blinking;

a facial muscle movement emotional state threshold parameter associated with a user'"'"'s facial expression;

an acoustic characteristics emotional state threshold parameter associated with a user'"'"'s voice and/or speech;

a contact pressure emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system; and

a contact rate emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system.

8. The method of claim 1, wherein the emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed are customized based on emotional state profile data associated with the specific user.

9. The method of claim 8, wherein the customized emotional state threshold parameters include at least one of the customized emotional state threshold parameter selected from the group of customized emotional state threshold parameters consisting of:

a heart rate emotional state threshold parameter associated with the user'"'"'s heart beat

a heart rate variability level emotional state threshold parameter associated with the user'"'"'s heart beat;

a scan path emotional state threshold parameter associated with the user'"'"'s eye rotation;

a fixation period emotional state threshold parameter associated with the user'"'"'s eye rotation;

a skin conductance level emotional state threshold parameter associated with the user'"'"'s perspiration;

a respiration rate emotional state threshold parameter associated with the user'"'"'s respiration;

an oxygen saturation level emotional state threshold parameter associated with the user'"'"'s oxygen saturation;

a blood pressure level emotional state threshold parameter associated with the user'"'"'s blood pressure;

a skin temperature emotional state threshold parameter associated with the user'"'"'s skin temperature;

a muscle tension level emotional state threshold parameter associated with the user'"'"'s muscle tension;

a neural activity level emotional state threshold parameter associated with the user'"'"'s neural activity;

an eye blink rate emotional state threshold parameter associated with the user'"'"'s eye blinking;

a facial muscle movement emotional state threshold parameter associated with the user'"'"'s facial expression;

an acoustic characteristics emotional state threshold parameter associated with the user'"'"'s voice and/or speech;

a contact pressure emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system; and

a contact rate emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system.

10. The method of claim 1, wherein the historical and/or current operational data includes historical and current operational data selected from the group of historical and current operational data consisting of:

data indicating the historical payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating the historical grouping or relation of one or more invoices similar to one or more of the current invoices represented by the current invoice data;

data indicating recent historical payments made on invoices by the business;

data indicating accounts receivable amounts due to be paid to the business in a defined future time period;

data indicating the historical cash position of the business;

data indicating the historical connection between the cash position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the cash position of the business;

data indicating the current cash position of the business;

data indicating the current cash position of the business compared to the historical cash position of the business;

data indicating the historical revenue position of the business;

data indicating the historical connection between the revenue position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the revenue position of the business;

data indicating the current revenue position of the business;

data indicating the current revenue position of the business compared to the historical revenue position of the business;

data indicating the historical profit position of the business;

data indicating the historical connection between the profit position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the profit position of the business;

data indicating the current profit position of the business;

data indicating the current profit position of the business compared to the historical profit position of the business;

data indicating the historical financial liability position of the business;

data indicating the historical connection between the financial liability position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the financial liability position of the business;

data indicating the current financial liability position of the business; and

data indicating the current financial liability position of the business compared to the historical financial liability position of the business.

11. The method of claim 1, wherein a future emotional state of the user includes at least one future emotional state selected from the group of emotional states consisting of:

a happy emotional state or happiness;

a sad emotional state or sadness;

a surprised emotional state or surprise;

a fearful emotional state or fear;

a disgusted emotional state or disgust;

an angry emotional state or anger;

a tense emotional state;

a nervous emotional state;

a stressed emotional state;

an upset emotional state;

a frustrated emotional state;

a depressed emotional state;

a bored emotional state;

a fatigued emotional state;

an alert emotional state;

an excited emotional state;

an elated emotional state;

a happy emotional state;

a contented emotional state;

a serene emotional state;

a relaxed emotional state; and

a calm emotional state.

12. The method of claim 1, wherein transforming the user experience provided through the interactive software system into a user experience adapted to the predicted future emotional state of the user includes adding one or more user experience components to the user experience.

13. The method of claim 1, wherein transforming the user experience provided through the interactive software system into a user experience adapted to the predicted future emotional state of the user includes removing one or more user experience components from the user experience.

14. The method of claim 1, wherein the user experience adapted to the predicted future emotional state of the user is provided to the user via a user display screen displayed on a user computing system accessible by the user.

15. The method of claim 1, wherein the emotional state data associated with a user is obtained on a periodic basis and the user experience adapted to the predicted future emotional state of the user is modified and provided to the user on a periodic basis.

16. A system comprising:

one or more computing processors;

one or more memories having computing processor executable instructions which when executed by the one or more computing processors, performs a method comprising;

defining user experience component data representing one or more user experience components capable of combination to provide a user experience through an interactive software system, the defined user experience components at least including images provided to the user;

storing the user experience component data in a partitioned user experience data section of a memory device and/or system;

generating emotional state threshold parameter data representing emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed;

storing the emotional state threshold parameter data in a partitioned emotional state threshold parameter data section of a memory device and/or system;

analyzing, using a currently installed analytics module implementing one or more user emotional state adaptation algorithms, historical and/or current operational data associated with a business employing the interactive software system, the user profile data associated with the user, the user attempting to fill out financial invoices, and current invoice content data to predict an emotional effect on the user of the current invoices represented by the current invoice content data;

analyzing obtained emotional state data and emotional state parameter data associated with the obtained emotional state data;

predicting, based at least in part on the analysis of the obtained emotional state data and the emotional state parameter data associated with the one or more types of obtained emotional state data, a future emotional state of the user;

selecting, based at least in part on the predicted future emotional state of the user, a portion of the user experience component data, the selected portion of the user experience component data representing one or more selected user experience components of the defined one or more user experience components and further determining, based at least in part on the predicted future emotional state of the user, in what manner, and/or at what time, and/or in what order or grouping to present the invoices represented by the current invoice content data to the user;

transforming, using the portion of the user experience component data representing one or more selected user experience components of the defined one or more user experience components, a user experience provided through the interactive software system into a user experience adapted to the predicted future emotional state of the user; and

providing the user experience adapted to the predicted future emotional state of the user to the user including invoices represented by the current invoice content data in the manner, and/or at the time, and/or in the grouping determined, based at least in part, on the predicted emotional effect on the user of the invoices represented by the current invoice content data, wherein the user experience components include at least one user experience component selected from the group of user experience components consisting of;

a sequence with which interview questions are presented to the user;

content or topics of the interview questions that are presented to the user;

user recommendations presented to the user.

17. The system of claim 16, wherein the interactive software system is selected from the group of interactive software systems consisting of:

a computing system implemented tax preparation software system;

a network accessed tax preparation software system;

a web-based tax preparation software system;

a cloud-based tax preparation software system;

a computing system implemented business management software system;

a network accessed business management software system;

a web-based business management software system;

a cloud-based business management software system;

a computing system implemented accounting software system;

a network accessed accounting software system;

a web-based accounting software system;

a cloud-based accounting software system;

a computing system implemented financial management system;

a network accessed financial management system;

a web-based financial management system; and

a cloud-based financial management system.

18. The system of claim 16, wherein the user experience is presented to the user via at least one computing system selected from the group of computing systems consisting of:

a server computing system;

a workstation;

a desktop computing system;

a mobile computing system;

a database system or storage cluster;

a virtual asset;

a switching system;

a router;

a hardware system;

a communications system;

a proxy system;

a gateway system;

a firewall system; and

a load balancing system.

19. The system of claim 16, wherein the user experience components further include at least one user experience component selected from the group of user experience components consisting of:

font sizes used while presenting information to the user;

themes presented to the user;

types of icons displayed to the user;

types of interface formats presented to the user;

interface displays presented to the user;

assistance resources listed and/or recommended to the user;

backgrounds presented to the user;

background audio presented to the user;

avatars and/or icons presented to the user;

avatars and/or icons presented to the user;

highlighting mechanisms used and highlighted components presented to the user;

support mechanisms presented to the user; and

supplemental actions and recommendations presented to the user.

20. The system of claim 16, wherein the emotional state data includes at least one of the emotional state data selected from the group of emotional state data consisting of:

data acquired from the user'"'"'s own characterization of his or her emotional state;

data acquired from historical user data;

data acquired from a segment of users having characteristics comparable to the user;

data acquired from measuring the user'"'"'s heart beat;

data acquired from measuring the user'"'"'s eye rotation;

data acquired from measuring the user'"'"'s perspiration;

data acquired from measuring the user'"'"'s respiration;

data acquired from measuring the user'"'"'s oxygen saturation;

data acquired from measuring the user'"'"'s blood pressure;

data acquired from measuring the user'"'"'s skin temperature;

data acquired from measuring the user'"'"'s neural activity;

data acquired from measuring the user'"'"'s eye blinking;

data acquired from measuring the user'"'"'s facial expression;

data acquired from measuring the user'"'"'s voice and/or speech; and

data acquired from measuring the user'"'"'s interactions with hardware associated with an interactive software system.

21. The system of claim 16, wherein the emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed are defined based on emotional state data associated with a general population.

22. The system of claim 16, wherein the emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed include at least one of emotional state threshold parameter selected from the group of emotional state threshold parameters consisting of:

a heart rate emotional state threshold parameter associated with a user'"'"'s heart beat;

a heart rate variability level emotional state threshold parameter associated with a user'"'"'s heart beat;

a scan path emotional state threshold parameter associated with a user'"'"'s eye rotation;

a fixation period emotional state threshold parameter associated with a user'"'"'s eye rotation;

a skin conductance level emotional state threshold parameter associated with a user'"'"'s perspiration;

a respiration rate emotional state threshold parameter associated with a user'"'"'s respiration;

an oxygen saturation level emotional state threshold parameter associated with a user'"'"'s oxygen saturation;

a blood pressure level emotional state threshold parameter associated with a user'"'"'s blood pressure;

a skin temperature emotional state threshold parameter associated with a user'"'"'s skin temperature;

a muscle tension level emotional state threshold parameter associated with a user'"'"'s muscle tension;

a neural activity level emotional state threshold parameter associated with a user'"'"'s neural activity;

an eye blink rate emotional state threshold parameter associated with a user'"'"'s eye blinking;

a facial muscle movement emotional state threshold parameter associated with a user'"'"'s facial expression;

an acoustic characteristics emotional state threshold parameter associated with a user'"'"'s voice and/or speech;

a contact pressure emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system; and

a contact rate emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system.

23. The system of claim 16, wherein the emotional state threshold parameters for each defined type of emotional state data to be obtained and analyzed are customized based on emotional state profile data associated with the specific user.

24. The system of claim 23, wherein the customized emotional state threshold parameters include at least one of the customized emotional state threshold parameter selected from the group of customized emotional state threshold parameters consisting of:

a heart rate emotional state threshold parameter associated with the user'"'"'s heart beat

a heart rate variability level emotional state threshold parameter associated with the user'"'"'s heart beat;

a scan path emotional state threshold parameter associated with the user'"'"'s eye rotation;

a fixation period emotional state threshold parameter associated with the user'"'"'s eye rotation;

a skin conductance level emotional state threshold parameter associated with the user'"'"'s perspiration;

a respiration rate emotional state threshold parameter associated with the user'"'"'s respiration;

an oxygen saturation level emotional state threshold parameter associated with the user'"'"'s oxygen saturation;

a blood pressure level emotional state threshold parameter associated with the user'"'"'s blood pressure;

a skin temperature emotional state threshold parameter associated with the user'"'"'s skin temperature;

a muscle tension level emotional state threshold parameter associated with the user'"'"'s muscle tension;

a neural activity level emotional state threshold parameter associated with the user'"'"'s neural activity;

an eye blink rate emotional state threshold parameter associated with the user'"'"'s eye blinking;

a facial muscle movement emotional state threshold parameter associated with the user'"'"'s facial expression;

an acoustic characteristics emotional state threshold parameter associated with the user'"'"'s voice and/or speech;

a contact pressure emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system; and

a contact rate emotional state threshold parameter associated with the user'"'"'s interactions with hardware associated with an interactive software system.

25. The system of claim 16, wherein the historical and/or current operational data includes historical and current operational data selected from the group of historical and current operational data consisting of:

data indicating the historical payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating the historical grouping or relation of one or more invoices similar to one or more of the current invoices represented by the current invoice data;

data indicating recent historical payments made on invoices by the business;

data indicating accounts receivable amounts due to be paid to the business in a defined future time period;

data indicating the historical cash position of the business;

data indicating the historical connection between the cash position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the cash position of the business;

data indicating the current cash position of the business;

data indicating the current cash position of the business compared to the historical cash position of the business;

data indicating the historical revenue position of the business;

data indicating the historical connection between the revenue position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the revenue position of the business;

data indicating the current revenue position of the business;

data indicating the current revenue position of the business compared to the historical revenue position of the business;

data indicating the historical profit position of the business;

data indicating the historical connection between the profit position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the profit position of the business;

data indicating the current profit position of the business;

data indicating the current profit position of the business compared to the historical profit position of the business;

data indicating the historical financial liability position of the business;

data indicating the historical connection between the financial liability position of the business and the payment date of invoices similar to the current invoices represented by the current invoice data;

data indicating seasonal, or temporally repeating, patterns of the financial liability position of the business;

data indicating the current financial liability position of the business; and

data indicating the current financial liability position of the business compared to the historical financial liability position of the business.

26. The system of claim 16, wherein a future emotional state of the user includes at least one future emotional state selected from the group of emotional states consisting of:

a happy emotional state or happiness;

a sad emotional state or sadness;

a surprised emotional state or surprise;

a fearful emotional state or fear;

a disgusted emotional state or disgust;

an angry emotional state or anger;

a tense emotional state;

a nervous emotional state;

a stressed emotional state;

an upset emotional state;

a frustrated emotional state;

a depressed emotional state;

a bored emotional state;

a fatigued emotional state;

an alert emotional state;

an excited emotional state;

an elated emotional state;

a happy emotional state;

a contented emotional state;

a serene emotional state;

a relaxed emotional state; and

a calm emotional state.

27. The system of claim 16, wherein transforming the user experience provided through the interactive software system into a user experience adapted to the predicted future emotional state of the user includes adding one or more user experience components to the user experience.

28. The system of claim 16, wherein transforming the user experience provided through the interactive software system into a user experience adapted to the predicted future emotional state of the user includes removing one or more user experience components from the user experience.

29. The system of claim 16, wherein the user experience adapted to the predicted future emotional state of the user is provided to the user via a user display screen displayed on a user computing system accessible by the user.

30. The system of claim 16, wherein the emotional state data associated with a user is obtained on a periodic basis and the user experience adapted to the predicted future emotional state of the user is modified and provided to the user on a periodic basis.