Systems and methods for emotive software usability

• US 8,869,115 B2
• Filed: 04/20/2012
• Issued: 10/21/2014
• Est. Priority Date: 11/23/2011
• Status: Active Grant

• Current Assignee: General Electric Company
• Original Assignee: General Electric Company
• Inventors: Bruns, Kirk Lars, Olivier, Christopher John, Das, Piali, Tu, Peter, Liu, Xiaoming
• Primary Examiner(s): Dao, Thuy
• Assistant Examiner(s): Sinha, Ravi K
• Application Number:
  US13/452,329
• Publication Number:
  US 20130132931A1
• Time in Patent Office: 914 Days
• Field of Search: None
• US Class Current: 717/124
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces

Abstract

Systems and methods are disclosed for emotive healthcare software usability. A method to improve software usability is described, the method comprising presenting a software application to a user. The method also including logging activities of the user with respect to the software application, wherein the logging includes recording the user using the software application, and wherein the activities include user action with respect to the software application and mouse location on a user interface displaying the software application. The method also including interpreting user emotion from the recording, and tracking an emotive index based on a combination of user emotion and user action with respect to the software application and mouse location. The method also including providing feedback based on the emotive index.

Claims

1. A method to improve software usability, the method comprising:

providing, with a first software application, feedback with respect to a feature of a second software application, the first software application executed independently from the second software application, the first software application providing feedback with respect to a feature by;

logging activities of a user while the user is using the second software application, the activities including user actions with respect to the second software application at a first time and a second time and mouse locations on a user interface displaying the second software application at the first time and the second time;

capturing an image stream of the user while the user is using the second software application, the image stream including gaze locations of the user while the user is using the second software application at the first time and the second time;

interpreting, from the image stream, user emotions of the user while the user is using the second software application at the first time and the second time;

tracking an emotive index of the user with respect to the user using the second software application, wherein the emotive index is based on changes in (1) the user emotions, (2) the gaze locations and (3) the mouse locations between the first time and the second time; and

correlating the emotive index with a feature of the second software application based on at least one of a user action or a mouse location of a logged activity at the first time or the second time,

wherein the feedback comprises a combination of the image stream of the user, gaze tracking of the user, and the logged activities with respect to the feature of the second software application.

2. The method of claim 1, further comprising providing training to the user based on the feedback.

3. The method of claim 1, further comprising tailoring a workflow for the user based on the feedback.

4. The method of claim 1, further comprising facilitating modification of the second software application based on the feedback.

5. The method of claim 1, wherein capturing the image stream further comprises recording the gaze locations of the user with respect to the user interface.

6. The method of claim 1, wherein the feedback comprises a playback of the combination of the image stream of the user, the gaze tracking of the user, and the logged activities with respect to the feature of the second software application.

7. The method of claim 1, wherein providing the feedback with respect to the feature further comprises:

identifying an area of the user interface of the second software application associated with frustration based on a combination of a screen capture including a gaze representation, mouse movement shown as lines on a screen, an emotive graph, and listing of user actions; and

associating the feature with the identified area of the user interface.

8. A non-transitory computer readable medium including computer program code to be executed by a processor, the computer program code, when executed, to implement a method to improve software usability, the method comprising:

providing, with a first software application, feedback with respect to a feature of a second software application, the first software application executed independently from the second software application, the first software application providing feedback with respect to a feature by;

logging activities of a user while the user is using the second software application, the activities including user actions and reactions with respect to the second software application at a first time and a second time and mouse locations on a user interface displaying the second software application at the first time and the second time;

capturing an image stream of the user while the user is using the second software application, the image stream including gaze locations of the user while the user is using the second software application at the first time and the second time;

interpreting, from the image stream, user emotions of the user while the user is using the second software application at the first time and the second time;

tracking an emotive index of the user with respect to the user using the second software application, wherein the emotive index is based on changes in (1) the user emotions, (2) the gaze locations and (3) the mouse locations between the first time and the second time; and

correlating the emotive index with a feature of the second software application based on at least one of a user action or a mouse location of a logged activity at the first time or the second time,

wherein the feedback comprises a combination of the image stream of the user, gaze tracking of the user, and the logged activities with respect to the feature of the second software application.

9. The non-transitory computer readable medium of claim 8, the method further comprising providing training to the user based on the feedback.

10. The non-transitory computer readable medium of claim 8, the method further comprising tailoring a workflow for the user based on the feedback.

11. The non-transitory computer readable medium of claim 8, the method further comprising facilitating modification of the second software application based on the feedback.

12. The non-transitory computer readable medium of claim 8, wherein the method further comprises capturing the image stream by recording the gaze locations of the user with respect to the user interface.

13. The non-transitory computer readable medium of claim 8, wherein the feedback comprises a playback of the combination of the image stream of the user, the gaze tracking of the user, and the logged activities with respect to the feature of the second software application.

14. The non-transitory computer readable medium of claim 8, wherein the method further comprises providing the feedback with respect to the feature by:

identifying an area of the user interface of the second software application associated with frustration based on a combination of a screen capture including a user gaze representation, mouse movement shown as lines on a screen, an emotive graph, and listing of user actions; and

associating the feature with the identified area of the user interface.

15. A system to improve software usability comprising:

a processor; and

a memory having machine readable instructions stored thereon which, when executed, cause the processor to at least;

log, with a first software application, activities of a user while the user is using a second software application, the second software application to display to the user via a user interface, wherein the activities to include user actions with respect to the second software application at a first time and a second time and mouse locations on the user interface displaying the second software application at the first time and the second time;

capture, with the first software application, an image stream of the user while the user is using the second software application, wherein the image stream is to include gaze locations of the user while the user is using the second software application at the first time and the second time;

interpret, with the first software application, user emotions of the user while the user is using the second software application, wherein the user emotions are interpreted from the image stream at the first time and the second time;

track, with the first software application, an emotive index of the user with respect to the user using the second software application, wherein the emotive index is based on changes in (1) the user emotions, (2) the gaze locations and (3) the mouse locations between the first time and the second time;

correlate the emotive index with a feature of the second software application based on at least one of user action or a mouse location of a logged activity at the first time or the second time; and

provide feedback with respect to the feature of the second software application, the feedback to comprise a combination of the image stream of the user, gaze tracking of the user, and the logged activities with respect to the feature of the second software application.

16. The system of claim 15, wherein the instructions cause the processor to provide training to the user based on the feedback.

17. The system of claim 15, wherein the instructions cause the processor to tailor a workflow for the user based on the feedback.

18. The system of claim 15, wherein the instructions cause the processor to facilitate modification of the second software application based on the feedback.

19. The system of claim 15, wherein the instructions cause the processor to capture the image stream by recording the gaze locations of the user with respect to the user interface.

20. The system of claim 15, wherein the feedback comprises a playback of the combination of the image stream of the user, the gaze tracking of the user, and the logged activities with respect to the feature of the second software application.

21. The system of claim 15, wherein to provide the feedback with respect to the feature further comprises instructions to cause the processor to:

identify an area of the user interface of the second software application associated with frustration based on a combination of screen captures including gaze representations, mouse movement shown as lines on a screen, an emotive graph, and listing of user actions; and

associate the feature with the identified area of the user interface.