ERGONOMIC AND SENSOR ANALYSIS BASED USER EXPERIENCE DESIGN

• US 20200133450A1
• Filed: 10/30/2018
• Published: 04/30/2020
• Est. Priority Date: 10/30/2018
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: LU, Fang, KOCHURA, Nadiya
• Granted Patent: US 10,831,346 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: A61B 5/1116 Determining posture transitions
• CPC Class Codes: A61B 5/746 Alarms related to a physiological condition, e.g. details of setting alarm thresholds or avoiding false alarms alarms per se G08B
• CPC Class Codes: G06F 3/023 Arrangements for converting discrete items of information into a coded form, e.g. arrangements for interpreting keyboard generated codes as alphanumeric codes, operand codes or instruction codes
• CPC Class Codes: G06F 3/033 Pointing devices displaced or positioned by the user , e.g. mice, trackballs, pens or joysticks; Accessories therefor digitisers characterised by the transducing means G06F3/041
• CPC Class Codes: G06F 3/038 Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04812 Interaction techniques based on cursor appearance or behaviour, e.g. being affected by the presence of displayed objects
• CPC Class Codes: G06F 3/0484 for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
• CPC Class Codes: G06F 3/04842 Selection of displayed objects or displayed text elements G06F3/0482 takes precedence
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 8/77 Software metrics
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G06V 20/52 Surveillance or monitoring of activities, e.g. for recognising suspicious objects recognising microscopic objects G06V20/69
• CPC Class Codes: G06V 40/174 Facial expression recognition

Abstract

Ergonomic and sensor analysis based user experience design improvement is provided in various embodiments by: detecting interactions between a user and an element of a user interface of a software application executing on a computing device; gathering biometric data for the user from interface devices of the computing device; identifying a task performed in the software application that corresponds to the interactions and associates the biometric data with the interactions; adjusting a user interface setting for the element to reduce a strain on the user, wherein the user interface setting comprises at least one of: a location of the element relative to a second element of the user interface involved in the task; a relative size of the element in the user interface; a color balance of the element; and a size and location of a hitbox of the element for registering user selection of the element.

Claims

1. A method, comprising:

detecting interactions between a user and a first element of a user interface of a software application executing on a computing device;

gathering biometric data for the user from interface devices of the computing device;

identifying a task performed in the software application that corresponds to the interactions and associates the biometric data with the interactions;

in response to determining that a strain on the user when performing the task exceeds an ergonomic threshold, adjusting a user interface setting for the first element to reduce the strain, wherein the user interface setting comprises at least one of;

a location of the first element relative to a second element of the user interface involved in the task;

a relative size of the first element in the user interface;

a color balance of the first element; and

a size and location of a hitbox of the first element for registering user selection of the first element; and

creating a shortcut in the user interface to perform the task a second time on selection of the shortcut.

2. The method of claim 1, further comprising:

in response to determining that the strain on the user when performing the task exceeds the ergonomic threshold, transmitting a strain report including the task and interactions related to the task to a development team associated with the software application.

3. The method of claim 1, wherein attempted interactions with the first element that do not result in actual interactions with the first element are included in the task.

5. The method of claim 1, wherein the strain on the user is determined based on:

a distance between the user and a display device displaying the user interface during performance of the task;

an angle between the user and the display device during performance of the task; and

a travel between an initial position of the first element and a final position of the first element during the task.

6. The method of claim 1, wherein the strain on the user is determined based on at least one of:

a number of interactions with a pointer device during performance of the task;

a length of drags of the first element within the user interface during performance of the task; and

a force of interactions with the pointing device during performance of the task.

7. The method of claim 1, wherein the strain on the user is determined based on a facial expression captured by a camera during performance of the task.

4. (canceled)

8. A computer readable medium including instructions that when executed by a processor enable the processor to perform an operation, the operation comprising:

detecting interactions between a user and a first element of a user interface of a software application running on a computing device;

gathering biometric data for the user from interface devices of the computing device;

identifying a task performed in the software application that corresponds to the interactions and associates the biometric data with the interactions;

determining a strain on the user based on;

a distance between the user and a display device displaying the user interface during performance of the task;

an angle between the user and the display device during performance of the task; and

a travel between an initial position of the first element and a final position of the first element during the task;

in response to determining that the strain on the user in performing the task exceeds an ergonomic threshold, adjusting a user interface setting for the first element to reduce the strain, wherein the user interface setting comprises at least one of;

a location of the first element relative to a second element of the user interface involved in the task;

a relative size of the first element in the user interface;

a color balance of the first element; and

a size and location of a hitbox of the first element for registering user selection of the first element.

9. The computer readable medium of claim 8, further comprising:

in response to determining that the strain on the user in performing the task exceeds the ergonomic threshold, transmitting a strain report including the task and interactions related to the task to a development team associated with the software application.

10. The computer readable medium of claim 8, wherein attempted interactions with the first element are included in the task.

11. The computer readable medium of claim 8, wherein in addition to adjusting the user interface setting for the first element, a shortcut to perform the task is created.

13. The computer readable medium of claim 8, wherein the strain on the user is determined based on at least one of:

a number of interactions with a pointer device during performance of the task;

a length of drags of the first element within the user interface during performance of the task; and

a force of interactions with the pointing device during performance of the task.

14. The computer readable medium of claim 8, wherein the strain on the user is determined based on a facial expression captured by a camera during performance of the task.

12. (canceled)

15. A system, comprising:

a processor; and

a memory storage device, including instructions that when executed by the processor enable the system to;

detect interactions between a user and a first element of a user interface of a software application running on a computing device;

gather biometric data for the user from interface devices of the computing device;

identify a task performed in the software application that corresponds to the interactions and associates the biometric data with the interactions;

determine a strain on the user based on at least one of;

a number of interactions with a pointer device during performance of the task;

a length of drags of the first element within the user interface during performance of the task; and

a force of interactions with the pointing device during performance of the task;

in response to determining that the strain on the user in performing the task exceeds an ergonomic threshold, adjust a user interface setting for the first element to reduce the strain, wherein the user interface setting comprises at least one of;

a location of the first element relative to a second element of the user interface involved in the task;

a relative size of the first element in the user interface;

a color balance of the first element; and

a size and location of a hitbox of the first element for registering user selection of the first element.

16. The system of claim 15, wherein the instructions further enable the system to:

in response to determining that the strain on the user in performing the task exceeds the ergonomic threshold, transmit a strain report including the task and interactions related to the task to a development team associated with the software application.

17. The system of claim 15, wherein attempted interactions with the first element are included in the task.

18. The system of claim 15, wherein in addition to adjusting the user interface setting for the first element, a shortcut to perform the task is created.

19. The system of claim 15, wherein the strain on the user is determined based on:

a distance between the user and a display device displaying the user interface during performance of the task;

an angle between the user and the display device during performance of the task; and

a travel between an initial position of the first element and a final position of the first element during the task.

20. (canceled)