Methods and apparatus for controlling a user interface based on the emotional state of a user

• US 8,701,003 B2
• Filed: 10/19/2009
• Issued: 04/15/2014
• Est. Priority Date: 07/22/2002
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Verizon Services Corporation (Verizon Communications Inc.)
• Inventors: Kondziela, James Mark
• Primary Examiner(s): PITARO, RYAN F
• Application Number:
  US12/581,675
• Publication Number:
  US 20100037187A1
• Time in Patent Office: 1,639 Days
• Field of Search: 715/707, 715/744
• US Class Current: 715/707
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 9/451 Execution arrangements for user interfaces
• CPC Class Codes: G10L 17/26 Recognition of special voice characteristics, e.g. for use in lie detectors; Recognition of animal voices

Abstract

Methods and apparatus for modifying a user interface as a function of the detected emotional state of a system user are described. In one embodiment, stress analysis is performed on received speech to generate an emotional state indicator value, e.g., a stress level indicator value. The stress level indicator value is compared to one or more thresholds. If a first threshold is exceeded the user interface is modified, e.g., the presentation rate of speech is slowed. If a second threshold is not exceeded, another modification to the user interface is made, e.g., the speech presentation rate is accelerated. If the stress level indicator value is between first and second thresholds, user interface operation continues unchanged. The user interface modification techniques of the present invention may be used in combination with known knowledge or expertise based user interface adaptation features.

Claims

1. A method controlling an apparatus, the method comprising the steps of:

generating an emotional state indicator value by performing stress analysis on received user speech;

comparing said emotional state indicator value to a first threshold to determine if said emotional state indicator value is above said first threshold;

comparing said emotional state indicator value to a second threshold to determine if said emotional state indictor value is below said second threshold; and

modifying a user interface in response to determining that said emotional state indicator value is above said first threshold or below said second threshold.

2. The method of claim 1, wherein modifying a user interface includes slowing the rate at which prompts are provided to a user of said apparatus when said emotional state indicator value is above said first threshold.

3. The method of claim 1, wherein modifying said user interface includes, when said emotional state indicator value is determined to be below said second threshold, speeding up the rate at which prompts are provided to a user of said apparatus.

4. The method of claim 1, further comprising:

leaving said user interface unmodified when said emotional state indicator value is between said first and second thresholds.

5. The method of claim 1, wherein the step of modifying a user interface includes:

altering the length of a pause between sentences included in speech generated by said user interface.

6. The method of claim 1, wherein the step of modifying a user interface includes:

changing the number of options presented to a user as part of a menu from which the user can make a selection.

7. The method of claim 1, wherein the step of modifying a user interface includes:

changing the rate at which prompts are provided to a user of said apparatus;

changing the length of a pause between sentences included in speech generated by said user interface; and

changing the number of options presented to a user as part of a menu from which the user can make a selection.

8. The method of claim 1, further comprising the steps of:

periodically performing said generating step using speech input received during different periods of time to generate a set of emotional state indicator values, each emotional state indicator value corresponding to a different period of time; and

monitoring said emotional state indicator values to determine if the generated emotional state indicator values stay above said second threshold for a predetermined period of time.

9. The method of claim 8, wherein said apparatus is a telephone call servicing apparatus, the method further comprising the step of:

connecting a caller from which said speech input is received to a human service representative when it is determined that the generated emotional state indicator values stay above said second threshold for said predetermined period of time.

10. The method of claim 9, wherein said telephone call servicing apparatus is a telephone operator terminal.

11. An apparatus used to interact with a user of said apparatus, the apparatus comprising:

a stress analysis module configured to generate an emotional state indicator value by performing stress analysis on received user speech;

a control module configured to;

compare said emotional state indicator value to a first threshold to determine if said emotional state indicator value is above said first threshold;

compare said emotional state indicator value to a second threshold to determine if said emotional state indictor value is below said second threshold; and

modify a user interface in response to determining that said emotional state indicator value is above said first threshold or below said second threshold.

12. The apparatus of claim 11, wherein said control module is further configured to, in modifying a user interface, slow the rate at which prompts are provided to a user of said apparatus when said emotional state indicator value is above said first threshold.

13. A machine-readable digital data storage device comprising a set of machine executable instructions for controlling a machine to perform the steps of:

generating an emotional state indicator value by performing stress analysis on received user speech;

comparing said emotional state indicator value to a first threshold to determine if said emotional state indicator value is above said first threshold;

comparing said emotional state indicator value to a second threshold to determine if said emotional state indictor value is below said second threshold; and

modifying a user interface in response to determining that said emotional state indicator value is above said first threshold or below said second threshold.

14. The machine-readable digital data storage device of claim 13, wherein the machine executable instructions for modifying a user interface include instructions for modifying the rate at which prompts are provided to a user of said apparatus.