Assistive technology (AT) responsive to cognitive states

• US 10,359,836 B2
• Filed: 12/31/2015
• Issued: 07/23/2019
• Est. Priority Date: 03/25/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Keohane, Susann Marie, Kraft, Maureen Elizabeth, Nielsen, Holly Lee, Yan, Shunguo
• Primary Examiner(s): Spieler, William
• Application Number:
  US14/986,570
• Publication Number:
  US 20160283847A1
• Time in Patent Office: 1,300 Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/01 Input arrangements or combined input and output arrangements for interaction between user and computer G06F3/16 takes precedence
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback

Abstract

A method of providing guidance in connection with a technological task includes determining, from an interaction of an end user with a computer system, a current context and task of the end user, identifying a change in a cognitive state of the end user, where the change in the cognitive state is associated with performance of the task using the computer system, searching a plurality of user experiences corresponding to the change in the cognitive state of the end user and the user'"'"'s current context, selecting at least one of the user experiences according to a selection parameter for determining a selected user experience, and injecting the selected user experience into the current context of the end user via an output device of the computer system.

Claims

1. A method of providing guidance in connection with a technological task comprises:

executing, by a computer system, a first application;

executing, by the computer system, a screen reader application augmenting the first application;

determining, from a plurality of end user interactions of an end user with the first application executing on the computer system, a current context for input received by the computer system and a current technological task of an end user providing the end user interactions, wherein sequential ones of the current context form a current context stream of contexts and end user interactions, and wherein the current context stream is stored in a buffer of the screen reader application from which announcements are output;

identifying, from the end user interactions, a change in a cognitive state of the end user, where the cognitive states are detected upon determining respective indications in the end user interactions;

identifying a plurality of user experiences corresponding to the change in the cognitive state of the end user and the current context;

selecting at least one of the user experiences according to a selection parameter for determining a selected user experience; and

injecting the selected user experience into the current context stream stored in the buffer, wherein the screen reader application announces the selected user experience within the current context stream.

2. The method of claim 1, wherein the injection of the selected user experience is a textual input into the current context stream stored in the buffer of the screen reader application.

3. The method of claim 2, further comprising:

operating the screen reader application to announce a first audio reading corresponding to the current context of the current context stream;

operating the screen reader application to pause the current context stream including the first audio reading corresponding to the current context using the audio output device; and

operating the screen reader application to announce a second audio reading, during the pause in the first audio reading, corresponding to the textual input.

4. The method of claim 3, further comprising distinguishing the first audio reading from the second audio reading using a parameter of the screen reader application.

5. The method of claim 3, wherein the parameter of the screen reader application controls a voice characteristic of the screen reader application reading.

6. The method of claim 3, wherein the parameter of the screen reader application controls one or more of a speed and a tone of the screen reader application reading.

7. The method of claim 1, wherein identifying the plurality of user experiences corresponding to the change in the cognitive state of the end user comprises:

accessing a social network of users; and

filtering a plurality of available user experiences using the change in the cognitive state of the end user to determine a plurality of potential user experiences.

8. The method of claim 7, wherein selecting the at least one of the user experiences according to the selection parameter for determining the selected user experience comprises:

determining the selecting parameter corresponding to the end user; and

applying the selection parameter to the plurality of potential user experiences to determine the selected user experience.

9. The method of claim 1, wherein identifying the change in the cognitive state of the end user further comprises identifying an issue corresponding to the change in the cognitive state of the end user, wherein the identification of the plurality of user experiences further includes searching a plurality of user experiences corresponding to the change in the cognitive state of the end user and the issue.