Assistive technology (AT) responsive to cognitive states

• US 10,318,094 B2
• Filed: 03/25/2015
• Issued: 06/11/2019
• Est. Priority Date: 03/25/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Keohane, Susann Marie, Kraft, Maureen Elizabeth, Nielsen, Holly Lee, Yan, Shunguo
• Primary Examiner(s): Spieler, William
• Application Number:
  US14/668,943
• Publication Number:
  US 20160283846A1
• Time in Patent Office: 1,539 Days
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/04847 Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
• CPC Class Codes: G06F 40/30 Semantic analysis
• CPC Class Codes: G06F 9/453 Help systems

Abstract

A method of providing guidance in connection with a technological task includes determining, from an interaction of an end user with a computer system, a current context and task of the end user, identifying a change in a cognitive state of the end user, where the change in the cognitive state is associated with performance of the task using the computer system, searching a plurality of user experiences corresponding to the change in the cognitive state of the end user and the user'"'"'s current context, selecting at least one of the user experiences according to a selection parameter for determining a selected user experience, and injecting the selected user experience into the current context of the end user via an output device of the computer system.

Claims

1. A computer program product for providing guidance in connection with a technological task, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processor to:

execute a first application;

execute a screen reader application announcing text output by the first application;

receive a plurality of end user interactions with the first application executing on the computer program product, wherein the screen reader application announces the end user interactions;

determine, as the end user interactions are received, a current context for input received by the computer program product and a current technological task of an end user providing the end user interactions, wherein sequential ones of the current context form a current context stream of contexts and end user interactions, and wherein the current context stream is stored in the buffer of the screen reader application from which the announcements are output;

identify, from the end user interactions, a change in a cognitive state of the end user, where the cognitive states are detected upon determining respective indications in the end user interactions;

identify a plurality of user experiences corresponding to the change in the cognitive state of the end user and the current context;

select at least one of the user experiences according to a selection parameter for determining a selected user experience; and

inject the selected user experience into the current context stream stored in the buffer, wherein the screen reader announces the selected user experience within the current context stream.

2. The computer program product of claim 1, wherein the injection of the selected user experience is a textual input into the current context stream stored in the buffer of the screen reader application.

3. The computer program product of claim 2, further comprising an audio output device, wherein the program instructions executable by the processor further cause the processor:

operate the screen reader application to announce a first audio reading corresponding to the current context of the current context stream using the audio output device;

operate the screen reader application to pause the current context stream including the first audio reading corresponding to the current context using the audio output device; and

operate the screen reader application to announce a second audio reading, during the pause in the first audio reading, corresponding to the textual input using the audio output device.

4. The computer program product of claim 3, wherein the program instructions executable by the processor further cause the processor to distinguish the first audio reading from the second audio reading using a parameter of the screen reader application.

5. The computer program product of claim 3, wherein the parameter of the screen reader application controls a voice characteristic of the screen reader application reading.

6. The computer program product of claim 3, wherein the parameter of the screen reader application controls one or more of a speed and a tone of the screen reader application reading.

7. The computer program product of claim 1, wherein in identifying the plurality of user experiences corresponding to the change in the cognitive state of the end user, the program instructions executable by the processor further cause the processor to:

access a social network of users; and

filter a plurality of available user experiences using the change in the cognitive state of the end user to determine a plurality of potential user experiences.

8. The computer program product of claim 7, wherein in selecting the at least one of the user experiences according to the selection parameter for determining the selected user experience, the program instructions executable by the processor further cause the processor to:

determine the selecting parameter corresponding to the end user; and

apply the selection parameter to the plurality of potential user experiences to determine the selected user experience.

9. A computer system configured to provide guidance in connection with a technological task, the computer system comprising:

a memory storing program instructions of a cognitive monitor, the program instructions embodying;

a context module configured to receive a plurality of end user interactions with the computer system and to determine, as the end user interactions are received, a current context for input received by the computer system and the technological task of an end user providing the end user interactions, wherein sequential ones of the current context form a current context stream of contexts and end user interactions, and wherein the current context stream is stored in a buffer of the cognitive monitor;

a cognitive state module configured to identify, from the end user interactions, a change in a cognitive state of the end user, where the cognitive states are detected upon determining respective indications in the end user interactions;

a database query module configured to interface with a database and to identify a plurality of user experiences in the database corresponding to the change in the cognitive state of the end user and the current context;

an experience selection module configured to select at least one of the user experiences according to a selection parameter for determining a selected user experience; and

an injection module configured to inject the selected user experience into the current context stream stored in the buffer, wherein the injection of the selected user experience is a tangible output of the computer system capable of being perceived by the end user; and

a processor configured to execute the program instructions of the cognitive monitor.

10. The computer system of claim 9, wherein the processor is configured to execute an application facilitating the technological task of the user, and a screen reader outputting an audio based interface of the application, the computer system further comprising an audio output device, wherein the injection of the selected user experience is an audio output of the selected user experience, and wherein the injection of the selected user experience is an audio output of the selected user experience augmenting the audio based interface of the screen reader.