Gaze based classroom notes generator

• US 10,643,485 B2
• Filed: 03/30/2017
• Issued: 05/05/2020
• Est. Priority Date: 03/30/2017
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Ahuja, Karan, Dey, Kuntal, Dwivedi, Utkarsh, Nagar, Seema, Vaculin, Roman
• Primary Examiner(s): Egloff, Peter R
• Application Number:
  US15/474,475
• Publication Number:
  US 20180286260A1
• Time in Patent Office: 1,132 Days
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G09B 5/02 with visual presentation of the material to be studied, e.g. using film strip
• CPC Class Codes: G09B 5/08 providing for individual presentation of information to a plurality of student stations

Abstract

Techniques are provided for creating presentation notes based upon gaze tracking information associated with observers of a presentation. In one example, a computer-implemented method comprises: obtaining, by a system operatively coupled to a processor, gaze information associated with observers of a presentation; determining, by the system, respective content clarity scores for content elements of the presentation based on a content clarity function; selecting, by the system, respective content from one or more content sources for the content elements based on the respective content clarity scores; and generating, by the system, presentation notes based on the presentation and the selected respective content for the content elements of the presentation.

Claims

1. A system, comprising:

a memory that stores computer executable components;

a processor, operably coupled to the memory, and that executes computer executable components stored in the memory, wherein the computer executable components comprise;

a gaze tracking component that obtains gaze information associated with observers of a presentation;

an attention scoring component that determines respective attention scores for content elements of the presentation based on the gaze information, wherein the attention scoring component determines an attention score for a content element of the content elements based on an attention function that employs a cluster average of saccade paths for the content element determined based on the gaze information, wherein the cluster average of saccade paths comprises a set of respective representative saccade paths for clusters of saccade paths associated with the content element;

a content recommendation component that;

determines respective content clarity scores for content elements of the presentation based on a content clarity function and the gaze information, wherein the content clarity function employs the attention score for the content element; and

selects respective content, from one or more content sources, for the content elements based on the respective content clarity scores; and

a notes generation component that generates presentation notes based on the presentation and the selected respective content for the content elements of the presentation.

2. The system of claim 1, wherein the notes generation component further controls a device associated with an observer to display the presentation notes as the observer is reviewing the presentation.

3. The system of claim 1, wherein the attention function further employs one or more statistics selected from a group consisting of a total fixation duration for the content element, a number of fixations for the content element, an average fixation duration for the content element, and a percentage viewing time for the content element.

4. The system of claim 1, further comprising a content component that obtains respective importance levels for the content elements, wherein the content clarity function further employs an importance level for the content element.

5. The system of claim 1, further comprising a content component that obtains respective effort levels for the content elements, wherein the content clarity function further employs an effort level for the content element.

6. The system of claim 1, further comprising a content component that obtains a heuristic for the content element, wherein the content clarity function further employs the heuristic for the content element.

7. The system of claim 1, wherein the content recommendation component determines the content clarity function based upon a deep learning model that is optimized for learning for the observers.

8. A computer program product facilitating generation of presentation notes for a presentation, the computer program product comprising a computer readable storage medium having program instructions embodied therewith, the program instructions executable by a processor to cause the processing component to:

obtain gaze information associated with observers of a presentation;

determine respective attention scores for content elements of the presentation based on the gaze information, wherein the determining an attention score for a content element of the content elements is based on an attention function that employs a cluster average of saccade paths for the content element determined based on the gaze information, wherein the cluster average of saccade paths comprises a set of respective representative saccade paths for clusters of saccade paths associated with the content element;

determine respective content clarity scores for content elements of the presentation based on a content clarity function and the gaze information, wherein the content clarity function employs the attention score for the content element;

select respective content, from one or more content sources, for the content elements based on the respective content clarity scores; and

generate presentation notes based on the presentation and the selected respective content for the content elements of the presentation.

9. The computer program product of claim 8, wherein the program instructions executable by the processor further cause the processor to:

control a device associated with an observer to display the presentation notes as the observer is reviewing the presentation.

10. The computer program product of claim 9, wherein attention function further employs one or more statistics selects from a group consisting of a total fixation duration for the content element, a number of fixations for the content element, an average fixation duration for the content element, and a percentage viewing time for the content element.

11. The computer program product of claim 8, wherein the program instructions executable by the processor further cause the processor to:

obtain respective importance levels for the content elements, wherein the content clarity function further employs an importance level for the content element.

12. The computer program product of claim 11, wherein the program instructions executable by the processor further cause the processor to:

obtain respective effort levels for the content elements, wherein the content clarity function further employs an effort level for the content element.

13. The computer program product of claim 11, wherein the program instructions executable by the processor further cause the processor to:

obtain a heuristic for the content element, wherein the content clarity function further employs the heuristic for the content element.