Intelligent and context aware reading systems

• US 10,306,311 B1
• Filed: 03/22/2017
• Issued: 05/28/2019
• Est. Priority Date: 03/24/2016
• Status: Active Grant

• Current Assignee: Massachusetts Mutual Life Insurance Company
• Original Assignee: Massachusetts Mutual Life Insurance Company
• Inventors: Knas, Michal, John, Jiby
• Primary Examiner(s): Kumar, Pankaj
• Assistant Examiner(s): Boyd, Alexander
• Application Number:
  US15/466,445
• Time in Patent Office: 797 Days
• CPC Class Codes: G06F 2203/011 Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06V 40/174 Facial expression recognition
• CPC Class Codes: G06V 40/19 Sensors therefor
• CPC Class Codes: H04N 1/00381 Input by recognition or interpretation of visible user gestures manual input means, e.g. digitisers, writing tablets H04N1/00392; gesture input to computers G06F3/017; recognition algorithms G06V40/20
• CPC Class Codes: H04N 21/25866 Management of end-user data customer care in data networks H04L41/5077
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4722 for requesting additional data associated with the content

Abstract

Methods and systems for navigating content displayed on a user computing device comprising generating content data configured for presentation on a display of a user computing device, transmitting, the content data to the display of the user computing device for display on the user computing device, collecting behavior data of the user sensed by the tracking module, and determining whether behavior engagement data represents a value that satisfies a threshold value. When the value satisfies the threshold value an instruction request that is associated with the value of the behavior engagement data is generated and when the value does not satisfy the threshold value the same content on the display of the user computing device is displayed to the user.

Claims

1. A method comprising:

generating, by a processing module of a server, content data configured for presentation on a user interface displayed on a user computing device, the content data comprising a first region and a second region where the first region comprises a selectable graphical component;

transmitting, by the processing module of the server, the content data to be displayed on the user interface displayed on the user computing device;

upon the content data displaying on the user interface, activating, by a tracking module of the server, a tracking sensor configured to collect behavior data of a user sensed by the tracking sensor in the user computing device while the user views the content data;

receiving, by the processing module of the server via the tracking sensor, behavior data of the user;

automatically generating, by the tracking module, engagement data based on the collected behavior data;

dynamically determining, by the processing module of the server, whether the engagement data represents a value that satisfies a threshold value;

when the value satisfies the threshold value, generating, by the processing module of the server, an instruction request that is associated with the value of the engagement data, the instruction request configured to automatically select the selectable graphical component, expand the selectable graphical component, and display additional content data in the selected graphical component expanded on the same user interface while the server automatically pauses content data displayed on the second region, the additional content data corresponding to the selection of the selectable graphical component; and

when the value does not satisfy the threshold value, resume presenting, by the processing module of the server, the content data within the second region,

wherein the tracking sensor is an eye-tracking sensor, which extracts information regarding the user'"'"'s eye movement outside of a boundary of the regions of the displayed content data and a time duration of the eye movement of the user, and wherein the threshold value is between 500 ms to 3 seconds.

2. The method according to claim 1, wherein collecting behavior data further comprises automatically extracting behavior data from the user captured using at least one sensor of a head-tracking sensor or an expression processing sensor, wherein the sensor is communicatively coupled to the user computing device.

3. The method according to claim 1, wherein a response to the instruction request is generated by the processing module.

4. The method according to claim 3, wherein the instruction request is further configured to at least one of display supplemental content data, navigate the displayed content data, and change the presentation of the content data.

5. The method according to claim 4, wherein navigating the content includes one of pausing the displayed content data, resuming the displayed content data, scrolling through the content data, skipping forward in the content data or rewinding in the content data.

6. The method according to claim 4, wherein changing the presentation of the content data is one of a request to change the content data layout or change the text size of the content data.

7. The method according to claim, 4 wherein the supplemental content data includes information related to the content data.

8. A system comprising:

a user computing device;

a tracking module of a server, communicatively coupled to the user computing device and a tracking sensor, the tracking module configured to collect, via the tracking sensor, behavior data of a user sensed by the tracking sensor communicatively coupled with the user computing device;

the tracking module configured to generate engagement data based on the collected behavior data; and

a processing module of a server configured to;

display content data on a user interface displayed on a user computing device, the content data comprising a first region second where the first region comprises a selectable graphical component,

upon the content data displaying on the user interface, activating the tracking sensor,

determine if the engagement data represents a value that satisfies a threshold value,

dynamically generate an instruction request that is associated with the value of the engagement data when the value satisfies the threshold value, the instruction request configured to automatically select the selectable graphical component, expand the selectable graphical component, and display additional content data in the selectable graphical component expanded on the same user interface while the server automatically pauses content data displayed on the second region, the additional content data corresponding to the selection of the selectable graphical component,

resume displaying content data within the second region when the value does not satisfy the threshold value,

wherein the tracking sensor is an eye-tracking sensor, which extracts information regarding the user'"'"'s eye movement outside of a boundary of the regions of the displayed content data and a time duration of the eye movement of the user, and wherein the threshold value is between 500 ms to 3 seconds.

9. The system of claim 8, further comprising an external data source.

10. The system of claim 8, wherein the tracking module and the content processing module are located in the server, and the server communicates with the user computing device via a first communication network.

11. The system of claim 10, further comprising an external data source, wherein the server communicates with the external data source via a second communication network.

12. A method comprising:

generating, by a processing module of a server, content data configured for presentation on a user interface displayed on a user computing device, the content data comprising a first region and a second region where the first region comprises a selectable graphical component;

transmitting, by the processing module of the server, the content data to be displayed on the user interface displayed on the user computing device;

upon the content data displaying on the user interface, activating, by a tracking module of the server, a tracking sensor configured to collect behavior data of a user sensed by the tracking sensor in the user computing device while the user views the content data;

receiving, by the processing module of the server via the tracking sensor, behavior data of the user;

automatically generating, by the tracking module, engagement data based on the collected behavior data;

dynamically determining, by the processing module of the server, whether the engagement data represents a value that satisfies a threshold value;

when the value satisfies the threshold value, generating, by the processing module of the server, an instruction request that is associated with the value of the engagement data, the instruction request configured to automatically select the selectable graphical component, expand the selectable graphical component, and display additional content data in the selected graphical component expanded on the same user interface while the server automatically pauses content data displayed on the second region, the additional content data corresponding to the selection of the selectable graphical component; and

when the value does not satisfy the threshold value, resume presenting, by the processing module of the server, the content data within the second region,

wherein the tracking sensor is a head-tracking sensor, which extracts information regarding the user'"'"'s head movement in a direction outside of a boundary of the regions of the displayed content data and a time duration of the head movement of the user, and wherein the threshold value is between 1 second to 4 seconds.

13. The method according to claim 12, wherein collecting behavior data further comprises automatically extracting behavior data from the user captured using at least one sensor of an eye-tracking sensor or an expression processing sensor, wherein the sensor is communicatively coupled to the user computing device.

14. The method according to claim 12, wherein a response to the instruction request is generated by the processing module.

15. The method according to claim 14, wherein the instruction request is further configured to at least one of display supplemental content data, navigate the displayed content data, and change the presentation of the content data.

16. The method according to claim 15, wherein navigating the content includes one of pausing the displayed content data, resuming the displayed content data, scrolling through the content data, skipping forward in the content data or rewinding in the content data.

17. The method according to claim 15, wherein changing the presentation of the content data is one of a request to change the content data layout or change the text size of the content data.

18. The method according to claim, 15 wherein the supplemental content data includes information related to the content data.