Updating displays based on attention tracking data

• US 10,198,233 B2
• Filed: 03/01/2016
• Issued: 02/05/2019
• Est. Priority Date: 03/01/2016
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Grosse-Puppendahl, Tobias Alexander, Scott, James, Hodges, Stephen Edward, Helmes, John Franciscus Marie, Taylor, Stuart Alastair, Chen, Nicholas Yen-Cherng, Knierim, Pascal, Wood, Gavin, Fromm, Josh, Schmidt, Albrecht
• Primary Examiner(s): Wang, Samantha (Yuehan)
• Assistant Examiner(s): Le, Michael
• Application Number:
  US15/058,012
• Publication Number:
  US 20170255439A1
• Time in Patent Office: 1,071 Days
• CPC Class Codes: G06F 3/012 Head tracking input arrangements
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/0481 based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
• CPC Class Codes: G06F 3/1407 General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
• CPC Class Codes: G06Q 10/10 Office automation; Time management
• CPC Class Codes: G06Q 10/109 Time management, e.g. calendars, reminders, meetings or time accounting
• CPC Class Codes: H04N 21/4312 involving specific graphical features, e.g. screen layout, special fonts or colors, blinking icons, highlights or animations
• CPC Class Codes: H04N 21/4316 for displaying supplemental content in a region of the screen, e.g. an advertisement in a separate window
• CPC Class Codes: H04N 21/44218 Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program methods or arrangements for recognising human body or animal bodies or body parts G06V40/10; methods or arrangements for acquiring or recognising human faces, facial parts, facial sketches, facial expressions G06V40/16; methods or arrangements for recognising movements or behaviour G06V40/20; arrangements for identifying users in broadcast systems H04H60/45
• CPC Class Codes: H04N 21/4882 for displaying messages, e.g. warnings, reminders arrangements for providing short real-time information to substation in data switching networks H04L12/1895

Abstract

A system is described which comprises an input arranged to receive attention tracking data from attention tracking hardware. The attention tracking data, along with parameters associated with each display change request, is then used to select a display change request from a plurality of stored display change requests received from a plurality of different applications and a change defined by the selected display change request is then rendered in a GUI on a display device.

Claims

1. A system comprising:

an input device configured to receive attention tracking data from attention tracking hardware;

a processor; and

a memory arranged to store pending display change requests and a display change control module comprising device-executable instructions, which when executed, cause the processor to;

receive display change requests from a plurality of applications;

store the change requests in the memory;

select a change request from those stored in the memory based on the attention tracking data and parameters associated with each change request, wherein the parameters include a first time window parameter indicating a first time window for rendering a change defined in the selected change request;

identify one or more other change requests of the stored change requests that have a second time window indicating a second time for rendering one or more changes defined in the one or more other change requests, wherein the one or more other change requests is identified based on an overlap between the first time window and the second time window;

identify a plurality of screens of a display device;

based on the attention tracking data and parameters, determine a screen of the plurality of screens for rendering the change defined by the selected change request; and

cause the change defined by the selected change request and the one or more other changes defined by the one or more other change requests to be rendered in a graphical user interface (GUI) on the determined screen of the display device, wherein determination of the screen is based at least in part on an attention position relative to members of the plurality of screens, and wherein the change defined by the selected change request and the one or more other changes defined by the one or more other change requests are rendered as a group.

2. A system according to claim 1, wherein the memory is further arranged to store a attention tracking module comprising device-executable instructions, which when executed, cause the processor to determine a user'"'"'s attention location based on the attention tracking data,

and wherein the display change control module comprises device-executable instructions, which when executed, cause the processor to select the change request from those stored in the memory based on the user'"'"'s attention location and parameters associated with each change request.

3. A system according to claim 1, further comprising the attention tracking hardware.

4. A system according to claim 1, further comprising the display device.

5. A system according to claim 1, wherein the display device is a shared display device.

6. A system according to claim 1, wherein the display device is a persistent display device.

7. A system according to claim 1, wherein the display change control module further comprises device-executable instructions, which when executed, cause the processor to determine, based on the parameters associated with the selected change request, if a user acknowledgement is required and in response to determining that a user acknowledgement is required, to determine whether a user acknowledgement has been received by analyzing the attention tracking data.

8. A system according to claim 7, wherein the display change control module further comprises device-executable instructions, which when executed, cause the processor, in response to determining that a user acknowledgement has not been received, to reverse the rendering of the change defined by the selected change request in the GUI.

9. A system according to claim 7, wherein the display change control module further comprises device-executable instructions, which when executed, cause the processor, in response to determining that a user acknowledgement has been received, to send a trigger to activate the application from which the selected change was received.

10. A system according to claim 1, wherein the display change control module further comprises device-executable instructions, which when executed, cause the processor, after causing a change defined by the selected change request to be rendered in a GUI on the display device, to send a trigger to activate the application from which the selected change was received.

11. A system according to claim 1, wherein the display change control module further comprises device-executable instructions, which when executed, cause the processor to select the display device on which to render the change from a plurality of display devices.

12. A system according to claim 11, wherein the plurality of display devices comprise display devices in a plurality of different user computing devices.

13. A system according to claim 1, wherein the change request is selected based on the attention tracking data and parameters associated with each change request and user identity data.

14. A system according to claim 1, wherein the parameters associated with a change request specify at least one of:

whether selection of the change request can he delayed, a manner in which the change request is to be displayed to a user, and whether user acknowledgment of the change request is required.

15. A system as claimed in claim 1, wherein the display change control module is at least partially implemented using hardware logic selected from any one or more of:

a field-programmable gate array, a program-specific integrated circuit, a program-specific standard product, a system-on-a-chip, a complex programmable logic device.

16. A method comprising:

receiving, at a computing device, display change requests generated from a plurality of different applications and one or more parameters associated with each display change request;

receiving attention tracking data from attention tracking hardware associated with a display device;

selecting a display change request based on the attention tracking data and the parameters associated with each stored display request, wherein the parameters include a first time window parameter indicating a first time window for rendering a change defined in the selected change request;

identifying one or more other change requests of the display change requests that have a second time window indicating a second time for rendering one or more changes defined in the one or more other change requests, wherein the one or more other change requests is identified based on an overlap between the first time window and the second time window;

identifying a plurality of screens of the display device, based on the attention tracking data and parameters;

determining a screen of the plurality of screens for rendering a display change defined by the selected change request, wherein determining the screen is based at least in part on an attention position relative to members of the plurality of screens; and

causing the display change defined by the selected display change request and the one or more other changes defined by the one or more other change requests to be rendered in a graphical user interface (GUI) on the determined screen of the display device, wherein the change defined by the selected change request and the one or more other changes defined by the one or more other change requests are rendered as a group.

17. A method according to claim 16, further comprising:

determining, based on the parameters associated with the selected display change request, whether an acknowledgement by a user is required; and

in response to determining that an acknowledgement is required, analyzing the attention tracking data to determine if a user acknowledgement is present.

18. A method according to claim 17, further comprising at least one of:

in response to determining that no user acknowledgement is present, causing the display change defined by the selected display change request to be reversed;

or

in response to determining that a user acknowledgement is present, sending a trigger to wake the application from which the selected display change request was received.

19. A method as claimed in claim 16 at least partially carried out using hardware logic.

20. A system comprising:

an input device configured to receive attention tracking data from an attention tracking system associated with a display device;

a memory device configured to store display change requests and parameters associated with each stored display change request; and

centralized display change control circuitry that causes at least one processor to;

receive display change requests from a plurality of applications;

store each received display change request in the memory;

select a display change request from the stored display change requests based on the attention tracking data and parameters associated with each stored display change request, wherein the parameters include a first time window parameter indicating a first time window for rendering a change defined in the selected change request

identify one or more other change requests of the stored change requests that have a second time window indicating a second time for rendering one or more changes defined in the one or more other change requests, wherein the one or more other change requests is identified based on an overlap between the first time window and the second time window;

identify a plurality of screens of the display device;

based on the attention tracking data and parameters, determine a screen of the plurality of screens for rendering a display change defined by the selected change request; and

render the display change defined by the selected display change request and the one or more other changes defined by the one or more other change requests in a graphical user interface (GUI) on the determined screen of the display device, wherein determination of the screen is based at least in part on an attention position relative to members of the plurality of screens, and wherein the change defined by the selected change request and the one or more other changes defined by the one or more other change requests are rendered as a group.