Attention detection service

• US 10,642,335 B2
• Filed: 10/18/2017
• Issued: 05/05/2020
• Est. Priority Date: 06/03/2017
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Biron, Benjamin, Misra, Ronnie G., Thomas, Christopher K., Zhang, Weijie, Yepez, Roberto G., Guetta, Anthony J., Ho, Kelsey Y., Chinn, Paul W., Haggerty, Myra, Rothert, Curtis, Anton, Peter D., Drummond, Jonathan E., Cato, Andrew B.
• Primary Examiner(s): Johnson, Terrell S
• Application Number:
  US15/787,307
• Publication Number:
  US 20180348842A1
• Time in Patent Office: 930 Days
• CPC Class Codes: G06F 1/3231 Monitoring the presence, absence or movement of users
• CPC Class Codes: G06F 1/324 by lowering clock frequency
• CPC Class Codes: G06F 1/3265 Power saving in display device
• CPC Class Codes: G06F 1/3287 by switching off individual functional units in the computer system
• CPC Class Codes: G06F 1/3296 by lowering the supply or operating voltage
• CPC Class Codes: Y02D 10/00 Energy efficient computing, e.g. low power processors, power management or thermal management

Abstract

In an embodiment, a device may include an attention detection service. The attention detection service may monitor various peripheral devices in the device for indications that a user is paying attention to the device. Various clients may register for notification of attention detection and attention lost (attention no longer detected) events, or may poll the service for the events. If a user is not paying attention to the device, it may be possible to take various actions to permit reduced energy consumption.

Claims

1. A non-transitory computer accessible storage medium storing instructions forming an attention detection service which, when executed on a system:

monitor a plurality of peripheral devices in the system for a plurality of events that indicate that a user is potentially paying attention to the system; and

responsive to a first event of the plurality of events from one of the plurality of peripheral devices, communicate an indication of the potential user attention to one or more clients of a plurality of clients executable on the system, wherein the one or more clients have previously registered with the attention detection service to indicate that the first event is an event that the one or more clients interpret as indicative of attention, and wherein one or more other clients of the plurality of clients have previously registered with the attention detection service to indicate that one or more second events of the plurality of events are events that the one or more other clients interpret as indicative of attention but do not indicate that the first event is an event that the one or more other clients interpret as indicative of attention.

2. The non-transitory computer accessible storage medium as recited in claim 1 wherein the instructions, when executed responsive to a register request from a first client to indicate which of the plurality of events the first client interprets as indicative of attention:

record a sample length timeout and one or more attention lost timeouts for the first client, wherein the sample length timeout and the one or more attention lost timeouts are included in the register request, and wherein the sample length timeout is a maximum amount of time that is acceptable without attempting detection of user attention to the system, and wherein the attention lost timeout indicates a length of time after which the user'"'"'s attention is indicated as lost; and

record an indication of which events of the plurality of events are interpreted as indicative of attention by the first client.

3. The non-transitory computer accessible storage medium as recited in claim 2 wherein the instructions, when executed:

detect the sample length timeout has occurred for the first client;

sample a first one or more peripheral devices of the plurality of peripheral devices for one or more third events that are included in the plurality of events and that are different from the first event and the one or more second events; and

responsive to detecting the one or more third events, reset the sample length timeout and the one or more attention lost timeouts to initial values provided by the first client.

4. The non-transitory computer accessible storage medium as recited in claim 3 wherein the instructions, when executed responsive to not detecting the one or more third events, update the one or more attention lost timeouts to reflect passage of an amount of time reflected by the timeout.

5. The non-transitory computer accessible storage medium as recited in claim 2 wherein the instructions, when executed responsive to detecting the first event, reset the sample length timeout and the one or more attention lost timeouts to initial values provided by the first client.

6. The non-transitory computer accessible storage medium as recited in claim 2 wherein the instructions, when executed:

detect that a first attention lost timeout of the one or more attention lost timeouts has occurred for the first client; and

responsive to detecting the first attention lost timeout, communicate an indication of attention lost to the first client.

7. The non-transitory computer accessible storage medium as recited in claim 1 wherein the instructions, when executed:

receive a poll request from a second client of the one or more clients, wherein the poll request specifies an indication of current attention and a poll duration during which attention is to be reported to the second client, wherein the poll request registers a third event in the plurality of events with the attention detection service;

detect that the user is currently paying attention to the system responsive to the third event in the plurality of events; and

communicate the indication of the attention to the second client responsive to detecting that the user is currently paying attention.

8. The non-transitory computer accessible storage medium as recited in claim 7 wherein the instructions, when executed:

detect a fourth event of the plurality of events during the poll duration, wherein the fourth event is specified in the poll request as one of the plurality of events that the second client is registering with the attention detection service as an event that the second client interprets as indicative of attention; and

communicate the indication of attention to the second client responsive to detecting the fourth event and not having detected that the user was currently paying attention to the system at a time the poll request was received.

9. The non-transitory computer accessible storage medium as recited in claim 8 wherein the instructions, when executed:

detect a timeout without detecting the plurality of events during the poll duration; and

communicate an indication of attention lost to the second client responsive to detecting the timeout.

10. The non-transitory computer accessible storage medium as recited in claim 7 wherein the instructions, when executed:

detect that the poll duration has expired; and

discontinue communication of the indication of attention and an indication of attention lost to the second client responsive to detecting that the poll duration has expired.

11. The non-transitory computer accessible storage medium as recited in claim 7 wherein the poll request further includes an active polling interval, wherein the instructions, when executed, poll for the plurality of events during the active polling interval.

12. The non-transitory computer accessible storage medium as recited in claim 11 wherein the instructions, when executed:

detect that the user is paying attention responsive to the third event of the plurality of events during the active polling interval; and

reset the active polling interval responsive to detecting that the user is paying attention during the active polling interval.

13. A system comprising:

a plurality of peripheral devices; and

a processing system coupled to the plurality of peripheral devices, wherein the processing system comprises one or more processors coupled to a memory system configured to store a plurality of instructions forming an attention detection service, wherein the plurality of instructions, when executed by the one or more processors;

detect a first event of a plurality of events from the plurality of peripheral devices, the plurality of events potentially indicating that a user of the system is paying attention to the system; and

selectively report that the user is paying attention to the system to a first client of a plurality of clients executable on the system, wherein selection of the first client is based on the first event and a first indication from the first client to the attention detection service that the first event is to be indicated as the user paying attention to the system for the first client, wherein a second client of the plurality of clients is not selected to report that the user is paying attention based on the first event and a second indication from the second client to the attention detection service that the first event is not to be indicated as the user paying attention to the system for the second client.

14. The system as recited in claim 13 wherein the first indication is provided by the first client in a request previously transmitted by the first client to the attention detection service.

15. The system as recited in claim 14 wherein the request further includes a timeout period over which the first client is to be informed of the first event.

16. The system as recited in claim 14 wherein the request includes one or more timeout periods during which, if the user is not detected as paying attention to the system, the first client is to be informed that the user is not attentive to the system.

17. A non-transitory computer accessible storage medium storing instructions which, when executed on a system:

receive a first request from a first client identifying a first plurality of events that potentially indicate that a user is paying attention to the system;

receive a second request from a second client identifying a second plurality of events that potentially indicate that the user is paying attention to the system, wherein the second plurality of events excludes a first event;

monitor a plurality of peripheral devices in the system for the first plurality of events and the second plurality of events; and

responsive to detecting the first event of the first plurality of events and further responsive to the first request, communicate a first indication of the first event to the first client, wherein the first indication is not communicated to the second client responsive to the first event being excluded from the second plurality of events.

18. The non-transitory computer accessible storage medium as recited in claim 17 wherein the instructions, when executed responsive to receiving the first request from the first client:

record one or more first timeouts for the first client; and

responsive to detecting at least one of the one or more first timeouts and communicating the first indication, transmitting a second indication to the first client indicating that the user is no longer paying attention to the system.

19. The non-transitory computer accessible storage medium as recited in claim 17 wherein the instructions, when executed responsive to receiving the first request from the first client:

record a first timeout from the first request, wherein the first timeout specifies an interval over which the system is to sample for the first event;

sample for the first event during the interval; and

responsive to the first event not being detected during the interval, sampling one or more additional peripheral devices to determine if the user is paying attention.

20. The non-transitory computer accessible storage medium as recited in claim 19 wherein the one or more additional peripheral devices include a camera.