Presence sensing

• US 10,402,624 B2
• Filed: 02/28/2017
• Issued: 09/03/2019
• Est. Priority Date: 05/12/2011
• Status: Active Grant

• Current Assignee: Apple Inc.
• Original Assignee: Apple Inc.
• Inventors: Valko, Edward Allen, Waldon, Matthew C., Van der Merwe, Rudolph, Vieta, William Matthew, Haggerty, Myra M., Nelson, Alex T.
• Primary Examiner(s): Faragalla, Michael A
• Assistant Examiner(s): Bibbee, Chayce R
• Application Number:
  US15/445,623
• Publication Number:
  US 20170193282A1
• Time in Patent Office: 917 Days
• CPC Class Codes: G06T 2207/20076 Probabilistic image processing
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 40/166 using acquisition arrangements
• CPC Class Codes: G09G 2330/022 in absence of operation, e.g. no data being entered during a predetermined time
• CPC Class Codes: G09G 2354/00 Aspects of interface with display user
• CPC Class Codes: G09G 5/00 Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators image data processing or generation, in general G06T

Abstract

One embodiment may take the form of a method of operating a computing device to provide presence based functionality. The method may include operating the computing device in a reduced power state and collecting a first set of data from a first sensor. Based on the first set of data, the computing device determines if an object is within a threshold distance of the computing device and, if the object is within the threshold distance, the device activates a secondary sensor to collect a second set of data. Based on the second set of data, the device determines if the object is a person. If the object is a person, the device determines a position of the person relative to the computing device and executes a change of state in the computing device based on the position of the person relative to the computing device. If the object is not a person, the computing device remains in a reduced power state.

Claims

1. A method of operating a computing device to provide presence based functionality, comprising:

receiving a current image at the computing device from an image capture device associated with the computing device;

determining at least one light level of the received current image;

identifying multiple windows within the received current image;

computing statistics of each of the windows, wherein computing statistics for at least one of the windows uses the at least one determined light level;

performing a motion detection routine that includes;

comparing the statistics of the windows within the received current image to respective statistics of corresponding multiple windows within a previously received image;

computing a respective excursion value for each window within the received current image based on the comparison;

determining a first probability of user presence based on the excursion values;

making a binary determination of user presence; and

deciding if a user is present based on at least one of the first probability of user presence and the binary determination of user presence.

2. The method of claim 1, wherein the multiple windows within the received current image and the previously received image comprise multiple non-overlapping windows.

3. The method of claim 1, wherein at least one of the computed statistics is based on at least one of a luminance and an intensity gradient.

4. The method of claim 1, wherein the received current image and the previously received image are frames from a video sequence.

5. The method of claim 4, further comprising:

comparing a first frame of the video sequence with a second frame that is delayed by an integer value and with a third frame that is delayed by twice the integer value.

6. The method of claim 1, further comprising determining a distance to the user in the case that it is decided that the user is present.

7. The method of claim 1, wherein making the binary determination of user presence comprises:

summing the number of excursion values that exceed a first threshold; and

comparing the sum to a second threshold.

8. A method of operating a computing device to provide presence based functionality, the method comprising:

receiving a current image at the computing device from an image capture device associated with the new computing device;

determining at least one light level of the received current image;

identifying multiple concentric overlapping windows within the received current image;

computing statistics of each of the windows;

performing a motion detection routine that includes;

comparing the statistics of the windows within the received current image to respective statistics of corresponding multiple concentric overlapping windows within a previously received image; and

determining a first probability of user presence using the comparison of the statistics; and

deciding if a user is present using the first probability of user presence;

wherein computing statistics of at least one of the windows within the received current image uses the at least one determined light level.

9. The method of claim 8, wherein the computed statistics include at least one of a luminance and an intensity gradient.

10. The method of claim 8, further comprising performing a face detection routine, wherein the face detection routine comprises:

applying a face detector to the received current image, the face detector providing a face presence score indicating a probability that a face is present; and

scaling and averaging the face presence score of the received current image and a face presence score of the previously received image to determine a second probability of user presence;

wherein deciding if the user is present uses a combination of the first probability of user presence and the second probability of user presence.

11. The method of claim of claim 10, further comprising:

comparing the first probability of user presence to a first threshold;

comparing the second probability of user presence to a second threshold;

deciding that the user is present when the first probability of user presence exceeds the first threshold and the second probability of user presence exceeds the second threshold.

12. An electronic device comprising:

a camera-based sensor;

a second sensor; and

at least one processor, wherein the processor is configured to apply a sequence of operations including;

receiving a current image from the camera-based sensor;

determining at least one light level of the received current image;

identifying multiple windows within the received current image;

computing statistics of each of the windows, wherein computing the statistics for at least one of the windows uses the at least one determined light level;

performing a motion detection routine that includes;

comparing the statistics of the windows within the received current image to respective statistics of corresponding multiple windows within a previously received image;

computing a respective excursion value for each window within the received current image using the comparison;

determining a first probability of user presence based on the excursion values; and

making a binary determination of user presence; and

deciding if a user is present based on at least one of the first probability of user presence and the binary determination of user presence.

13. The electronic device of claim 12, wherein:

the camera-based sensor includes a main camera; and

the camera-based sensor is linked to an image processor that receives the current image and the previously received image from the camera-based sensor.

14. The electronic device of claim 12, wherein the second sensor is a proximity sensor.

15. The electronic device of claim 12, wherein the multiple windows within the received current image and the previously received image comprise multiple non-overlapping windows.

16. The electronic device of claim 12, wherein the multiple windows within the received current image and the previously received image comprise multiple concentric overlapping windows.

17. The electronic device claim 12, wherein the received current image and the previously received image are frames from a video sequence;

and the sequence of operations includes

performing a face detection routine that includes;

applying a face detector to the frames of the video sequence, the face detector providing a face presence score for each frame; and

scaling and averaging the face presence score of the received current image and a face presence score of the previously received image to determine a second probability of user presence;

wherein deciding if a user is present uses a combination of the first probability of user presence and the second probability of user presence.

18. The electronic device claim 12, wherein the sequence of operations comprises:

putting the electronic device into a first operational state when it is decided that the user is present; and

putting the electronic device into a second operational state when it is decided that the user is not present.

19. A method of operating an electronic device to determine if a user is present, the method comprising:

receiving in a processor of the electronic device a first image at the electronic device from a camera-based sensor associated with the electronic device;

receiving in the processor of the electronic device initial statistics calculated from multiple windows that are part of a second image received from the camera-based sensor prior to the reception of the first image;

calculating updated statistics from the multiple windows that are part of the first image;

comparing the initial statistics to the updated statistics;

determining respective excursion values for the multiple windows using on the comparison;

determining a probability of user presence using the excursion values;

making binary comparisons of the excursion values to a threshold; and

deciding that the user is present based on at least one of the probability of user presence and a sum of the binary comparisons.

20. The method of claim 19, wherein the initial statistics and the updated statistics are based on at least one of luminance data and intensity gradient data.