Power saving approaches to object detection

• US 9,860,517 B1
• Filed: 09/24/2013
• Issued: 01/02/2018
• Est. Priority Date: 09/24/2013
• Status: Expired due to Fees

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ramaswamy, Sharadh, Stafford, David Wayne
• Primary Examiner(s): Ren, Zhubing
• Application Number:
  US14/035,880
• Time in Patent Office: 1,561 Days
• CPC Class Codes: G06T 2207/10012 Stereo images
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/254 involving subtraction of images
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: H04N 13/243 using three or more 2D image sensors
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals
• CPC Class Codes: H04N 23/65 Control of camera operation in relation to power supply

Abstract

Various embodiments provide a method for determining conditions associated with the presence of a user in order to perform object detection. For example, various heuristic tests can be used to identify movement within a sequence of images or to identify user engagement therewith and, if movement or user engagement is identified, stereo disparity can be computed in an attempt to detect an object within a predetermined distance of the computing device. Accordingly, if the distance of the object is within the predetermined distance, object detection is performed. If the object is beyond the threshold distance, however, object detection is not performed in order to conserve battery life of the computing device.

Claims

1. A non-transitory computer-readable storage medium including instructions that, when executed by a processor of a computing device, cause the computing device to:

receive first image data that represents a set of images of an environment from at least one of a first camera or a second camera of the computing device;

determine, using the first image data, a first sum of pixel values that represent an estimate of a background image;

receive second image data that represents a current image captured by the first camera;

determine, using the second image data, a second sum of pixel values that represents the current image;

determine a difference between the first sum of pixel values and the second sum of pixel values;

identify a presence of an object in the current image when the difference between the first sum and the second sum meets or exceeds a threshold value;

receive third image data that represents the current image captured by the second camera;

determine a disparity between the object represented in the second image data and the object represented in the third image data;

determine a distance between the object and the computing device based on the disparity;

determine features of the object when the distance between the object and the computing device is within a threshold; and

analyze the features using a facial detection algorithm.

2. The non-transitory computer-readable storage medium of claim 1, wherein the instructions, when executed to determine the first sum of pixel values that represent the estimate of the background image, further cause the computing device to:

determine a first average of the pixel values that represent at least a portion of the estimate of the background image;

determine a second average of the pixel values that represent the current image; and

identify the presence of the object in the current image when the difference between the first average and the second average meets or exceeds the threshold value.

3. The non-transitory computer-readable storage medium of claim 1, wherein the disparity includes an amount of change between the object represented in the first image and the object represented in the second image.

4. A computer-implemented method, comprising:

under the control of one or more computer systems configured with executable instructions,

determining a first sum of pixel values that represent a background image from a set of images captured by at least one of a first camera or a second camera of a computing device;

determining a second sum of pixel values that represent a current image;

determining a difference between the first sum of pixel values and the second sum of pixel values;

identifying a presence of an object in the current image when the difference between the first sum and the second sum meets or exceeds a threshold value;

determining a first representation of the object in first image data captured by the first camera;

determining a second representation of the object in the second image data captured by the second camera;

determining a disparity between the first representation and the second representation;

determining, based at least in part on the disparity, a distance between the object and the computing device;

determining that the distance is less than a threshold distance;

determining features of the object; and

analyzing the features using an object detection algorithm.

5. The computer-implemented method of claim 4, further comprising:

comparing the features to stored feature information associated with a user;

determining a similarity value between the features and the stored feature information; and

determining the object is the user based on the similarity value meeting or exceeding a threshold similarity value.

6. The computer-implemented method of claim 4, wherein the object is at least one of a user, a head of the user, or a hand of the user.

7. The computer-implemented method of claim 4, further comprising:

detecting a user provided an input to the computing device; and

providing a signal, in response to the detected input, causing the first image data and the second image data to be captured.

8. The computer-implemented method of claim 4, further comprising:

determining movement of the object when the presence of the object is identified; and

providing a signal causing the first image data and the second image data to be captured.

9. The computer-implemented method of claim 4, further comprising:

determining the background image based on an average of the pixel values that represent the current image.

10. The computer-implemented method of claim 4, wherein the threshold value represents a noise pixel value.

11. The computer-implemented method of claim 4, further comprising:

detecting, using a motion sensor, motion of the computing device, the motion sensor including at least one of an accelerometer, an inertial sensor, an electronic compass, or an electronic gyroscope; and

providing a signal causing the first image data and the second image data to be captured based on the motion of the computing device.

12. The computer-implemented method of claim 4, further comprising:

capturing image information using at least one of the first camera or the second camera;

detecting, using a subsampled motion detection algorithm, movement in the image information; and

providing a signal causing the first image data and the second image data to be captured based on the movement in the image information.

13. The computer-implemented method of claim 4, wherein the object detection algorithm includes at least one of facial recognition algorithm or gesture recognition algorithm.

14. A computing device, comprising:

a processor;

a first camera;

a second camera;

a display screen; and

memory including instructions that, when executed by the processor, cause the computing device to;

determine a first sum of pixel values that represent a background image from a set of images captured by one of the first camera or the second camera;

determine a second sum of pixel values that represent a current image;

determine a difference between the first sum of pixel values and the second sum of pixel values;

identify a presence of an object in the current image when the difference between the first sum and the second sum meets or exceeds a threshold value;

determine a first representation of the object in first image data captured by the first camera;

determine a second representation of the object in second image data captured by the second camera;

determine a disparity between the first representation and the second representation;

determine, based at least in part on the disparity, a distance between the object and the computing device;

determine that the distance is less than a threshold distance;

determine features of the object; and

analyze the features using an object detection algorithm.

15. The computing device of claim 14, wherein the instructions, when executed by the processor to analyze the features, further cause the computing device to:

compare the features to stored feature information associated with a user;

determine a similarity value between the features and the stored feature; and

determine the object is the user based on the similarity value meeting or exceeding a threshold value.

16. The computing device of claim 14, wherein the object is at least one of a user, a head of the user, or a hand of the user.

17. The computing device of claim 14, wherein the instructions, when executed by the processor, further cause the computing device to:

determine movement of the object when the presence of the object is identified; and

provide a signal causing the first image data and the second image data to be captured.

18. The computing device of claim 17, wherein the difference between the first sum and the second sum meets or exceeds a noise pixel value.

19. The computing device of claim 14, wherein the instructions, when executed by the processor, further cause the computing device to:

detect an input provided by a user; and

provide a signal causing the first image data and the second image data to be captured based on the input.

20. The computing device of claim 14, wherein the instructions, when executed by the processor, further cause the computing device to:

detect, using a motion sensor, motion of the computing device; and

provide a signal causing the first image data and the second image data to be captured based on the motion of the computing device.