Multiple camera based motion tracking

• US 9,672,627 B1
• Filed: 05/09/2013
• Issued: 06/06/2017
• Est. Priority Date: 05/09/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Ramaswamy, Sharadh
• Primary Examiner(s): Park, Chan
• Assistant Examiner(s): Thomas, Mia M
• Application Number:
  US13/891,036
• Time in Patent Office: 1,489 Days
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06T 7/292 Multi-camera tracking

Abstract

A computing device with multiple image capture elements can selectively activate those elements in order to keep an object of interest within the field of view of at least one of those elements, while conserving resources by not keeping all the image capture elements active. The object can be located using an appropriate object recognition process. The location, and thus the motion, of the object then can be tracked over time using one or the image capture elements. The motion can be monitored to determine when the object is likely to pass into the field of view of another image capture element. The determination can be made based upon factors such as location, direction, speed, acceleration, and predicted time within the current field of view. Such an approach allows an identified object to be tracked and kept in a field of view while conserving resources on the device.

Claims

1. A non-transitory computer-readable storage medium storing instructions that, when executed by at least one processor of a computing device, cause the computing device to:

capture a first image;

analyze the first image to determine a position of an object relative to a plurality of gesture sensors of the computing device;

determine a first gesture sensor, of the plurality of gesture sensors of the computing device, having the object in a first field of view of the first gesture sensor;

cause the first gesture sensor to be in an activate state and other gesture sensors of the plurality of gesture sensors to be in a deactivated state;

capture one or more additional images of the object using the first gesture sensor while the object is in the first field of view of the first gesture sensor;

track motion of the object relative to the computing device by analyzing the one or more additional images captured using the first gesture sensor;

determine that the object is approaching a second field of view of a second gesture sensor of the plurality of gesture sensors; and

cause the second gesture sensor to be in the active state when the object, determined to be approaching the second field of view, satisfies at least one activation criterion for the second gesture sensor, the activation criterion including at least determining the object will enter the second field of view within a predetermined period of time associated with the second gesture sensor, wherein the computing device is configured to track motion of the object between the first field of view and the second field of view.

2. The non-transitory computer-readable storage medium of claim 1, wherein the at least one activation criterion further includes at least one of the object passing into the second field of view, or passing within a determined distance of the second field of view.

3. The non-transitory computer-readable storage medium of claim 1, wherein the instructions when executed further cause the computing device to:

capture the first image using a camera separate from the plurality of gesture sensors on the computing device.

4. The non-transitory computer-readable storage medium of claim 1, wherein the instructions when executed further cause the computing device to:

cause the first gesture sensor to enter the deactivated state after the object is located in the second field of view of the second gesture sensor.

5. A computer-implemented method, comprising:

determining a position of an object relative to a plurality of image capture elements of a computing device;

causing a first image capture element, of the plurality of image capture elements on the computing device, to capture one or more images of the object to determine a new position of the object, relative to the first image capture element of the computing device, over a period of time, the new position of the object being within a first field of view of the first image capture element;

determining, based at least in part upon analyzing the one or more images captured by the first image capture element, that the object, with respect to the first image capture element of the computing device, is moving toward a second field of view of a second image capture element and will enter the second field of view within a predetermined period of time associated with the second image capture element; and

causing the second image capture element, of the plurality of image capture elements, to be in an active state to capture one or more subsequent images to enable determination of where the object is while the object is in the second field of view.

6. The computer-implemented method of claim 5, further comprising:

determining whether motion of the object meets at least one activation criterion before causing the second image capture element to enter the active state.

7. The computer-implemented method of claim 6, wherein the at least one activation criterion further includes at least one of passing into the second field of view, or passing within a determined distance of the second field of view.

8. The computer-implemented method of claim 7, further comprising:

analyzing at least one of a position, a velocity, an acceleration, or a direction of movement to predict, for a current path of motion of the object, a determined amount of time in which the object is likely to enter the second field of view.

9. The computer-implemented method of claim 5, further comprising:

causing the first image capture element to enter a low power state after the object is determined to be within the second of field of view and the second image capture element is in the active state.

10. The computer-implemented method of claim 5, further comprising:

activating at least a third image capture element when a predicted range of motion of the object includes at least a third field of view of the at least a third image capture element.

11. The computer-implemented method of claim 5, wherein a plurality of fields of view for the plurality of image capture elements at least partially overlaps with at least one other field of view of the plurality of fields of view, wherein the computing device has a larger effective field of view than any of the plurality of fields of view and there are no dead zones within the effective field of view.

12. The computer-implemented method of claim 5, wherein the computing device is configured to capture stereoscopic images to determine the relative position of the object in three dimensions, and wherein at least two of the image capture elements are in the active state at any time to track movement of the object in the three dimensions.

13. The computer-implemented method of claim 5, wherein the object is at least one of a hand, a head, a finger, or a facial feature of a user, or an object being held by the user.

14. A computing device, comprising:

at least one processor;

a plurality of cameras; and

memory including instructions that, when executed by the at least one processor, cause the computing device to;

cause a first camera, of the plurality of cameras, to capture one or more images of a feature to determine motion of the feature while the feature is within a first field of view of the first camera;

determine, based at least in part upon analyzing the one or more images, that the feature is moving toward a second field of view of a second camera of the plurality of cameras;

cause the second camera to capture one or more subsequent images to enable determination of motion of the feature while the feature is in the second field of view;

determine an amount of time needed for the feature to move into the second field of view based at least in part upon a rate of motion of the feature with respect to the computing device;

compare the amount of time to a latency period of the second camera; and

instruct the second camera to enter an active state when the amount of time needed for the feature to move into the second field of view at least meets the latency period.

15. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

capture an initial image using at least one camera of the plurality of cameras; and

analyze the initial image to recognize the feature and determine an initial relative position of the feature with respect to the computing device, the initial relative position determining the first field of view.

16. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

cause the first camera to enter a low power state when changes in the relative position of the feature are being determined using the second camera.

17. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

cause the second camera to enter a standby state if the feature does not enter the second field of view for at least a determined amount of time.

18. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

update a display of information for the feature based at least in part upon a current relative position of the feature with respect to the computing device.

19. The computing device of claim 14, wherein the instructions when executed further cause the computing device to:

track relative motion of the feature with respect to the computing device using a fast correlation tracking algorithm.

20. The computing device of claim 19, wherein the instructions when executed further cause the computing device to:

model the relative motion using at least one of a Kalman filter or a linear quadratic estimation process.