Determining time-of-fight measurement parameters

• US 9,558,563 B1
• Filed: 09/25/2013
• Issued: 01/31/2017
• Est. Priority Date: 09/25/2013
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Tikare, Amit, Kamarshi, Vijay, Degges, Ronald Joseph Jr., Wang, Eric, Coley, Christopher David
• Primary Examiner(s): Elahi, Shan
• Application Number:
  US14/036,523
• Time in Patent Office: 1,224 Days
• Field of Search: 348/169
• US Class Current: 1/1
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments

Abstract

In a system that monitors the positions and movements of objects within an environment, a depth camera may be configured to produce depth images based on configurable measurement parameters such as illumination intensity and sensing duration. A supervisory component may be configured to roughly identify objects within an environment and to specify observation goals with respect to the objects. The measurement parameters of the depth camera may then be configured in accordance with the goals, and subsequent analyses of the environment may be based on depth images obtained using the measurement parameters.

Claims

1. A system comprising:

one or more processors;

one or more cameras configured to produce images of an environment, wherein an individual image comprises pixel values corresponding respectively to surface points of the environment;

the one or more cameras including a time-of-flight camera, wherein the images include depth images, and wherein a depth image is produced by the time-of-flight camera using a sensing duration within which a reflected light signal is sensed, an individual pixel value of the depth image indicating a distance of a corresponding surface point from the time-of-flight camera;

one or more non-transitory computer-readable media storing computer-executable instructions that, when executed by the one or more processors, cause the one or more processors to perform acts comprising;

analyzing one or more of the images to detect an object within the environment;

defining a region of the environment that contains the object;

determining a distance of the object from the time-of-flight camera;

selecting a new sensing duration of the time-of-flight camera based at least in part on the sensing duration and the distance of the object from the time-of-flight camera, wherein the selecting the new sensing duration comprises at least one of increasing the sensing duration to increase spatial depth accuracy or decreasing the sensing duration to reduce motion blur;

obtaining a sequence of the depth images produced by the time-of-flight camera using the new sensing duration; and

analyzing a portion of an individual depth image of the sequence that corresponds to the defined region of the environment to track the object over time.

2. The system of claim 1, wherein analyzing one or more of the images to detect an object within the environment comprises one or more of:

model comparison;

template matching;

shape recognition;

color comparison;

or

feature detection.

3. The system of claim 1, wherein analyzing a portion of the individual depth image of the sequence that corresponds to the defined region of the environment to track the object over time comprises determining a sequence of locations of the object based on one or more of:

model comparison;

template matching;

shape recognition;

color comparison;

or

feature detection.

4. The system of claim 1, wherein the time-of-flight camera determines phase differences between a transmitted light signal and the reflected light signal and converts the phase differences based at least on part on a known propagation speed of light to obtain the distances of the surface points from the time-of-flight camera.

5. The system of claim 1, wherein selecting the new sensing duration based at least in part on the sensing duration and the distance of the object to the time-of-flight camera comprises selecting the sensing duration based at least in part on a time needed for a light signal to propagate from the time-of-flight camera, to a surface of the object, and back to the time-of-flight camera.

6. A computer-implemented method implemented at least in part by a computing device coupled to a time-of-flight camera, the computer-implemented method comprising:

receiving one or more images of an environment;

analyzing at least one of the one or more images to identify an object within the environment;

changing a sensing duration of the time-of-flight camera based at least in part on a distance to the object from the time-of-flight camera, the changing of the sensing duration comprising at least one of increasing the sensing duration to increase spatial depth accuracy or decreasing the sensing duration to reduce motion blur;

configuring the time-of-flight camera to produce a depth image using the sensing duration as changed; and

analyzing the depth image to determine one or more characteristics of the object.

7. The computer-implemented method of claim 6, further comprising identifying a portion of the depth image corresponding to a region of the environment containing the object, wherein analyzing the depth image comprises analyzing the portion of the depth image.

8. The computer-implemented method of claim 6, the acts further comprising:

determining a time for a light signal to travel from the time-of-flight camera, to the object, and back to the time-of-flight camera; and

wherein the sensing duration is changed to be at least as long as the time.

9. The computer-implemented method of claim 6, wherein at least one of the one or more received images indicates distances of surface points within the environment relative to the time-of-flight camera.

10. The computer-implemented method of claim 6, wherein analyzing the depth image comprises one or more of:

model comparison;

template matching;

shape recognition;

color comparison;

or

feature detection.

11. The computer-implemented method of claim 6, wherein the time-of-flight camera determines phase differences between a transmitted light signal and a reflected light signal and converts the phase differences to distances based at least on part on a known propagation speed of light to produce the depth image.

12. A method implemented at least in part by a computing device coupled to a time-of-flight camera, the method comprising:

receiving one or more images of an environment;

analyzing at least one of the one or more images to detect an object within the environment;

determining one or more observation goals for the object, the one or more observation goals comprising at least one of (a) increasing spatial depth resolution or (b) increasing temporal motion resolution to reduce motion blur;

selecting one or more measurement parameters for the time-of-flight camera based at least in part on the one or more observation goals, wherein the one or more measurement parameters comprise at least an illumination intensity;

configuring the time-of-flight camera to capture a depth image using the one or more measurement parameters; and

analyzing the depth image to determine one or more characteristic of the object.

13. The method of claim 12, further comprising identifying a portion of the depth image corresponding to a region of the environment containing the object, wherein analyzing the depth image comprises analyzing the portion of the depth image.

14. The method of claim 12, wherein:

the one or more measurement parameters further comprise a sensing duration; and

the selecting comprises selecting a relatively longer sensing duration when the observation goal is increasing spatial depth resolution than when the observation goal is increasing temporal motion resolution.

15. The method of claim 12, wherein:

the selecting comprises selecting a relatively higher illumination intensity when the observation goal is increasing spatial depth resolution than when the observation goal is increasing temporal motion resolution.

16. The method of claim 12, wherein the one or more measurement parameters further comprise a sensing duration, and wherein the method further comprises:

determining a time for a light signal to travel from the time-of-flight camera, to the object, and back to the time-of-flight camera; and

wherein the sensing duration is selected to be at least as long as the time.

17. The method of claim 12, wherein at least one of the one or more images indicates relative distances of surface points within the environment.

18. The method of claim 12, wherein analyzing the depth image comprises one or more of:

model comparison;

template matching;

shape recognition;

color comparison;

or

feature detection.

19. The method of claim 12, wherein the time-of-flight camera determines phase differences between a transmitted light signal and a reflected light signal and converts the phase differences to distances based at least on part on a known propagation speed of light to produce the depth image.

20. The method of claim 12, further comprising selecting the one or more parameters for the time-of-flight camera based on a distance to the object from the time-of-flight camera.