Object Detection in Depth Images

• US 20120269384A1
• Filed: 04/19/2011
• Published: 10/25/2012
• Est. Priority Date: 04/19/2011
• Status: Active Grant

• Current Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Original Assignee: Mitsubishi Electric Research Laboratories, Inc. (Mitsubishi Electric Corporation)
• Inventors: Jones, Michael J., Tuzel, Oncel, Si, Weiguang
• Granted Patent: US 8,406,470 B2
• Time in Patent Office: Days
• US Class Current: 382/103
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

A method for detecting an object in a depth image includes determining a detection window covering a region in the depth image, wherein a location of the detection window is based on a location of a candidate pixel in the depth image, wherein a size of the detection window is based on a depth value of the candidate pixel and a size of the object. A foreground region in the detection window is segmented based on the depth value of the candidate pixel and the size of the object. A feature vector is determined based on depth values of the pixels in the foreground region and the feature vector is classified to detect the object.

Claims

1. A method for detecting an object in a depth image, comprising:

determining a detection window covering a region in the depth image, wherein a location of the detection window is based on a location of a candidate pixel in the depth image, wherein a size of the detection window is based on a depth value of the candidate pixel and a size of the object;

segmenting a foreground region in the detection window based on the depth value of the candidate pixel and the size of the object;

classifying the foreground region as not including the object, if a number of pixels in the foreground region is less than a threshold; and

otherwise

resizing the foreground region based on a training size of a classifier;

determining a feature vector based on depth values of the pixels in the foreground region; and

classifying the feature vector to detect the object.

2. The method of claim 1, further comprising:

selecting a set of candidate pixels; and

repeating the determining the detection window, the segmenting, the resizing, determining the feature vector and the classifying for each candidate pixel in the set.

3. The method of claim 1, wherein the selecting comprises:

determining foreground pixels in the depth image; and

selecting the foreground pixels in the set of candidate pixels.

4. The method of claim 1, wherein the segmenting comprises:

determining, for each pixel in the detection window, a difference between a depth value of the pixel and the depth value of the candidate pixel; and

setting the depth value of the pixel to NULL, if the difference is greater than a depth of the object scaled based on a resolution of the depth image.

5. The method of claim 4, further comprising:

setting the depth value of the pixel to one, if the difference is less or equal to the depth of the object scaled based on the resolution of the depth image.

6. The method of claim 4, further comprising:

normalizing the depth value of the pixel, if the difference is less or equal to the depth of the object scaled based on the resolution of the depth image.

7. The method of claim 6, wherein the normalizing comprises:

subtracting the depth value of the candidate pixel from the depth value of the pixel.

8. The method of claim 1, wherein the determining the detection window comprises:

determining a length of the detection window based on a length of the object; and

determining a width of the detection window based on a width of the object.

9. The method of claim 1, wherein the detection window is centered on the location of the candidate pixel.

10. The method of claim 1, wherein the determining the detection window comprises:

selecting the size of the detection window from a lookup table using the depth value of the candidate pixel as a key.

11. The method of claim 10, further comprising:

populating the lookup table based on the size of the object, a range of depth values from a sensor, and a resolution of the sensor.

12. The method of claim 11, further comprising:

populating the lookup table based on a pose of the object.

13. The method of claim 1, further comprising:

determining the threshold based on the size of the object and a noise statistic of a depth sensor.

14. The method of claim 1, further comprising:

modifying the depth values of the pixels in the foreground region with depth values of corresponding pixels from a temporally adjacent depth image.

15. The method of claim 1, further comprising:

determining the size of the object based on a class of the object, wherein the size includes a length of the object, a width of the object, and a depth of the object.

16. The method of claim 15, further comprising:

training the classifier for the class of the object.

17. The method of claim 15, wherein the class of the object is selected from a group including at least one of people, and vehicles.

18. A method for detecting an object in a depth image, comprising:

selecting, for a candidate pixel, a size of a detection window as a function of a depth value of the candidate pixel;

arranging the detection window in the depth image around a location of the candidate pixel;

setting a depth value of a pixel in the detection window to NULL, if a difference between a depth value of the pixel and the depth value of the candidate pixel is greater than a depth threshold, wherein a value of the depth threshold is a function of the depth value of the candidate pixel and a depth of the object;

classifying the detection window as not including the object, if a number of pixels in the detection window having a non-NULL value is less than a threshold; and

otherwise

subtracting the depth value of the candidate pixel from non-NULL depth values of pixels in the detection window;

resizing the detection window based on a training size of a classifier;

determining a feature vector based on depth values of the pixels in the detection window; and

classifying the feature vector to detect the object.

19. A system for detecting an object in a depth image, comprising:

a depth sensor for acquiring the depth image;

a memory storing a lookup table for retrieval a size of a detection window based on a depth value;

a classifier for detecting the object in an input image, wherein the input image has a training size; and

a processor for determining the input image to the classifier and for executing the classifier to detect the object in the input image, such that during an operation of the system, the input image includes a foreground region segmented within the detection window arranged aground a candidate pixel, wherein the size of the detection window is selected from the lookup table using a depth value of the candidate pixel.

20. The system of claim 19, wherein the processor is further configured to normalize the depth values of pixels in the foreground region and to resize the foreground region based on the training size.