BODY FEATURE DETECTION AND HUMAN POSE ESTIMATION USING INNER DISTANCE SHAPE CONTEXTS

• US 20100215271A1
• Filed: 02/19/2010
• Published: 08/26/2010
• Est. Priority Date: 02/25/2009
• Status: Active Grant

• Current Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Original Assignee: Honda Motor Co., Ltd. (Honda Motor Company)
• Inventors: Dariush, Behzad, Gopalan, Raghuraman
• Granted Patent: US 9,904,845 B2
• Time in Patent Office: Days
• US Class Current: 382/180
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A system, method, and computer program product for estimating human body pose are described. According to one aspect, a human figure silhouette is segmented from a depth image of a human actor. Contour points are sampled along the human figure silhouette. Inner Distance Shape Context (IDSC) descriptors of the sample contour points are determined and compared to IDSC descriptors of the feature points in an IDSC gallery for similarity. For each of the feature points, the sample contour point with the IDSC descriptor that is most similar to an IDSC of the feature point is identified as that feature point in the depth image. An estimated pose of a human model is estimated based on the detected feature points and kinematic constraints of the human model.

Claims

1. A computer based method for detecting a feature point of an object in an image of the object, the method comprising:

segmenting an image region of the object from an image region of background in the image;

sampling a plurality of points along a contour of the segmented image region of the object;

determining Inner Distance Shape Context (IDSC) descriptors for the plurality of points; and

detecting the feature point in the plurality of points based on their IDSC descriptors.

2. The method of claim 1, wherein detecting the feature point comprises:

matching the IDSC descriptors of the plurality of points with IDSC descriptors of the feature point in an IDSC gallery for similarity; and

identifying one of the plurality of points as the feature point of the object in the image responsive to the IDSC descriptor of said point is more similar to an IDSC descriptor of the feature point in the IDSC gallery than the IDSC descriptors of rest of the plurality of points.

3. The method of claim 2, wherein the IDSC gallery comprises IDSC descriptors determined for the feature point of the object in a plurality of images.

4. The method of claim 2, wherein identifying one of the plurality of points as the feature point of the object in the image comprises:

for each of the plurality of points, comparing a difference between its IDSC descriptor and the IDSC descriptor of the feature point in the IDSC gallery with a threshold value, and identifying the point as the feature point of the object in the image responsive to the threshold value exceeding the difference.

5. The method of claim 4, further comprising:

responsive to the threshold value exceeding differences associated with two or more of the plurality of points, selecting one of the two or more points as the feature point of the object in the image based on a previously generated corresponding predicted feature point.

6. The method of claim 1, wherein the plurality of points is sampled uniformly along the contour of the segmented image region of the object.

7. The method of claim 1, wherein segmenting the image region of the object comprises:

identifying image regions in the depth image with depth values exceeding a predetermined depth working volume as background; and

identifying image regions with vertical depth image normal vectors as background.

8. The method of claim 1, wherein the image comprises a depth image and the object comprises a human actor.

9. The method of claim 8, wherein the feature point comprises one of:

head top, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left waist, right waist, groin, left knee, right knee, left ankle, and right ankle.

10. The method of claim 8, further comprising:

augmenting a missing feature point based on a previously generated corresponding predicted feature point; and

estimating a pose of the human actor in a human model based on the augmented feature point and kinematic constraints of the human model.

11. The method of claim 10, wherein estimating the pose of the human actor in the human model further comprises:

tracking the estimated pose of the human model with an observed pose of the human actor.

12. The method of claim 10, further comprising:

generating a predict feature point based on the augmented feature point and the kinematic constraints of the human model.

13. The method of claim 10, wherein the kinematic constraints of the human model comprises joint limitations and self penetration avoidance.

14. The method of claim 13, further comprises:

constructing a virtual surface surrounding an actual surface of a body segment of the human model;

monitoring a distance between the body segment and an unconnected structure;

detecting that the unconnected structure penetrates the virtual surface;

determining a redirected joint motion that prevents the unconnected structure from colliding with the body segment; and

redirecting the body segment based on the redirected joint motion to avoid colliding with the unconnected structure.

15. The method of claim 8, further comprising:

performing a skeleton analysis on the image region of the human actor to generate a skeleton image of the human actor;

performing distance transformation on the skeleton image to generate a distance transformed skeleton image of the human actor; and

detecting the feature point of the human actor in the distance transformed skeleton image.

16. The method of claim 15, wherein detecting the feature point of the human actor in the distance transformed skeleton image further comprises:

determining whether self occlusion is present in the depth image based on the distance transformed skeleton image; and

responsive to self occlusion being determined present in the depth image, conducting additional analysis of the depth image to detect the feature point of the human actor.

17. The method of claim 8, wherein the depth image is taken by a single time-of-flight camera.

18. The method of claim 1, further comprising:

labeling the detected feature point in the image.

19. A computer program product for detecting a feature point of an object in an image of the object, the computer program product comprising a computer-readable storage medium containing executable computer program code for performing a method comprising:

segmenting an image region of the object from an image region of background in the image;

sampling a plurality of points along a contour of the segmented image region of the object;

determining Inner Distance Shape Context (IDSC) descriptors for the plurality of points; and

detecting the feature point in the plurality of points based on their IDSC descriptors.

20. A system for detecting a feature point of an object in an image of the object, the system comprising:

a computer processor for executing executable computer program code;

a computer-readable storage medium containing the executable computer program code for performing a method comprising;

segmenting an image region of the object from an image region of background in the image;

sampling a plurality of points along a contour of the segmented image region of the object;

determining Inner Distance Shape Context (IDSC) descriptors for the plurality of points; and

detecting the feature point in the plurality of points based on their IDSC descriptors.