Three dimensional object pose estimation which employs dense depth information

• US 7,158,656 B2
• Filed: 08/01/2005
• Issued: 01/02/2007
• Est. Priority Date: 03/08/1999
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Vulcan Patents LLC
• Inventors: Covell, Michele M., Lin, Michael Hongmai, Rahimi, Ali, Harville, Michael, Darrell, Trevor J., Woodfill, John I., Baker, Harlyn, Gordon, Gaile G.
• Primary Examiner(s): Mehta, Bhavesh M.
• Assistant Examiner(s): LAVIN, CHRISTOPHER L
• Application Number:
  US11/195,039
• Publication Number:
  US 20050265583A1
• Time in Patent Office: 519 Days
• Field of Search: None
• US Class Current: 382/106
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

Dense range data obtained at real-time rates is employed to estimate the pose of an articulated figure. In one approach, the range data is used in combination with a model of connected patches. Each patch is the planar convex hull of two circles, and a recursive procedure is carried out to determine an estimate of pose which most closely correlates to the range data. In another aspect of the invention, the dense range data is used in conjunction with image intensity information to improve pose tracking performance. The range information is used to determine the shape of an object, rather than assume a generic model or estimate structure from motion. In this aspect of the invention, a depth constraint equation, which is a counterpart to the classic brightness change constraint equation, is employed. Both constraints are used to jointly solve for motion estimates.

Claims

1. A method for estimating the pose of an articulated figure, comprising the steps of:

obtaining dense range data which describes the distance of points on the figure from a reference;

shifting a focus of expansion of a point on the figure independently by an integer value; and

processing said dense range data to estimate the pose of the figure;

wherein said dense range data is compared with an estimate of pose to produce an error value and said estimate is iteratively revised to minimize said error and wherein the estimate of pose is generated with reference to a model of the figure.

2. The method of claim 1 wherein the dense range data is processed in accordance with a set of depth constraints to estimate the pose.

3. The method of claim 2 wherein said depth constraints are linear.

4. The method of claim 2 wherein said depth constraints are represented by means of twist mathematics.

5. The method of claim 1 wherein said model comprises a set of planar patches which respectively correspond to segments of the articulated figure.

6. The method of claim 5 wherein each patch comprises the convex hull of two circles.

7. The method of claim 1, wherein shifting the focus of expansion improves an accuracy of a constraint.

8. A method for estimating the pose of an object, comprising the steps of:

obtaining dense range data which describes the distance of points on the object from a reference;

shifting a focus of expansion of a point on the object independently by an integer value; and

processing said dense range data in accordance with a set of linear depth constraints to estimate the pose of the object;

wherein said dense range data is compared with an estimate of pose to produce an error value and said estimate is iteratively revised to minimize said error and wherein the estimate of pose is generated with reference to a model of the object.

9. The method of claim 8 wherein the object is articulated.

10. The method of claim 9 wherein said depth constraints are represented by means of twist mathematics.

11. The method of claim 10 further including the steps of mapping parameters which describe rotation and translation of the object to a set of linear parameters, solving for the depth constraints, and re-mapping back to the original parameters to provide a pose estimate.

12. The method of claim 9, wherein said reference comprises a location on the object, and the pose is estimated, at least in part, from the positions of points on the object relative to said location.

13. The method of claim 9, wherein an estimate of the pose of the object includes an estimate for each of the orientation and translational positions of the object, and further including the steps of decoupling the estimate of orientation from the estimate of translational position.

14. The method of claim 8, wherein the pose of the object is estimated for each image in a sequence of images, and further including the step of selecting a rigid translation value for each point on the object from one image to the next.

15. The method of claim 14, wherein said rigid translation value is an integer value.

16. The method of claim 14, wherein the rigid translation values are different for different points on the object.

17. A method for estimating the pose of an articulated object appearing in a sequence of video images, comprising the steps of:

establishing a model for the surfaces of the articulated object;

obtaining dense range data for pixels in each of said video images;

shifting a focus of expansion of a point on the object independently by an integer value;

generating a hypothetical pose for the object and determining the correlation of the hypothetical pose to the range data for an image; and

recursively generating successive hypothetical poses and determining the correlation of each hypothetical pose to identify the pose having the closest correlation to the range data.

18. The method of claim 17, wherein the estimate of pose is generated with reference to a model of the object.

19. The method of claim 18, wherein said model comprises a set of planar patches which respectively correspond to segments of the articulated object.

20. The method of claim 18, wherein the correlation is determined with respect to pixels in an image that are located within a predetermined distance of edges of said planar patches.