Three dimensional object pose estimation which employs dense depth information

• US 7,003,134 B1
• Filed: 03/08/2000
• Issued: 02/21/2006
• Est. Priority Date: 03/08/1999
• Status: Expired due to Term

• Current Assignee: Intel Corporation
• Original Assignee: Vulcan Patents LLC
• Inventors: Covell, Michele M., Lin, Michael Hongmai, Rahimi, Ali, Harville, Michael, Darrell, Trevor J., Woodfill, John I., Baker, Harlyn, Gordon, Gaile G.
• Primary Examiner(s): Johns, Andrew W.
• Assistant Examiner(s): NAKHJAVAN, SHERVIN K
• Application Number:
  US09/520,964
• Time in Patent Office: 2,176 Days
• Field of Search: 382/106, 382/107, 382/103, 382/181, 700/62, 348/169, 348/135, 348/139, 348/140, 348/340, 356/3, 356/4.01, 356/4.07, 356/21, 356/22, 345/619, 345/622, 414/2
• US Class Current: 382/103
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/251 involving models
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/107 Static hand or arm

Abstract

Dense range data obtained at real-time rates is employed to estimate the pose of an articulated figure. In one approach, the range data is used in combination with a model of connected patches. Each patch is the planar convex hull of two circles, and a recursive procedure is carried out to determine an estimate of pose which most closely correlates to the range data. In another aspect of the invention, the dense range data is used in conjunction with image intensity information to improve pose tracking performance. The range information is used to determine the shape of an object, rather than assume a generic model or estimate structure from motion. In this aspect of the invention, a depth constraint equation, which is a counterpart to the classic brightness change constraint equation, is employed. Both constraints are used to jointly solve for motion estimates.

Claims

1. A method for estimating the pose of an articulated figure, comprising the steps of:

obtaining dense range data which describes the distance of points on the figure from a reference;

shifting a focus of expansion of a point on the figure independently by an integer value; and

processing said dense range data to estimate the pose of the figure.

2. The method of claim 1 wherein the dense range data is processed in accordance with a set of depth constraints to estimate the pose.

3. The method of claim 2 wherein said depth constraints are linear.

4. The method of claim 2 further including the steps of obtaining brightness data from an image of the figure, and processing said brightness data in accordance with a set of linear brightness constraints to estimate the pose.

5. The method of claim 2 wherein said depth constraints are represented by means of twist mathematics.

6. The method of claim 1 wherein said dense range data is compared with an estimate of pose to produce an error value, and said estimate is iteratively revised to minimize said error.

7. The method of claim 1 further including the steps of obtaining brightness data from an image of the figure, and processing said brightness data in accordance with a set of brightness constraints to estimate the pose.

8. The method of claim 1, wherein shifting the focus of expansion improves an accuracy of a constraint.

9. A method for estimating the pose of an object, comprising the steps of:

obtaining dense range data which describes the distance of points on the object from a reference;

shifting a focus of expansion of a point on the object independently by an integer value; and

processing said dense range data in accordance with a set of linear depth constraints to estimate the pose of the object.

10. The method of claim 9 wherein the object is articulated.

11. The method of claim 10 wherein said depth constraints are represented by means of twist mathematics.

12. The method of claim 11 further including the steps of mapping parameters which describe rotation and translation of the object to a set of linear parameters, solving for the depth constraints, and re-mapping back to the original parameters.

13. The method of claim 10, wherein said reference comprises a location on the object, and the pose is estimated, at least in part, from the positions of points on the object relative to said location.

14. The method of claim 9 further including the steps of obtaining brightness data from an image of the object, and processing said brightness data in accordance with a set of linear brightness constraints to estimate the pose.

15. The method of claim 9, wherein an estimate of the pose of the object includes an estimate for each of the orientation and translational positions of the object, and further including the steps of decoupling the estimate of orientation from the estimate of translational position.

16. The method of claim 9, wherein the pose of the object is estimated for each image in a sequence of images, and further including the step of selecting a rigid translation value for each point on the object from one image to the next.

17. The method of claim 16, wherein said rigid translation value is an integer value.

18. The method of claim 16, wherein the rigid translation values are different for different points on the object.

19. A method for estimating the pose of an object appearing in a sequence of video images, comprising the steps of:

obtaining dense brightness data for pixels in each of said video images;

obtaining dense range data for pixels in each of said video images;

determining an initial pose for the object in one of said video images; and

estimating changes in at least one of the translational position and rotational orientation of the object for successive images, on the basis of said brightness data and said range data, shifting a focus of expansion of a point on the object independently by an integer to thereby estimate the pose of the object in successive images.

20. The method of claim 19, wherein the object is articulated.

21. The method of claim 19, wherein said estimates are obtained by means of linear constraint equations that are applied to said brightness data and said range data.