Method and apparatus for estimating body shape

• US 9,189,886 B2
• Filed: 08/14/2009
• Issued: 11/17/2015
• Est. Priority Date: 08/15/2008
• Status: Active Grant

• Current Assignee: Brown University
• Original Assignee: Brown University
• Inventors: Black, Michael J., Balan, Alexandru O., Weiss, Alexander W., Sigal, Leonid, Loper, Matthew M., St. Clair, Timothy S.
• Primary Examiner(s): Liew, Alex
• Application Number:
  US12/541,898
• Publication Number:
  US 20100111370A1
• Time in Patent Office: 2,286 Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appear in minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Claims

1. A method comprising:

obtaining data representing a body of an individual in a plurality of poses, wherein the data comprises one of image data of the body captured via a camera and partial depth information of the body captured via a range sensor; and

estimating the body of the individual by fitting a parametric body model of the body to the data to generate a set of pose parameters and a set of shape parameters, the set of shape parameters being consistent with the plurality of poses, and the parametric body model comprising a statistical polygonal mesh 3D model of triangles.

2. The method of claim 1, wherein the fitting of the parametric body model to the data comprises processing an objective function defined at least in part by the set of pose parameters and the set of shape parameters.

3. The method of claim 1, wherein the individual has at least one of an associated gender and an associated ethnicity, and wherein the fitting of the parametric body model to the data comprises processing an objective function defined at least in part by the set of pose parameters, the set of shape parameters, and a specified parameter corresponding to at least one of the gender and the ethnicity of the individual.

4. The method of claim 1, wherein the obtaining of the data comprises obtaining at least part of the data from an infrared sensor.

5. The method of claim 1, wherein the data represents a partially clothed body of the individual, wherein the estimating further comprising estimating a body shape of a portion of the partially clothed body that is covered by at least one piece of clothing of the partially clothed body based on the parametric body model and the data representing the partially clothed body.

6. The method of claim 5, wherein estimating the body shape further comprises detecting, via image classifiers, regions corresponding to at least one of skin, hair, and clothing.

7. The method of claim 6, wherein the fitting of the parametric body model of the at least partially clothed body utilizes an objective function that permits the estimation to be substantially within the second data.

8. The method of claim 1, wherein the parametric body model is a statistical parametric body model.

9. A system comprising:

a processor; and

a computer-readable storage medium storing instructions which, when executed by the processor, cause the processor to perform operations comprising;

obtaining data representing a body in a plurality of poses, wherein the data comprises one of image data of the body captured via a camera and partial depth information of the body captured via a range sensor; and

estimating the body of the individual by fitting a parametric body model of the body to data to generate a set of pose parameters and a set of shape parameters, the set of shape parameters being consistent with the plurality of poses, and the parametric body model comprising a statistical polygonal mesh 3D model of triangles.

10. The system of claim 9, wherein the fitting of the parametric body model to the data comprises processing an objective function defined at least in part by the set of pose parameters and the set of shape parameters.

11. The system of claim 9, wherein the individual has at least one of an associated gender and an associated ethnicity, and wherein the fitting of the parametric body model to the data comprises processing an objective function defined at least in part by the set of pose parameters, the set of shape parameters, and a specified parameter corresponding to at least one of the gender and the ethnicity of the individual.

12. The system of claim 9, wherein the obtaining of the data comprises obtaining at least part of the data from an infrared sensor.

13. The system of claim 9, wherein the data represents a partially clothed body of the individual, the method further comprising estimating a body shape of a portion of the partially clothed body that is covered by at least one piece of clothing of the partially clothed body based on the parametric body model and the data representing the partially clothed body.

14. The system of claim 13, wherein estimating the body shape further comprises detecting, via image classifiers, regions corresponding to at least one of skin, hair, and clothing.

15. The system of claim 14, wherein the fitting of the parametric body model of the at least partially clothed body utilizes an objective function that permits the estimation to be substantially within the second data.

16. The system of claim 9, wherein said parametric body model is a statistical parametric body model.

17. A computer-readable storage device storing information which, when executed by a processor, cause the processor to perform operations comprising:

obtaining data representing a body of an individual in a plurality of poses, wherein the data comprises one of image data of the body captured via a camera and partial depth information of the body captured via a range sensor; and

estimating the body of the individual by fitting a parametric body model of the body to the data to generate a set of pose parameters and a set of shape parameters, the set of shape parameters being consistent with the plurality of poses, and the parametric body model comprising a statistical polygonal mesh 3D model of triangles.

18. The computer-readable storage device of claim 17, wherein the fitting of the parametric body model to the data comprises processing an objective function defined at least in part by the set of pose parameters and the set of shape parameters.

19. The computer-readable storage device of claim 17, wherein the individual has at least one of an associated gender and an associated ethnicity, and wherein the fitting of the parametric body model to the data comprises processing an objective function defined at least in part by the set of pose parameters, the set of shape parameters, and a specified parameter corresponding to at least one of the gender and the ethnicity of the individual.

20. The computer-readable storage device of claim 17, wherein the obtaining of the data comprises obtaining at least part of the data from an infrared sensor.

21. The computer-readable storage device of claim 17, wherein the data represents a partially clothed body of the individual, the method further comprising estimating a body shape of a portion of the partially clothed body that is covered by at least one piece of clothing of the partially clothed body based on the parametric body model and the data representing the partially clothed body.

22. The computer-readable storage device of claim 21, wherein estimating the body shape further comprises detecting, via image classifiers, regions corresponding to at least one of skin, hair, and clothing.

23. The computer-readable storage device of claim 22, wherein the fitting of the parametric body model of the at least partially clothed body utilizes an objective function that permits the estimation to be substantially within the second data.

24. The computer-readable storage device of claim 17, wherein the parametric body model is a statistical parametric body model.