Method and apparatus for estimating body shape

• US 10,546,417 B2
• Filed: 06/26/2019
• Issued: 01/28/2020
• Est. Priority Date: 08/15/2008
• Status: Active Grant

• Current Assignee: Brown University
• Original Assignee: Brown University
• Inventors: Black, Michael J., Balan, Alexandru O., Weiss, Alexander W., Sigal, Leonid, Loper, Matthew M., St. Clair, Timothy S.
• Primary Examiner(s): Liew, Alex Kok S
• Application Number:
  US16/453,135
• Publication Number:
  US 20190333267A1
• Time in Patent Office: 216 Days
• CPC Class Codes: G06F 18/2321 using statistics or function optimisation, e.g. modelling of probability density functions
• CPC Class Codes: G06Q 30/0601 Electronic shopping [e-shopping]
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/75 involving models
• CPC Class Codes: G06T 7/77 using statistical methods
• CPC Class Codes: G06V 10/763 Non-hierarchical techniques, e.g. based on statistics of modelling distributions
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition

Abstract

A system and method of estimating the body shape of an individual from input data such as images or range maps. The body may appear in one or more poses captured at different times and a consistent body shape is computed for all poses. The body may appearin minimal tight-fitting clothing or in normal clothing wherein the described method produces an estimate of the body shape under the clothing. Clothed or bare regions of the body are detected via image classification and the fitting method is adapted to treat each region differently. Body shapes are represented parametrically and are matched to other bodies based on shape similarity and other features. Standard measurements are extracted using parametric or non-parametric functions of body shape. The system components support many applications in body scanning, advertising, social networking, collaborative filtering and Internet clothing shopping.

Claims

1. A method comprising:

receiving, at a merchant site, a body model of a user who is navigating the merchant site, wherein the body model of the user is generated by;

obtaining data representing a body of the user in a first pose and a second pose, wherein the data comprises one of image data of the body captured via a camera and partial depth information of the body captured via a range sensor; and

estimating a consistent body shape of the user across the first pose and the second pose by fitting a parametric body model of the body to the data to generate a set of pose parameters and a set of shape parameters associated with the body model of the user, wherein the estimating of the consistent body shape of the user separately factors (1) changes in a shape of the body due to changes between the first pose and the second pose to yield the set of pose parameters from (2) changes in the body of the user due to identity to yield the set of shape parameters; and

presenting, on the merchant site, clothing on the body model of the user.

2. The method of claim 1, wherein the fitting of the parametric body model of the body to the data comprises executing an objective function defined at least in part by the set of pose parameters and the set of shape parameters.

3. The method of claim 2, wherein the objective function processes first data from the first pose and second data from the second pose and implements the consistent body shape across the first pose and the second pose.

4. The method of claim 1, wherein the user has at least one of an associated gender and an associated ethnicity, and wherein the fitting of the parametric body model of the body to the data comprises processing an objective function defined at least in part by the set of pose parameters, the set of shape parameters, and a specified parameter corresponding to at least one of the associated gender and the associated ethnicity of the user.

5. The method of claim 3, wherein the data represents an at least partially clothed body of the user, wherein the estimating further comprising estimating a body shape of a portion of the at least partially clothed body that is covered by at least one piece of clothing of the at least partially clothed body based on the parametric body model of the body and the data representing the at least partially clothed body.

6. The method of claim 5, wherein the estimating of the body shape of the portion of the at least partially clothed body further comprises detecting, via image classifiers, regions corresponding to at least one of skin, hair, and clothing.

7. The method of claim 6, wherein the fitting of the parametric body model of the body of the at least partially clothed body utilizes an objective function that permits the estimating to be substantially within the second data.

8. The method of claim 1, wherein the parametric body model of the body is a statistical parametric body model.

9. The method of claim 1, further comprising:

receiving a payment for the clothing and shipping the clothing to the user.

10. A system comprising:

a processor; and

a computer-readable storage medium storing instructions which, when executed by the processor, cause the processor to perform operations comprising;

receiving a body model of a user who is navigating a merchant site operated by the system, wherein the body model of the user is generated by;

obtaining data representing a body of the user in a first pose and a second pose, wherein the data comprises one of image data of the body captured via a camera and partial depth information of the body captured via a range sensor; and

estimating a consistent body shape of the user across the first pose and the second pose by fitting a parametric body model of the body to the data to generate a set of pose parameters and a set of shape parameters associated with the body model of the user, wherein the estimating separately factors (1) changes in a shape of the body due to changes between the first pose and the second pose to yield the set of pose parameters from (2) changes in the body of the user due to identity to yield the set of shape parameters; and

presenting clothing on the body model of the user.

11. The system of claim 10, wherein the fitting of the parametric body model of the body to the data comprises executing an objective function defined at least in part by the set of pose parameters and the set of shape parameters.

12. The system of claim 11, wherein the objective function processes first data from the first pose and second data from the second pose and implements the consistent body shape across the first pose and the second pose.

13. The system of claim 12, wherein the user has at least one of an associated gender and an associated ethnicity, and wherein the fitting of the parametric body model of the body to the data comprises processing an objective function defined at least in part by the set of pose parameters, the set of shape parameters, and a specified parameter corresponding to at least one of the associated gender and the associated ethnicity of the user.

14. The system of claim 13, wherein the data represents an at least partially clothed body of the user, wherein the estimating further comprising estimating a body shape of a portion of the at least partially clothed body that is covered by at least one piece of clothing of the at least partially clothed body based on the parametric body model of the body and the data representing the at least partially clothed body.

15. The system of claim 14, wherein the estimating of the consistent body shape further comprises detecting, via image classifiers, regions corresponding to at least one of skin, hair, and clothing.

16. The system of claim 15, wherein the fitting of the parametric body model of the body of the at least partially clothed body utilizes an objective function that permits the estimating to be substantially within the second data.

17. The system of claim 10, wherein the parametric body model of the body is a statistical parametric body model.

18. The system of claim 10, wherein the computer-readable storage medium stores additional instructions which, when executed by the processor, cause the processor to perform operations further comprising:

receiving a payment for the clothing and shipping the clothing to the user.