Method and system for generating garment model data

• US 10,380,794 B2
• Filed: 12/14/2015
• Issued: 08/13/2019
• Est. Priority Date: 12/22/2014
• Status: Active Grant

• Current Assignee: Reactive Reality GmbH
• Original Assignee: Reactive Reality GmbH
• Inventors: Hauswiesner, Stefan, Grasmug, Philipp
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US15/535,942
• Publication Number:
  US 20170372515A1
• Time in Patent Office: 1,338 Days
• Field of Search: None
• CPC Class Codes: G06T 17/00 Three dimensional [3D] modelling, e.g. data description of 3D objects
• CPC Class Codes: G06T 17/10 Constructive solid geometry [CSG] using solid primitives, e.g. cylinders, cubes
• CPC Class Codes: G06T 2207/20116 Active contour; Active surface; Snakes
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/149 involving deformable models, e.g. active contour models
• CPC Class Codes: G06T 7/344 involving models

Abstract

In a process for generating garment model data representative of a piece of garment, input image data containing a view of the piece of garment are processed. A type of wearing condition is determined as at least one of a first type of worn garment and of a second type of not-worn garment. If the first type is determined, a shape of the piece of garment and a shape of the person wearing the garment are identified utilizing an active contour modelling approach based on a preset body model. The identified shapes are adapted based on a garment template model. The garment model data are determined from the input image data based on the adapted identified shapes. If the second type is determined, a shape of the piece of garment is identified. The input image data are iteratively compared with a respective garment template model to identify at least one matching garment template model. The identified shape is aligned with a shape of the at least one matching garment template model and the garment model data are determined from the input image data based on the identified shape and on results of the aligning.

Claims

1. A method for generating garment model data representative of a piece of garment, the method comprising

processing, using a processor, input image data retrieved from a storage location in a memory, the input image data containing a view of the piece of garment;

determining, using the processor, a type of wearing condition from the input image data as at least one of a first type, where the piece of garment is worn by a person, and of a second type, where the piece of garment is not worn;

if the first type is determined,

identifying, using the processor, a shape of the piece of garment and a shape of the person wearing the piece of garment in the input image data utilizing an active contour modelling approach based on a preset body model;

adapting, using the processor, the identified shape of the piece of garment and the identified shape of the person, of the piece of garment and the identified shape of the person, based on a garment template model retrieved from a garment template model database stored in the memory; and

determining, using the processor, the garment model data from the input image data based on the adapted identified shapes; and

if the second type is determined,

identifying, using the processor, a shape of the piece of garment in the input image data;

iteratively comparing the input image data with a respective garment template model retrieved from the garment template model database to identify at least one matching garment template model;

aligning, using the processor, the identified shape with a shape of the at least one matching garment template model; and

determining, using the processor, the garment model data from the input image data based on the identified shape and on results of the aligning.

2. The method according to claim 1, wherein each of the garment template models stored in the garment template model database contains at least one of the following:

a set of control points that define a relationship between a shape of a garment and a body wearing said garment;

a geometrical description that defines a relationship between a shape of a garment and a body wearing said garment;

metadata associated with a shape of a garment;

metadata associated with a type of a garment;

metadata associated with fitting information of a garment.

3. The method according to claim 1, wherein adapting the identified shapes comprises aligning control points and/or a geometrical description contained in the garment template model with corresponding points and/or a geometrical description of the shape of the piece of garment and of the shape of the person.

4. The method according to claim 1, wherein, if the first type is determined, determining the garment model data comprises separating background image data and body image data, which show parts of the person not being covered by the piece of garment, from the input image data to obtain garment image data, the separating being based on the adapted identified shapes and on color identification in the input image data.

5. The method according to claim 4, wherein, if the first type is determined, determining the garment model data further comprises identifying regions in the garment image data, which are occluded, and substituting such identified regions with matching image data, the identifying and/or the substituting being based on at least one garment template model retrieved from the garment template model database.

6. The method according to claim 1, wherein identifying the at least one matching garment template model comprises iteratively determining a similarity metric between the respective garment template model and the input image data and/or the identified shape, and selecting the matching garment template model based on the iteratively determined similarity metrics.

7. The method according to claim 1, wherein aligning the identified shape comprises aligning control points and/or a geometrical description contained in the at least one matching garment template model with corresponding points and/or a geometrical description of the shape of the piece of garment.

8. The method according to claim 1, wherein, if the second type is determined, determining the garment model data comprises obtaining garment image data associated with the piece of garment from the input image data based on the aligned identified shapes, and further comprises identifying at least one region in the garment image data that would not be visible, if the piece of garment was worn by a person, and excluding such identified region from the garment image data and the garment model data.

9. The method according to claim 8, wherein, if the second type is determined, identifying at least one region comprises determining image gradients in the garment image data, comparing the image gradients to a set of geometrical shapes to achieve a matching quantity for each of the geometrical shapes, selecting one of the geometrical shapes based on the matching quantities, and defining the at least one region based on the selected geometrical shape.

10. The method according to claim 1, wherein determining the garment model data further comprises, if the piece of garment includes a sleeve or a pant leg, identifying a first and a second point along the sleeve or the pant leg, respectively, determining image gradients in the input image data, comparing the image gradients to a set of interconnection paths between the first and the second point to achieve a matching quantity for each of the interconnection paths, and selecting one of the interconnection paths based on the matching quantities to be used as separation information in the garment model data.

11. The method according to claim 1, wherein the garment model data are generated such that they contain depth information and/or 3-dimensional information.

12. The method according to claim 1, wherein the determination of the type of wearing condition is performed based on at least one of the following:

face detection;

body detection;

body part detection;

detection of skin regions;

hair detection;

garment recognition;

a wearing condition indicated in metadata comprised by the input image data.

13. The method according to claim 1, wherein retrieving a garment template model from the garment template model database is based on at least one of the following:

a type of garment indicated in metadata comprised by the input image data;

a shooting condition indicated in metadata comprised by the input image data;

a shooting angle indicated in metadata comprised by the input image data.

16. The method according to claim 1, wherein the active contour modelling approach is initialized with a face detector and the preset body model, which in particular is a statistical body model.

17. The method according to claim 1, wherein the input image data originate from one of the following:

picked from a website through a browser;

picked from a website via a computer application;

picked from a website via a smartphone application;

a photograph taken by a user;

scans or screenshots of media like catalogues, magazines or screens.

14. A system for generating garment model data representative of a piece of garment, the system comprising a memory and a processor that is configured to:

process input image data retrieved from the memory, the input image data containing a view of the piece of garment;

determine a type of wearing condition from the input image data as at least one of a first type, where the piece of garment is worn by a person, and of a second type, where the piece of garment is not worn;

if the first type is determined,

identify a shape of the piece of garment and a shape of the person wearing the piece of garment in the input image data utilizing an active contour modelling approach based on a preset body model;

adapt the identified shapes of the piece of garment and the identified shape of the person, based on a garment template model retrieved from a garment template model database; and

determine the garment model data from the input image data based on the adapted identified shapes; and

if the second type is determined,

identify a shape of the piece of garment in the input image data;

iteratively compare the input image data with at least one respective garment template model retrieved from the garment template model database to identify a matching garment template model;

align the identified shape with a shape of the at least one matching garment template model; and

determine the garment model data from the input image data based on the identified shape and on results of the aligning.

18. The system according to claim 14, wherein the active contour modelling approach is initialized with a face detector and the preset body model, which in particular is a statistical body model.

15. A non-transitory computer program product comprising a program code that, upon execution by a processor, is configured to perform a method of generating garment model data representative of a piece of garment, comprising:

processing, using a processor, input image data retrieved from a storage location in a memory, the input image data containing a view of the piece of garment;

determining, using the processor, a type of wearing condition from the input image data as at least one of a first type, where the piece of garment is worn by a person, and of a second type, where the piece of garment is not worn;

if the first type is determined,

identifying, using the processor, a shape of the piece of garment and a shape of the person wearing the piece of garment in the input image data utilizing an active contour modelling approach based on a preset body model;

adapting, using the processor, the identified shape of the piece of garment and the identified shape of the person, of the piece of garment and the identified shape of the person, based on a garment template model retrieved from a garment template model database stored in the memory; and

determining, using the processor, the garment model data from the input image data based on the adapted identified shapes; and

if the second type is determined,

identifying, using the processor, a shape of the piece of garment in the input image data;

iteratively comparing the input image data with a respective garment template model retrieved from the garment template model database to identify at least one matching garment template model;

aligning, using the processor, the identified shape with a shape of the at least one matching garment template model; and

determining, using the processor, the garment model data from the input image data based on the identified shape and on results of the aligning.