THREE-DIMENSIONAL MODEL GENERATION BASED ON TWO-DIMENSIONAL IMAGES

• US 20160379419A1
• Filed: 11/16/2015
• Published: 12/29/2016
• Est. Priority Date: 06/26/2015
• Status: Active Grant

• Current Assignee: Virtual Outfits, LLC
• Original Assignee: Virtual Outfits, LLC
• Inventors: Khalili, Matthew, Khalili, Farzaneh
• Granted Patent: US 9,870,646 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06T 19/20 Editing of 3D images, e.g. changing shapes or colours, aligning objects or positioning parts
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 2219/2021 Shape modification

Abstract

A 3D model generation engine is programmed to generate a 3D model of a piece of clothing based on a number of 2D images of the piece of clothing. Specifically, the 3D model is generated by deforming a 3D model template using control points that are automatically pinned to different locations of a representation of the piece of clothing on the 2D image(s). The 3D model template includes multiple bisectional planes. Each bisectional plane has multiple vertices and 3D control points. At least one 2D image of the piece of clothing is obtained, and different locations on the 2D image are determined for several 2D control points. The 3D model template is modified by deforming the bisectional planes (and its associated vertices) of the 3D model template based on the determined location of the 2D control points on the 2D image.

Claims

1. A computer-implemented method of generating a three-dimensional (3D) model of an object based on a two-dimensional (2D) image of the object, comprising:

obtaining a 3D model having a plurality of bisectional planes, wherein each bisectional plane comprises a group of 3D control points along the edge of the bisectional plane;

determining locations for a plurality of 2D control points on a representation of the object appeared in the 2D image, wherein each 2D control points in the plurality of 2D control points corresponds to a distinct 3D control point; and

modifying the 3D model by adjusting each bisectional plane based on the determined locations of the plurality of 2D control points on the 2D image.

2. The computer-implemented method of claim 1, wherein determining the locations for the plurality of 2D control points comprising:

obtaining a 2D contour template having an outline of the object and 2D control points located along the outline;

overlaying the 2D image over the 2D contour template; and

adjusting the locations of the 2D control points on the 2D contour template by analyzing a difference between the outline of the object in the 2D contour template and the representation of the object in the 2D image.

3. The computer-implemented method of claim 2, wherein analyzing the difference comprises performing a pixel-level analysis.

4. The computer-implemented method of claim 2, wherein analyzing the difference comprises matching a pixel on the 2D image with a 2D control points on the 2D contour template.

5. The computer-implemented method of claim 2, wherein analyzing the difference comprises determining a first non-alpha pixel from one side of the 2D image to the other side of the 2D image.

6. The computer-implemented method of claim 1, wherein at least two of the bisectional planes are parallel to each other.

7. The computer-implemented method of claim 1, wherein the 2D image comprises a representation of a front view of the object.

8. The computer-implemented method of claim 7, further comprising:

obtaining a second 2D image comprising a representation of a back view of the object;

determining locations for a second plurality of 2D control points on the representation of the back view of the object appeared on the second 2D image;

modifying the 3D model by adjusting each bisectional plane based on the determined locations of the second plurality of 2D control points on the second 2D image.

9. The computer-implemented method of claim 1, wherein the object is a piece of clothing.

10. The computer-implemented method of claim 1, wherein obtaining the 3D model comprises selecting a 3D model template from a group of 3D model templates comprising at least a 3D pants template, a 3D top template, and a 3D dress template.

11. A system for generating a three-dimensional (3D) model of an object based on a two-dimensional (2D) image of the object, comprising:

a database storing a plurality of 3D model templates associating with different types of objects, wherein each 3D model has a plurality of bisectional planes, wherein each bisectional plane comprises a group of 3D control points along the edge of the bisectional plane; and

a 3D model generation engine comprising at least one processor and a memory, the memory storing software instructions that when executed by the at least one processor, causing the at least one processor to perform the following steps;

selecting a 3D model template from the plurality of 3D model templates based on a type of object appeared in the 2D image;

determining locations for a plurality of 2D control points on a representation of the object appeared in the 2D image, wherein each 2D control points in the plurality of 2D control points corresponds to a distinct 3D control point; and

modifying the 3D model template by adjusting each bisectional plane based on the determined locations of the plurality of 2D control points on the 2D image.

12. The system of claim 11, wherein the 3D model generation engine is further programmed to determining the locations for the plurality of 2D control points by:

obtaining a 2D contour template having an outline of the object and 2D control points located along the outline;

overlaying the 2D image over the 2D contour template; and

adjusting the locations of the 2D control points on the 2D contour template by analyzing a difference between the outline of the object in the 2D contour template and the representation of the object in the 2D image.

13. The system of claim 12, wherein the 3D model generation engine is programmed to analyze the difference by performing a pixel-level analysis.

14. The system of claim 12, wherein the 3D model generation engine is programmed to analyze the difference by matching a pixel on the 2D image with a 2D control points on the 2D contour template.

15. The system of claim 12, wherein the 3D model generation engine is programmed to analyze the difference by determining a first non-alpha pixel from one side of the 2D image to the other side of the 2D image.

16. The system of claim 11, wherein at least two of the bisectional planes are parallel to each other.

17. The system of claim 11, wherein the 2D image comprises a representation of a front view of the object.

18. The system of claim 17, wherein the 3D model generation engine is further programmed to:

obtain a second 2D image comprising a representation of a back view of the object;

determine locations for a second plurality of 2D control points on the representation of the back view of the object appeared on the second 2D image;

modify the 3D model template by adjusting each bisectional plane based on the determined locations of the second plurality of 2D control points on the second 2D image.

19. The system of claim 11, wherein the object is a piece of clothing.

20. The system of claim 11, wherein the plurality of 3D model templates comprise at least a 3D pants template, a 3D top template, and a 3D dress template.