Systems and methods for image generation and modeling of complex three-dimensional objects

• US 9,734,631 B2
• Filed: 09/08/2015
• Issued: 08/15/2017
• Est. Priority Date: 07/22/2014
• Status: Expired due to Fees

• Current Assignee: Trupik, Inc.
• Original Assignee: Trupik, Inc.
• Inventors: Katpally Reddy, Vikranth, Tirumala, Sridhar, Inumpudi, Aravind, Harmon, David Joseph, Pulimamidi, Shivani
• Primary Examiner(s): Amin, Jwalant
• Application Number:
  US14/848,007
• Publication Number:
  US 20160071324A1
• Time in Patent Office: 707 Days
• Field of Search: None
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06T 7/62 of area, perimeter, diameter or volume
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation

Abstract

Exemplary embodiments described herein relate to systems and methods for generating an image comprising a three-dimensional (“3D”) model or replica of a subject. Such images may include the face of a human subject as well as views of the subject from various angles.

Claims

1. A computer-implemented method comprising:

receiving, by an image generation system from a client computing device over a network, data regarding dimensions of a human subject;

receiving, by the image generation system from the client computing device over the network, data regarding a garment to be displayed in conjunction with the human subject;

generating, by the image generation system and based on the data regarding the dimensions of the human subject and the data regarding the garment, an image that includes a model of the human subject wearing the garment; and

transmitting, by the image generation system, the image of the model of the human subject wearing the garment to the client computing device for display on a display screen in communication with the client computing device,

wherein the data regarding the dimensions of the human subject include an image received from an image creation device in communication with the client computing device, and the data regarding the garment includes an image of the garment from an image creation device, wherein generating the image includes analyzing the image of the garment to classify a thickness of the garment and generating the image based on the classified thickness of the garment.

2. The method of claim 1, wherein the data regarding the dimensions of the human subject includes an image of the human subject from an image creation device, and wherein generating the image includes:

analyzing the image of the human subject to identify a feature of the human subject including one or more of;

a texture of the human subject'"'"'s skin, a geometrical measurement associated with the human subject, and a color of the human subject'"'"'s skin; and

generating the image of the model of the human subject based on the identified feature.

3. The method of claim 1, wherein one or more of the data regarding the dimensions of the human subject and the data regarding the garment includes a plurality of images received from a plurality of image creation devices in communication with the client computing device, wherein the plurality of image creation devices are located in different positions relative to each other.

4. The method of claim 1, wherein the data regarding the dimensions of the human subject includes an image of the human subject from an image creation device, and wherein generating the image includes:

extracting a three-dimensional mesh from the image received from the image creation device; and

eliminating noise in the image by applying a depth limit associated with a typical human body to the three-dimensional mesh to remove data points outside the depth limit.

5. The method of claim 1, wherein generating the image includes:

extracting a three-dimensional mesh from the image received from the image creation device;

registering the three-dimensional mesh using an iterative closest point algorithm; and

generating the image of the model of the human subject based on the registered three-dimensional mesh representing a geometric likeness of the human subject wearing the garment.

6. The method of claim 5, wherein generating the image includes iteratively modifying the geometric likeness of a portion of the human subject by minimizing a least mean squared based energy function in each iteration until a cost function falls below a predetermined difference from the registered three-dimensional mesh.

7. The method of claim 6, wherein the portion of the human subject is a face of the human subject.

8. The method of claim 7, wherein the likeness of the face of the human subject is generated by deforming a spherical mesh, and wherein a curvature in the deformed spherical mesh is determined using a direct application of a second derivative.

9. The method of claim 8, wherein the likeness of the face of the human subject is generated by identifying a region of interest in the deformed spherical mesh by delimiting the curvature of the deformed spherical mesh within a range and supplementing the deformed spherical mesh using point data from the registered three-dimensional mesh.

10. The method of claim 9, wherein a surface of the likeness of the face of the human subject is generated by employing a Poisson surface creation algorithm using the point data from the registered three-dimensional mesh.

11. The method of claim 1, wherein generating the image includes applying a color to a feature in the generated image by:

identifying a color of the feature in the image received from the image capture device using a Euclidean distance based metric; and

modifying the identified color based on representative values for hue, saturation, and brightness selected for the feature.

12. The method of claim 1, wherein the data regarding the dimensions of the human subject includes measurements of at least a portion of the human subject.

13. The method of claim 1, wherein the data regarding the garment includes an identification of a plurality of different garments for display in the generated image, and wherein the image generation system generates a plurality of images, each of the plurality of images displaying a model of the human subject wearing a different combination of the plurality of different garments.

14. The method of claim 13, wherein generating the plurality of images includes generating, by the image generation system, combinations of the plurality of garments based on one or more of:

garments that are of different types, garments that match a preference of a user of the client computing device, garments previously purchased by a user of the client computing device, colors of the garments, and sizes of the garments.

15. The method of claim 13, wherein generating the plurality of images includes generating an image of the model of the human subject wearing a representative garment and modifying a feature of the representative garment to correspond to a garment from the plurality of garments.

16. The method of claim 1, wherein the model of the human subject is stored on the client computing device and transmitting the image of the model of the human subject wearing the garment includes transmitting unit size pattern information for the garment and an area of the garment for which the unit size pattern is repeatedly displayed.

17. A tangible, non-transitory, computer-readable medium storing instructions that, when executed by an image generation system, cause the image generation system to:

receive, from a client computing device over a network, data regarding dimensions of a human subject;

receive, from the client computing device over the network, data regarding a garment to be displayed in conjunction with the human subject;

generate, based on the data regarding the dimensions of the human subject and the data regarding the garment, an image that includes a model of the human subject wearing the garment; and

transmit the image of the model of the human subject wearing the garment to the client computing device for display on a display screen in communication with the client computing device,

wherein the data regarding the dimensions of the human subject include an image received from an image creation device in communication with the client computing device, and the data regarding the garment includes an image of the garment from an image creation device, wherein generating the image includes analyzing the image of the garment to classify a thickness of the garment and generating the image based on the classified thickness of the garment.

18. An image generation system comprising:

a processor; and

memory in communication with the processor and storing instructions that, when executed by the processor, cause the image generation system to;

receive, from a client computing device over a network, data regarding dimensions of a human subject;

receive, from the client computing device over the network, data regarding a garment to be displayed in conjunction with the human subject;

generate, based on the data regarding the dimensions of the human subject and the data regarding the garment, an image that includes a model of the human subject wearing the garment; and

transmit the image of the model of the human subject wearing the garment to the client computing device for display on a display screen in communication with the client computing device,

wherein the data regarding the dimensions of the human subject include an image received from an image creation device in communication with the client computing device, and the data regarding the garment includes an image of the garment from an image creation device, wherein generating the image includes analyzing the image of the garment to classify a thickness of the garment and generating the image based on the classified thickness of the garment.