Assessing clothing style and fit using 3D models of customers

• US 10,664,903 B1
• Filed: 05/30/2017
• Issued: 05/26/2020
• Est. Priority Date: 04/27/2017
• Status: Active Grant

• Current Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Original Assignee: Amazon Technologies, Inc. (Amazon.com, Inc.)
• Inventors: Haitani, Robert Yuji, Hazlewood, William R., Mendili, Alaa-Eddine, Pham, Dominick Khanh
• Primary Examiner(s): Airapetian, Mila
• Application Number:
  US15/608,944
• Time in Patent Office: 1,092 Days
• Field of Search: None
• CPC Class Codes: A63F 2300/5553 user representation in the game field, e.g. avatar
• CPC Class Codes: G01S 17/894 3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
• CPC Class Codes: G02B 2027/0134 of stereoscopic type
• CPC Class Codes: G02B 2027/0138 comprising image capture systems, e.g. camera
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 1/163 Wearable computers, e.g. on a belt
• CPC Class Codes: G06F 1/1686 the I/O peripheral being an integrated camera
• CPC Class Codes: G06F 16/38 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 21/31 User authentication
• CPC Class Codes: G06F 3/011 Arrangements for interaction with the human body, e.g. for user immersion in virtual reality blind teaching G09B21/00
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06F 3/0486 Drag-and-drop
• CPC Class Codes: G06F 3/1407 General aspects irrespective of display type, e.g. determination of decimal point position, display with fixed or driving decimal point, suppression of non-significant zeros
• CPC Class Codes: G06Q 30/0633 Lists, e.g. purchase orders, compilation or processing
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 19/006 Mixed reality object pose determination, tracking or camera calibration for mixed reality G06T7/00
• CPC Class Codes: G06T 2210/16 Cloth
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: H04W 88/02 : Terminal devices

Abstract

Three-dimensional models (or avatars) may be defined based on imaging data captured from a customer. The avatars may be based on a virtual mannequin having one or more dimensions in common with the customer, a body template corresponding to the customer, or imaging data captured from the customer. The avatars are displayed on displays or in user interfaces and used for any purpose, such as to depict how clothing will appear or behave while being worn by a customer alone or with other clothing. Customers may drag-and-drop images of clothing onto the avatars. One or more of the avatars may be displayed on any display, such as a monitor or a virtual reality headset, which may depict the avatars in a static or dynamic mode. Images of avatars and clothing may be used to generate print catalogs depicting the appearance or behavior of the clothing while worn by the customer.

Claims

1. A method comprising:

receiving a keyword from a customer over a network, wherein the keyword is received via a first network page displayed on a display of a computer device;

identifying a plurality of wearable items corresponding to the keyword;

causing a display of a second network page on the display, wherein the second network page comprises information regarding at least some of the plurality of wearable items;

receiving a selection of information regarding a first wearable item, wherein the first wearable item is one of the at least some of the plurality of wearable items, and wherein the first wearable item is available in one of a plurality of sizes;

accessing a customer profile of the customer, wherein the customer profile comprises a three-dimensional model of the customer and a purchasing history of the customer, and wherein the three-dimensional model is generated based at least in part on depth imaging data captured from the customer;

selecting one of the plurality of the sizes associated with the customer based at least in part on at least one of the three-dimensional model of the customer or the purchasing history of the customer;

determining material data for the first wearable item, wherein the material data comprises at least one of;

a tensile strength of at least one component of the first wearable item;

a yield strength of the at least one component of the first wearable item;

or

a Young'"'"'s modulus of the at least one component of the first wearable item;

generating a first predicted image of a visual representation of the first wearable item in the selected size on the customer based at least in part on the three-dimensional model and the material data; and

causing a display of a third network page on the display, wherein the third network page comprises;

the first predicted image; and

an indicator of a quality of a fit of the first wearable item in the selected size.

2. The method of claim 1, further comprising:

capturing, by a depth sensing camera in communication with the computer device, the depth imaging data from the customer at a first time, wherein the depth imaging data comprises at least a first depth image of the customer in a first pose and a second depth image of the customer in a second pose;

defining a point cloud associated with the customer based at least in part on the depth imaging data, wherein the point cloud comprises position data for a plurality of points in three-dimensional space, and wherein each of the first plurality of points corresponds to a point on an external surface of the customer; and

generating the three-dimensional model of the customer based at least in part on the point cloud, wherein the three-dimensional model represents at least a portion of an external surface of the customer, and

wherein the first predicted image depicts the visual representation of the first wearable item in the selected size on the customer in one of;

the first pose;

the second pose;

or

a third pose.

3. The method of claim 1, wherein the third network page further comprises:

a prompt to purchase the first wearable item in the selected size; and

a prompt to select another size of the first wearable item.

18. The method of claim 1, wherein the computer device is a mobile device, and

wherein the depth imaging data was captured using a depth sensing camera that is an integral component of the mobile device.

4. A computer-implemented method comprising:

receiving a selection of an article of clothing from a computer device associated with a customer;

identifying material data regarding at least one component of the article of clothing, wherein the material data comprises at least one of a tensile strength of the at least one component, a compression strength of the at least one component, a shear strength of the at least one component, a yield strength of the at least one component, a coefficient of friction of the at least one component, a Young'"'"'s modulus of the at least one component, a Poisson ratio of the at least one component, a shear modulus of the at least one component, a stress-strain curve of the at least one component, a hardness of the at least one component, or a toughness of the at least one component;

capturing, by a depth sensing camera in communication with the computer device, a plurality of depth images with at least a portion of the customer within a field of view of the depth sensing camera, wherein at least a first depth image of the plurality of depth images is captured with the customer in a first pose, and wherein at least a second depth image of the plurality of depth images is captured with the customer in a second pose;

determining at least one physical attribute of the customer based at least in part on at least some of the plurality of depth images;

generating an avatar of the customer based at least in part on the at least one physical attribute of the customer;

generating at least a first predicted image of an appearance of the article of clothing on the customer based at least in part on the avatar of the customer and the material data regarding the at least one component of the article of clothing; and

causing a display of information on at least one display, wherein the information comprises;

the first predicted image of the appearance of the article of clothing on the customer; and

an indicator of a quality of the appearance of the article of clothing on the customer.

5. The computer-implemented method of claim 4, wherein the indicator is that the quality of the appearance of the article of clothing on the customer is satisfactory.

6. The computer-implemented method of claim 4, wherein the first predicted image depicts the appearance of the article of clothing on the customer in at least one of:

the first pose;

the second pose;

or

a third pose.

7. The computer-implemented method of claim 4, further comprising:

generating a point cloud based at least in part on the at least some of the plurality of depth images, wherein the point cloud comprises coordinates of a plurality of points in space corresponding to the customer, and wherein each of the points in space is a point on an external surface of the customer; and

defining a textured mesh from the point cloud,

wherein the avatar is the textured mesh.

8. The computer-implemented method of claim 7, further comprising:

capturing, by a visual camera in communication with the computer device, a plurality of visual images with at least a portion of the customer within a field of view of the visual camera; and

wherein defining the textured mesh from the point cloud comprises mapping at least one of the visual images to the point cloud.

9. The computer-implemented method of claim 4, wherein the at least one display is at least one of:

a flat-panel display associated with the customer;

a display provided on a mobile device of the customer;

or

at least one display within a frame of a virtual reality system configured for wearing about a head of the customer.

10. The computer-implemented method of claim 4,

wherein the at least one physical attribute is at least one of:

a height of the customer;

a weight of the customer;

a shoulder width of the customer;

a chest width of the customer;

an arm circumference of the customer;

an arm length of the customer;

a waist circumference of the customer;

a leg length of the customer;

or

a leg circumference of the customer.

11. The computer-implemented method of claim 4, wherein the article of clothing is available in a plurality of sizes, and

wherein the computer-implemented method further comprises:

receiving a selection of one of the plurality of sizes from the customer; and

determining the appearance of the article of clothing on the customer based at least in part on the selected size.

12. The computer-implemented method of claim 11, wherein the information further comprises:

a prompt to purchase the article of clothing in the selected size; and

a prompt to select a size of the first wearable item other than the selected size.

13. The computer-implemented method of claim 4, further comprising:

receiving an account identifier of an associate of the customer, wherein the account identifier is one of a telephone number, an electronic mail address or a social network address;

wherein causing the display of the information on the at least one display comprises;

transmitting code for causing the display of the information to a computer device of the associate via the account identifier over a network.

19. The computer-implemented method of claim 4, wherein the computer device is a mobile device, and

wherein the depth sensing camera is an integral component of the mobile device.

14. A method comprising:

receiving, by a server associated with a marketplace, a keyword from a user over a network, wherein the keyword is entered by the user at a user interface displayed on a computer display of a computer device;

causing a display of a plurality of items on the user interface, wherein each of the items is associated with the keyword, and wherein at least one item is an article of clothing available in a plurality of sizes;

receiving a selection of one of the plurality of items via the user interface;

receiving a selection of one of the plurality of sizes of the selected item via the user interface;

accessing a profile of the user maintained in the server, wherein the profile comprises;

first information regarding at least one item previously purchased by the user from the marketplace, wherein the first information comprises a size of the at least one item previously purchased by the customer; and

second information regarding at least one physical attribute of the user, wherein the at least one physical attribute is determined based at least in part on at least one depth image captured of the user;

identifying one of the plurality of sizes based at least in part on at least one of the first information or the second information;

determining that the selected one of the plurality of sizes is not the one of the plurality of sizes identified based at least in part on at least one of the first information or the second information;

in response to determining that the selected one of the plurality of sizes is not the one of the plurality of sizes identified based at least in part on at least one of the first information or the second information,

determining a material property of at least one component of the article of clothing;

generating a three-dimensional model of the user based at least in part on the second information;

generating at least one predicted image of an appearance of the article of clothing on the user based at least in part on the material property and the three-dimensional model; and

causing a display of the at least one predicted image on the user interface.

15. The method of claim 14, wherein causing the display of the visual representation on the user interface comprises:

causing a display of each of an indicator of the appearance of the article of clothing, a prompt to purchase the article of clothing in the selected size, a prompt to select another size of the article of clothing and the visual representation on the user interface.

16. The method of claim 14, further comprising:

determining an account identifier of an associate of the user, wherein the account identifier is one of a telephone number, an electronic mail address or a social network address; and

causing at least the visual representation of the three-dimensional model with the appearance of the article of clothing on the user mapped thereon to be transmitted to a computer device of the associate via the account identifier.

17. The method of claim 14, further comprising:

receiving, from a depth sensing camera in communication with the computer device, a plurality of depth images with at least a portion of the user within a field of view of the depth sensing camera, wherein at least a first depth image of the plurality of depth images is captured with the user in a first pose, and wherein at least a second depth image of the plurality of depth images is captured with the user in a second pose;

determining the at least one physical attribute of the customer based at least in part on at least some of the plurality of depth images; and

storing the at least one physical attribute in the profile of the user,

wherein the at least one physical attribute is at least one of;

a height of the customer;

a weight of the customer;

a shoulder width of the customer;

a chest width of the customer;

an arm circumference of the customer;

an arm length of the customer;

a waist circumference of the customer;

a leg length of the customer;

or

a leg circumference of the customer.

20. The method of claim 17, wherein the computer device is a mobile device, and

wherein the depth sensing camera is an integral component of the mobile device.