Adaptive clothing 3D model

• US 10,740,591 B2
• Filed: 08/23/2016
• Issued: 08/11/2020
• Est. Priority Date: 08/04/2015
• Status: Active Grant

• Current Assignee: Adobe Inc.
• Original Assignee: Adobe Inc.
• Inventors: Antipa, Damien, Capt, Alexandre
• Primary Examiner(s): Cese, Kenny A
• Application Number:
  US15/244,749
• Publication Number:
  US 20170039416A1
• Time in Patent Office: 1,449 Days
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/56 relating to colour
• CPC Class Codes: G06V 10/759 Region-based matching
• CPC Class Codes: G06V 20/64 Three-dimensional objects

Abstract

Systems and methods provide adapted content to a visitor to a physical environment. An example method receives an image of a visitor to an environment. A visitor portion of the image is distinct from an environment portion of the image. The method detects one or more shapes in the visitor portion of the image using an automatic shape detection technique and defines an approximate boundary of the one or more shapes using a mask. The one or more shapes can be shapes of the visitor'"'"'s clothing items. The method then calculates an attribute for an area of the image within the mask and identifies electronic content based on the attribute for the area of the image within the mask. The attribute can be a color attribute for the area such as a median color or a dominant color. The method provides the identified electronic content for display in the environment.

Claims

1. A computer-implemented method comprising:

receiving, at a computing device, an image of a user and a mask that corresponds to a clothing shape depicted in the image of the user;

calculating, by the computing device, one or more color attributes for an area of the image within the mask, wherein each of the one or more calculated color attributes indicates a color of clothing worn by the user and is based on a color analysis of pixels in the image that correspond to the clothing shape indicated by the mask;

determining, based on the one or more calculated color attributes, a complimentary color scheme including multiple colors;

determining a compatible color attribute that is based on the one or more calculated color attributes for the area of the image and the complimentary color scheme;

identifying, by the computing device, electronic content comprising a three-dimensional (3D) model, the identifying based on the one or more calculated color attributes for the area of the image;

adapting, based on the compatible color attribute, the 3D model, wherein the adapted 3D model has a color scheme that is based on the compatible color attribute;

generating modified electronic content based on the adapted 3D model, wherein the modified electronic content includes the color scheme that is based on the compatible color attribute; and

providing the modified electronic content for display.

2. The method of claim 1, wherein the mask corresponds to multiple clothing shapes depicted in the image, the method further comprising:

determining, for each of the multiple clothing shapes, a respective subset of the pixels in the image, wherein the respective pixel subset corresponds to a respective one of the multiple clothing shapes and each pixel in the respective pixel subset is classified as belonging to the respective one of the multiple clothing shapes; and

determining, based on an additional color analysis of the respective pixel subset, an additional color attribute for an additional area of the image, the additional color attribute indicating an additional color of the respective one of the multiple clothing shapes,

wherein determining the complimentary color scheme is further based on the additional color attribute of each of the multiple clothing shapes.

3. The method of claim 1, wherein calculating the one or more calculated color attributes for the area of the image comprises one or more of:

defining, by the computing device, the mask corresponding to the clothing shape in the image, wherein the area of the image is within the mask;

analyzing pixels of the area of the image within the mask to calculate a statistical median color for the area of the image within the mask;

analyzing pixels of the area of the image within the mask to determine a dominant color of the area of the image within the mask;

determining a shape of the clothing shape in the area of the image within the mask to determine a type of the clothing worn by the user;

or

calculating dimensions of an item in the area of the image within the mask to determine a size of the clothing worn by the user.

4. The method of claim 3, wherein detecting the clothing shape comprises using an automatic shape detection technique to detect the clothing shape in the image of the user.

5. The method of claim 3, wherein determining the shape in the image comprises performing assistive clothing pattern recognition to detect the clothing shape in the image.

6. The method of claim 1, wherein the identified electronic content further comprises a plurality of images of a product.

7. The method of claim 6, wherein the 3D model is a product model depicting the product.

8. The method of claim 1, wherein the identified electronic content further includes:

web content from a content management tool; and

an asset from an asset modeler,

the method further comprising;

displaying the web content and the asset on a display of the computing device.

9. A system comprising:

a processing device;

a display device;

an input device;

at least one camera; and

a content management tool implemented on the processing device and configured to perform operations for adapting electronic content for presentation to a user, the operations comprising;

capturing, by the at least one camera, an image of the user;

presenting, on the display device, electronic content items retrieved from a content repository;

receiving, via the input device, a selection of one or more of the presented electronic content items;

calculating one or more color attributes for an area of the image within a mask, the mask corresponding to a clothing shape depicted in the image of the user, wherein each of the one or more calculated color attribute indicates a color of clothing worn by the user and is based on a color analysis of pixels in the image that correspond to the clothing shape indicated by the mask;

determining, based on the one or more calculated color attributes, a complimentary color scheme including multiple colors;

determining a compatible color attribute that is based on the one or more calculated color attributes for the area of the image and the complimentary color scheme;

identifying a three-dimensional (3D) model associated with the selected electronic content item, the identifying based at least in part on the selection and the one or more calculated color attributes for the area of the image;

adapting the 3D model, wherein the adapted 3D model has a color scheme that is based on the compatible color attribute; and

updating the display device to present additional electronic content items each having a respective color scheme similar to the compatible color attribute.

10. The system of claim 9, wherein presenting the electronic content items comprises displaying life-sized representations of one or more products on the display device.

11. The system of claim 9, wherein the content repository comprises additional 3D models depicting products.

12. The system of claim 9, wherein updating the display device comprises presenting the additional electronic content items from a remote server, the remote server containing electronic content for products offered by a store.

13. The system of claim 9, wherein calculating the one or more calculated color attributes for the area of the image comprises:

detecting, by a shape detection module of the content management tool, the clothing shape depicted in the image of the user;

defining the mask such that boundaries of the mask correspond to boundaries of the clothing shape;

analyzing pixels of the area of the image within the mask to calculate respective statistical median colors for the area of the image within the mask; and

analyzing pixels of the area of the image within the mask to determine a respective dominant color of the area of the image within the mask.

14. The system of claim 9, the operations further comprising:

calculating an item type attribute for the area of the image within the mask, the item type attribute indicating a type of the clothing worn by the user,

wherein the selected electronic content item has a product type that is different from the calculated item type attribute.

15. The system of claim 14, wherein the mask corresponds to multiple clothing shapes depicted in the image, the operations further comprising:

determining, for each of the multiple clothing shapes, a respective subset of the pixels in the image, wherein the respective pixel subset corresponds to a respective one of the multiple clothing shapes; and

calculating, for each of the multiple clothing shapes, an additional item type attribute,

wherein determining the complimentary color scheme is further based on the additional item type attribute of each of the multiple clothing shapes.

16. A non-transitory computer readable storage medium having executable instructions stored thereon, that, if executed by a computing device, cause the computing device to perform operations, the instructions comprising:

instructions for receiving an image of a user;

instructions for calculating one or more color attributes for an area of the image within a mask, the mask corresponding to a clothing shape depicted in the image of the user, wherein each of the one or more calculated color attribute indicates a color of clothing worn by the user and is based on a color analysis of pixels in the image that correspond to the clothing shape indicated by the mask;

instructions for determining, based on the one or more calculated color attributes, a complimentary color scheme including multiple colors;

instructions for determining a compatible color attribute that is based on the one or more calculated color attributes for the area of the image and the complimentary color scheme;

instructions for identifying electronic content comprising a three-dimensional (3D) model, the identifying based on the one or more calculated color attributes for the area of the image;

instructions for adapting, based on the compatible color attribute, the 3D model, wherein the adapted 3D model has a color scheme that is based on the compatible color attribute;

instructions for generating modified electronic content based on the adapted 3D model, wherein the modified electronic content includes the color scheme that is based on the compatible color attribute; and

instructions for providing the modified electronic content for display.

17. The non-transitory computer readable storage medium of claim 16, the instructions further comprising:

instructions for displaying the adapted 3D model; and

instructions for receiving input for manipulating a spatial orientation of the displayed adapted 3D model.

18. The non-transitory computer readable storage medium of claim 17, the instructions further comprising:

instructions for receiving user input for manipulating additional attributes of the displayed adapted 3D model, the additional attributes including one or more colors of the displayed adapted 3D model.

19. The non-transitory computer readable storage medium of claim 16, wherein the instructions further comprise:

detecting the clothing shape in the image using an automatic shape detection technique to detect the clothing shape in the image of the user.

20. The non-transitory computer readable storage medium of claim 19, the instructions further comprising:

instructions for comparing the clothing shape in the image of the user to pre-defined shapes of clothing items.