Processing user selectable product images and facilitating visualization-assisted virtual dressing

• US 10,489,955 B2
• Filed: 02/19/2019
• Issued: 11/26/2019
• Est. Priority Date: 08/10/2016
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Zeekit Online Shopping Ltd. (WalMart Inc.)
• Inventors: Kristal, Alon, Appleboim, Nir, Wiesel, Yael, Zimmerman, Israel Harry
• Primary Examiner(s): Tswei, YuJang
• Application Number:
  US16/279,558
• Publication Number:
  US 20190259190A1
• Time in Patent Office: 280 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

A technique for combining user selectable product images and facilitating visualization-assisted virtual dressing. A user image comprising a photograph or video of a real human subject may be received digital form. The user image may be incorporated into an anchor image representing a digital canvas to which one or more clothing product images may be added. One or more clothing products having one or more associated product images may be identified. The one or more product images may be image-processed in various ways. Virtual dressing of the user image may be performed by incorporating the one or more product images into the anchor image to generate a composite anchor image that depicts the one or more products being worn by the human subject according to an intended manner of use thereof. The composite anchor image is caused to be displayed.

Claims

1. A method for processing product images and facilitating virtual dressing, comprising:

receiving a user image comprising a photograph or video of a real human subject in digital form;

incorporating said user image into an anchor image representing a digital canvas to which one or more clothing product images may be added;

identifying one or more clothing products having one or more associated clothing product images;

performing one or more clothing product image processing operations on said one or more clothing product images, said one or more clothing product image processing operations being selected from a group consisting of;

(1) shape-estimation and shape in-painting, said shape-estimation comprising interpolating product shape for hidden areas of a clothing product that are not visible in a clothing product image, and said shape in-painting comprising filling in said hidden areas of a clothing product in a clothing product image that were interpolated by said shape-estimation;

(2) perspective and shape correction, said perspective and shape correction comprising one or both of (a) preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image, or (b) preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image;

(3) features and classification processing, said features and classification processing comprising extracting product characteristics and points-of-interest from a clothing product image and classifying said clothing product based on said product characteristics and points-of-interest;

(4) product extraction, said product extraction comprising generating a product mask in which one or more portions of said one or more product images are removed, added, completed or modified;

(5) adding gravity effects, said adding gravity effects comprising simulating an effect of gravity on one or more portions of a clothing product by modifying said portions to appear to be dropping down due to gravity; and

(6) unifying a blur level, lighting level and noise level of the user image and the one or more clothing product images;

performing virtual dressing of said user image by incorporating said one or more clothing product images into said anchor image to generate a composite anchor image that depicts said real human subject wearing said one or more clothing products according to an intended manner of use thereof; and

causing said composite anchor image to be displayed.

2. The method of claim 1, wherein said one or more clothing product image processing operations include said shape-estimation and shape in-painting.

3. The method of claim 1, wherein said one or more clothing product image processing operations include said perspective and shape correction that includes said preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image.

4. The method of claim 1, wherein said one or more clothing product image processing operations include said perspective and shape correction that includes said preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image.

5. The method of claim 1, wherein said one or more clothing product image processing operations include said features and classification processing.

6. The method of claim 1, wherein said one or more clothing product image processing operations include said product extraction.

7. The method of claim 1, wherein said one or more clothing product image processing operations include said adding gravity effects.

8. The method of claim 1, wherein said one or more clothing product image processing operations include said unifying a blur level, lighting level and noise level of the user image and the one or more clothing product images.

9. A system, comprising:

one or more processors operably coupled to one or more memory devices;

program instructions stored in said memory and executable by said one or more processors to perform operations for processing user selectable product images and facilitating visualization-assisted virtual dressing, said operations comprising;

receiving a user image comprising a photograph or video of a real human subject in digital form;

incorporating said user image into an anchor image representing a digital canvas to which one or more clothing product images may be added;

identifying one or more clothing products having one or more associated clothing product images;

performing one or more clothing product image processing operations on said one or more clothing product images, said one or more clothing product image processing operations being selected from a group consisting of;

(1) shape-estimation and shape in-painting, said shape-estimation comprising interpolating product shape for hidden areas of a clothing product that are not visible in a clothing product image, and said shape in-painting comprising filling in said hidden areas of a clothing product in a clothing product image that were interpolated by said shape-estimation;

(2) perspective and shape correction, said perspective and shape correction comprising one or both of (a) preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image, or (b) preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image;

(3) features and classification processing, said features and classification processing comprising extracting product characteristics and points-of-interest from a clothing product image and classifying said clothing product based on said product characteristics and points-of-interest;

(4) product extraction, said product extraction comprising generating a product mask in which one or more portions of said one or more product images are removed, added, completed or modified;

(5) adding gravity effects, said adding gravity effects comprising simulating an effect of gravity on one or more portions of a clothing product by modifying said portions to appear to be dropping down due to gravity; and

(6) unifying a blur level, lighting level and noise level of the user image and the one or more clothing product images;

performing virtual dressing of said user image by incorporating said one or more clothing product images into said anchor image to generate a composite anchor image that depicts said real human subject wearing said one or more clothing products according to an intended manner of use thereof; and

causing said composite anchor image to be displayed.

10. The system of claim 9, wherein said one or more clothing product image processing operations include said shape-estimation and shape in-painting.

11. The system of claim 9, wherein said one or more clothing product image processing operations include said perspective and shape correction that includes said preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image.

12. The system of claim 9, wherein said one or more clothing product image processing operations include said perspective and shape correction that includes said preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image.

13. The system of claim 9, wherein said one or more clothing product image processing operations include said features and classification processing.

14. The system of claim 9, wherein said one or more clothing product image processing operations include said product extraction.

15. The system of claim 9, wherein said one or more clothing product image processing operations include said adding gravity effects.

16. The system of claim 9, wherein said one or more clothing product image processing operations include said unifying a blur level, lighting level and noise level of the user image and the one or more clothing product images.

17. A computer program product, comprising:

one or more non-transitory data storage media;

program instructions stored on said data storage media and executable by one or more processors to perform operations for processing user selectable product images and facilitating visualization-assisted virtual dressing, said operations comprising;

receiving a user image comprising a photograph or video of a real human subject in digital form;

incorporating said user image into an anchor image representing a digital canvas to which one or more clothing product images may be added;

identifying one or more clothing products having one or more associated clothing product images;

performing one or more clothing product image processing operations on said one or more clothing product images, said one or more clothing product image processing operations being selected from a group consisting of;

(1) shape-estimation and shape in-painting, said shape-estimation comprising interpolating product shape for hidden areas of a clothing product that are not visible in a clothing product image, and said shape in-painting comprising filling in said hidden areas of a clothing product in a clothing product image that were interpolated by said shape-estimation;

(2) perspective and shape correction, said perspective and shape correction comprising one or both of (a) preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image, or (b) preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image;

(3) features and classification processing, said features and classification processing comprising extracting product characteristics and points-of-interest from a clothing product image and classifying said clothing product based on said product characteristics and points-of-interest;

(4) product extraction, said product extraction comprising generating a product mask in which one or more portions of said one or more product images are removed, added, completed or modified;

(5) adding gravity effects, said adding gravity effects comprising simulating an effect of gravity on one or more portions of a clothing product by modifying said portions to appear to be dropping down due to gravity; and

(6) unifying a blur level, lighting level and noise level of the user image and the one or more clothing product images;

performing virtual dressing of said user image by incorporating said one or more product images into said anchor image to generate a composite anchor image that depicts said real human subject wearing said one or more clothing products according to an intended manner of use thereof; and

causing said composite anchor image to be displayed.

18. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said shape-estimation and shape in-painting.

19. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said perspective and shape correction that includes said preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image.

20. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said perspective and shape correction that includes said preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image.

21. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said features and classification processing.

22. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said product extraction.

23. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said adding gravity effects.

24. The computer program product of claim 17, wherein said one or more clothing product image processing operations include said unifying a blur level, lighting level and noise level of the user image and the one or more clothing product images.