Processing user selectable product images and facilitating visualization-assisted coordinated product transactions

• US 10,290,136 B2
• Filed: 09/12/2016
• Issued: 05/14/2019
• Est. Priority Date: 08/10/2016
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Zeekit Online Shopping Ltd. (WalMart Inc.)
• Inventors: Kristal, Alon, Appleboim, Nir, Wiesel, Yael, Zimmerman, Israel Harry
• Primary Examiner(s): Tswei, YuJang
• Application Number:
  US15/263,034
• Publication Number:
  US 20180047192A1
• Time in Patent Office: 974 Days
• Field of Search: None
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/20 Ensemble learning
• CPC Class Codes: G06Q 30/0643 Graphical representation of items or shoppers
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2200/24 involving graphical user interfaces [GUIs]
• CPC Class Codes: G06T 3/40 Scaling of whole images or parts thereof, e.g. expanding or contracting

Abstract

A technique for combining user selectable product images and facilitating visualization-assisted coordinated product acquisition. One or more products having one or more associated product images may be identified. The one or more product images may be image processed by incorporating the image(s) into an anchor image to generate a composite anchor image that depicts the one or more products according to an intended manner of use thereof. The image processing may include using product dimension information to scale the one or more product images as necessary to match a scale of the anchor image. The composite anchor image may be caused to be displayed in association with one or more user interface elements, the user interface elements being operable to initiate a coordinated product transaction involving the one or more products. Responsive to the one or more user interface elements being activated, the coordinated product transaction may be initiated.

Claims

1. A method for processing user selectable product images and facilitating visualization-assisted coordinated product transactions, comprising:

receiving a user image comprising a photograph or video of a real human subject in digital form;

incorporating said user image into an anchor image representing a digital canvas to which one or more clothing product images may be added;

preparing said user image for virtual dressing by modifying said user image into a modified user image that depicts said real human subject in a display format that facilitates realistic combining with said one or more product images;

said user image modifying including one or more user image processing operations performed on said user image, said one or more user image processing operations being selected from the group consisting of image enhancement and perspective correction, user extraction, volumetric reflectance mapping, figure estimation, measurements estimation, body completion, and original garments in-painting;

identifying one or more clothing products having one or more associated clothing product images;

preparing said one or more clothing product images for virtual dressing by modifying said one or more clothing product images into one or more modified clothing product images that depict said clothing products in a display format that facilitates realistic combining with said modified user image, said clothing product image modifying including one or more clothing product image processing operations performed on said one or more clothing product images, said one or more clothing product image processing operations including shape-estimation and shape in-painting;

said shape-estimation comprising interpolating product shape for hidden areas of a clothing product that are not visible in a clothing product image;

said shape in-painting comprising filling in said hidden areas of a clothing product in a clothing product image that were interpolated by said shape-estimation;

performing virtual dressing of said modified user image with said one or more said one or more clothing product images by incorporating said one or more product images into said anchor image to generate a composite anchor image that depicts said real human subject wearing said one or more clothing products according to an intended manner of use thereof;

said virtual dressing including using product dimension information to scale said one or more clothing product images as necessary to match a scale of said anchor image and thereby depict how said one or more clothing products fit said real human subject;

said virtual dressing further including one or more of unification of blur level, sharpness level, lighting level and noise level of said modified user image and said one or more clothing product images;

causing said composite anchor image to be displayed;

causing one or more user interface elements to be displayed in association with said composite anchor image, said one or more user interface elements being operable to initiate a coordinated product transaction involving said one or more clothing products; and

responsive to said one or more user interface elements being activated, initiating said coordinated product transaction.

2. The method of claim 1, wherein there are two or more clothing products with two or more corresponding clothing product images, and said composite anchor image depicts said two or more clothing products in a spatial relationship corresponding to an intended manner of use of said two or more clothing products.

3. The method of claim 1, wherein said product dimension information is obtained from product dimension data maintained in association with said one or more clothing products or is derived from numeric or alphanumeric clothing product size data maintained in association with said one or more products.

4. The method of claim 1, wherein said coordinated product transaction is one of a purchase transaction or a redirect transaction.

5. The method of claim 1, wherein said one or more clothing product images are retrieved from separate sources of said one or more products.

6. The method of claim 1, wherein said one or more clothing product images are retrieved from a single source of all of said one or more clothing products.

7. The method of claim 1, wherein said composite anchor image depicts only said modified user image, said one or more clothing product images, and a background that is blank or includes a background image.

8. The method of claim 1, wherein said one or more clothing product image processing operations further include product extraction, said product extraction comprising generating a product mask in which one or more portions of said one or more product images are removed, added, completed or modified.

9. The method of claim 8, wherein said virtual dressing includes one or more virtual dressing processing operations performed on said composite anchor image, said one or more virtual dressing processing operations being selected from the group consisting of product size estimation, estimation of product shape on said real human subject according to said modified user image and fabric type, estimation of clothing product shape on said real human subject according to real world gravitation, light extrapolation, shadow estimation, processing to determine limitations for said one or more clothing products that overlap, and perspective correction reversal.

10. The method of claim 7, wherein said composite anchor image includes said modified user image superimposed on a background image that does not include any portion of said modified user image, and said modified user image or said background image are scaled to match each other to maintain correct proportion between said background image providing a background and said modified user image providing a foreground in said composite anchor image.

11. The method of claim 1, wherein said one or more clothing product image processing operations further include perspective and shape correction, said perspective and shape correction comprising preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image.

12. The method of claim 1, wherein said one or more clothing product image processing operations further include perspective and shape correction, said perspective and shape correction comprising preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image.

13. The method of claim 1, wherein said one or more clothing product image processing operations further include features and classification processing, said features and classification processing comprising extracting product characteristics and points-of-interest from a clothing product image and classifying said clothing product based on said product characteristics and points-of-interest.

14. A system, comprising:

one or more processors operably coupled to one or more memory devices;

program instructions stored in said memory and executable by said one or more processors to perform operations for processing user selectable product images and facilitating visualization-assisted coordinated product transactions, said operations comprising;

receiving a user image comprising a photograph or video of a real human subject in digital form;

incorporating said user image into an anchor image representing a digital canvas to which one or more clothing product images may be added;

preparing said user image for virtual dressing by modifying said user image into a modified user-image that depicts said real human subject in a display format that facilitates realistic combining with said one or more product images;

said user image modifying including one or more user image processing operations performed on said user image, said one or more user image processing operations being selected from the group consisting of image enhancement and perspective correction, user extraction, volumetric reflectance mapping, figure estimation, measurements estimation, body completion, and original garments in-painting;

identifying one or more clothing products having one or more associated clothing product images;

preparing said one or more clothing product images for virtual dressing by modifying said one or more clothing product images into one or more modified clothing product images that depict said clothing products in a display format that facilitates realistic combining with said modified user image, said clothing product image modifying including one or more clothing product image processing operations performed on said one or more clothing product images, said one or more clothing product image processing operations including shape estimation and shape in-painting;

said shape-estimation comprising interpolating product shape for hidden areas of a clothing product that are not visible in a clothing product image;

said shape in-painting comprising filling in said hidden areas of a clothing product in a clothing product image that were interpolated by said shape-estimation;

performing virtual dressing of said one or more clothing product images by incorporating said one or more clothing product images into said anchor image to generate a composite anchor image that depicts said real human subject wearing said one or more clothing products according to an intended manner of use thereof;

said virtual dressing including using product dimension information to scale said one or more clothing product images as necessary to match a scale of said anchor image and thereby depict how said one or more clothing products fit said real human subject;

said virtual dressing further including one or more of unification of blur level, sharpness level, lighting level and noise level of said modified user image and said one or more clothing product images;

causing said composite anchor image to be displayed;

causing one or more user interface elements to be displayed in association with said composite anchor image, said one or more user interface elements being operable to initiate a coordinated product transaction involving said one or more clothing products; and

responsive to said one or more user interface elements being activated, initiating said coordinated product transaction.

15. The system of claim 14, wherein there are two or more products with two or more corresponding clothing product images, and said composite anchor image depicts said two or more clothing products in a spatial relationship corresponding to an intended manner of use of said two or more clothing products.

16. The system of claim 14, wherein said product dimension information is obtained from product dimension data maintained in association with said one or more clothing products or is derived from numeric or alphanumeric product size data maintained in association with said one or more clothing products.

17. The system of claim 14, wherein said coordinated product transaction is one of a purchase transaction or a redirect transaction.

18. The system of claim 14, wherein said one or more clothing product images are retrieved from separate sources of said one or more clothing products.

19. The system of claim 14, wherein said one or more clothing product images are retrieved from a single source of all of said one or more clothing products.

20. The system of claim 14, wherein said composite anchor image depicts only said modified user image, said one or more clothing product images, and a background that is blank or includes a background image.

21. The system of claim 14, wherein said one or more clothing product image processing operations further include product extraction, said product extraction comprising generating a product mask in which one or more portions of said one or more product images are removed, added, completed or modified.

22. The system of claim 21, wherein said virtual dressing includes one or more virtual dressing processing operations performed on said composite anchor image, said one or more virtual dressing processing operations being selected from the group consisting of product size estimation, estimation of product shape on said real human subject according to said modified user image and fabric type, estimation of clothing product shape on said real human subject according to real world gravitation, light extrapolation, shadow estimation, processing to determine limitations for said one or more clothing products that overlap, and perspective correction reversal.

23. The system of claim 22, wherein said composite anchor image includes said modified user image superimposed on a background image that does not include any portion of said modified user image, and said modified user image or said background image are scaled to match each other to maintain correct proportion between said background image providing a background and said modified user image providing a foreground in said composite anchor image.

24. The system of claim 14, wherein said one or more clothing product image processing operations further include perspective and shape correction, said perspective and shape correction comprising preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image.

25. The system of claim 14, wherein said one or more clothing product image processing operations further include perspective and shape correction, said perspective and shape correction comprising preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image.

26. The system of claim 14, wherein said one or more clothing product image processing operations further include features and classification processing, said features and classification processing comprising extracting product characteristics and points-of-interest from a clothing product image and classifying said clothing product based on said product characteristics and points-of-interest.

27. A computer program product, comprising:

one or more non-transitory data storage media;

program instructions stored on said data storage media and executable by one or more processors to perform operations for processing user selectable product images and facilitating visualization-assisted coordinated product transactions, said operations comprising;

receiving a user image comprising a photograph or video of a real human subject in digital form;

incorporating said user image into an anchor image representing a digital canvas to which one or more clothing product images may be added;

preparing said user image for virtual dressing by modifying said user image into a modified user image that depicts said real human subject in a display format that facilitates realistic combining with said one or more product images;

said user image modifying including one or more user image processing operations performed on said user image, said one or more user image processing operations being selected from the group consisting of image enhancement and perspective correction, user extraction, volumetric reflectance mapping, figure estimation, measurements estimation, body completion, and original garments in-painting;

identifying one or more clothing products having one or more associated clothing product images;

preparing said one or more clothing product images for virtual dressing by modifying said one or more clothing product images into one or more modified clothing product images that depict said clothing products in a display format that facilitates realistic combining with said modified user image, said clothing product image modifying including one or more clothing product image processing operations performed on said one or more clothing product images, said one or more clothing product image processing operations including shape estimation and shape in-painting;

said shape-estimation comprising interpolating product shape for hidden areas of a clothing product that are not visible in a clothing product image;

said shape in-painting comprising filling in said hidden areas of a clothing product in a clothing product image that were interpolated by said shape-estimation;

performing virtual dressing of said modified user image with said one or more clothing product images by incorporating said one or more clothing product images into said anchor image to generate a composite anchor image that depicts said real human subject wearing said one or more clothing products according to an intended manner of use thereof;

said virtual dressing including using product dimension information to scale said clothing product images as necessary to match a scale of said anchor image and thereby depict how said one or more clothing products fit said real human subject;

said virtual dressing further including one or more of unification of blur level, sharpness level, lighting level and noise level of said modified user image and said one or more clothing product images;

causing said composite anchor image to be displayed;

causing one or more user interface elements to be displayed in association with said composite anchor image, said one or more user interface elements being operable to initiate a coordinated product transaction involving said one or more clothing products; and

responsive to said one or more user interface elements being activated, initiating said coordinated product transaction.

28. The computer program product of claim 27, wherein there are two or more clothing products with two or more corresponding clothing product images, and said composite anchor image depicts said two or more clothing products in a spatial relationship corresponding to an intended manner of use of said two or more clothing products.

29. The computer program product of claim 27, wherein said product dimension information is obtained from product dimension data maintained in association with said one or more clothing products or is derived from numeric or alphanumeric clothing product size data maintained in association with said one or more products.

30. The computer program product of claim 27, wherein said coordinated product transaction is one of a purchase transaction or a redirect transaction.

31. The computer program product of claim 27, wherein said one or more clothing product images are retrieved from separate sources of said one or more clothing products.

32. The computer program product of claim 27, wherein said one or more clothing product images are retrieved from a single source of all of said one or more clothing products.

33. The computer program product of claim 27, wherein said composite anchor image depicts only said modified user image, said one or more clothing product images, and a background that is blank or includes a background image.

34. The computer program product of claim 27, wherein said one or more clothing product image processing operations further include product extraction, said product extraction comprising generating a product mask in which one or more portions of said one or more product images are removed, added, completed or modified.

35. The computer program product of claim 34, wherein virtual dressing includes one or more virtual dressing processing operations performed on said composite anchor image, said one or more virtual dressing processing operations being selected from the group consisting of product size estimation, estimation of product shape on said real human subject according to said modified user image and fabric type, estimation of clothing product shape on said real human subject according to real world gravitation, light extrapolation, shadow estimation, processing to determine limitations for said one or more clothing products that overlap, and perspective correction reversal.

36. The computer program product of claim 35, wherein said composite anchor image includes said modified user image superimposed on a background image that does not include any portion of said modified user image, and said modified user image or said background image are scaled to match each other to maintain correct proportion between said background image providing a background and said modified user image providing a foreground in said composite anchor image.

37. The computer program product of claim 27, wherein said one or more clothing product image processing operations further include perspective and shape correction, said perspective and shape correction comprising preparing a clothing product for dressing to add, reduce, create or modify wrinkle effects in a clothing product image.

38. The computer program product of claim 27, wherein said one or more clothing product image processing operations further include perspective and shape correction, said perspective and shape correction comprising preparing a clothing product for dressing to add 3D effects in a clothing product image by modifying an angular perspective and product shape to change non-curved contour lines to curved contour lines in a clothing product image.

39. The computer program product of claim 27, wherein said one or more clothing product image processing operations further include features and classification processing, said features and classification processing comprising extracting product characteristics and points-of-interest from a clothing product image and classifying said clothing product based on said product characteristics and points-of-interest.