Method, system, and device of virtual dressing utilizing image processing, machine learning, and computer vision

• US 10,534,809 B2
• Filed: 10/12/2018
• Issued: 01/14/2020
• Est. Priority Date: 08/10/2016
• Status: Active Grant

• Current Assignee: Walmart Apollo, LLC (WalMart Inc.)
• Original Assignee: Zeekit Online Shopping Ltd. (WalMart Inc.)
• Inventors: Wiesel, Yael, Appleboim, Nir, Kristal, Alon
• Primary Examiner(s): Tswei, YuJang
• Application Number:
  US16/158,313
• Publication Number:
  US 20190050427A1
• Time in Patent Office: 459 Days
• Field of Search: None
• CPC Class Codes: G06F 16/538 Presentation of query results
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06N 20/00 Machine learning
• CPC Class Codes: G06N 20/10 using kernel methods, e.g. support vector machines [SVM]
• CPC Class Codes: G06N 7/01 Probabilistic graphical models, e.g. probabilistic networks
• CPC Class Codes: G06T 11/00 2D [Two Dimensional] image generation
• CPC Class Codes: G06T 19/00 Manipulating 3D models or images for computer graphics
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 40/10 Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands

Abstract

System, device, and method of image processing, particularly utilizing machine learning and computer vision. A computerized method includes: obtaining a user-image of a user of an electronic device; generating a user-mask image that corresponds to the user-image processed to exclude background image-data of that user-image; receiving a product-image of a particular product; generating from the product-image a product-mask image that corresponds to the product-image processed to exclude background image-data of the product-image; receiving the product-mask image and the user-mask image, and generating from them a combined or composite user-and-product image which depicts that particular user engaging with that particular product.

Claims

1. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

at a first website, receiving a self-image from said user;

tracking online operations of said user across multiple websites, and extracting at least one common characteristic from said online operations of said user across multiple websites;

subsequently, at a second website, automatically selecting a particular product that has said at least one common characteristic, and displaying on said website a combined image that depicts said user engaging with said particular product.

2. The computerized method of claim 1, further comprising:

receiving a plurality of search results in response to a user query that searches for a product;

modifying each of said search results by replacing each of said search results with a combination image that depicts said user engaging with said product.

3. The computerized method of claim 1, further comprising:

receiving a user-query for a clothing article;

generating a plurality of image-based search results in response to said user-query, wherein each image-based search result corresponds to a different instance of said clothing article;

modifying each image-based search result into a combined image that depicts a combination of said user wearing said instance of the clothing article;

presenting to said user a set of image-based search results, wherein each image-based search result comprises a combined image that depicts the combination of said user wearing said instance of the clothing article.

4. The computerized method of claim 1, comprising:

storing a list that associates between (i) a particular video, (ii) a particular time-segment of said particular video, (iii) an associated product that is depicted in said video during said particular time-segment;

during playback of said video, at said particular time-segment, and independent of any user interaction, modifying a content that is displayed outside said video to show information about said associated product.

5. The computerized method of claim 1, comprising:

storing a list that associates between (i) a particular video, (ii) a particular time-segment of said particular video, (iii) a particular spatial-area of said video, (iv) an associated product that is depicted in said video during said particular time-segment;

during playback of said video, detecting a user-interaction with said particular spatial-area of said video during said particular time-segment of said video; and

in response, displaying to said user information about said associated product.

6. The computerized method of claim 1, comprising:

during a playback of a video, displaying externally to said video information about a particular product during a particular time-segment of said video, based on a pre-defined heat map that associates between different spatial-areas of said video and multiple different products;

wherein said displaying is performed in response to a user interaction with a particular spatial-area of said video;

wherein the user interaction comprises one or more of;

a click event, a tap event, a hover event.

7. The computerized method of claim 1, further comprising:

storing a pre-defined timing scheme that associates between time-segments of a video and multiple different products;

during playback of said video, and based on said timing scheme, and independently of any user interactions, displaying externally to said video a combined user-and-product image that depicts said user wearing one particular product of said different products.

8. The computerized method of claim 1, further comprising:

receiving user selection of multiple products;

obtaining from a video repository multiple video segments, wherein each video segment depicts one of said product being utilized by a person;

joining said multiple video segments into a combined video clip;

causing said video clip to playback on an electronic device.

9. The computerized method of claim 1,

wherein said at least one common characteristic is inferred by analyzing responses of said user to third-party requests to select a preferred image out of two or more images presented to said user.

10. The A computerized method of claim 4, comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

receiving multiple product-identifiers of multiple respective products;

based on said multiple product-identifiers, extracting a group of images corresponding to said products;

for each of said images, generating a combined user-and-product image depicting said user engaging with said product;

generating an on-screen interactive carousel interface that comprises a rotatable group of combined user-and-product images of said multiple products;

in response to a user command to scroll said on-screen interactive carousel interface, causing at least one of said combined user-and-product images to be hidden, and causing at least one new combined user-and-product images to be displayed.

11. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

receiving multiple pointers of multiple respective products;

based on said multiple pointers, extracting a group of images corresponding to said products;

generating a first combined user-and-product image, which depicts said user being virtually dressed by a first product;

displaying the first combined user-and-product image, and further displaying at least a second product having same functionality of said first product;

receiving a user command to virtually swap said first product with said second product, and in response;

generating a second combined user-and-product image which depicts said user being virtually dressed by the second product; and

displaying the second combined user-and-product image instead of the first user-and-product image.

12. The computerized method of claim 11,

wherein the computerized method is implemented via an on-screen interactive ribbon that is rotatable by said user to indicate a user request for virtual swap of products.

13. The computerized method of claim 11,

wherein the computerized method is implemented via at least two separate on-screen interactive ribbons that are rotatable by said user to indicate a user request for virtual swap of products;

wherein user interaction with a first on-screen interactive ribbon causes the method to generate and display a first new combined user-and-product image in which a first body-part of the user is virtually dressed with the first product;

wherein user interaction with a second on-screen interactive ribbon causes the method to generate and display a second new combined user-and-product image in which a second body-part of the user is virtually dressed with the second product.

14. The computerized method of claim 13, further comprising:

receiving user-input indicating whether to virtually stitch image-portions of combined user-and-product images via a first image-stitching process or via a second image-stitching pattern;

based on said user-input, applying either the first image-stitching process or the second image-stitching process to stitch together a first image-portion and a second image portion and to depict said user wearing two different products on two different body parts.

15. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

during a playback of a video, displaying externally to said video information about a particular product during a particular time-segment of said video, based on a pre-defined timing scheme that associates between time-segments of said video and multiple different products.

16. The computerized method of claim 15, further comprising:

at a first website, receiving a self-image from said user;

tracking online operations of said user across multiple websites;

subsequently, during a first-ever visit of said user to a second website, generating and displaying to said user a combined image of said user being virtually dressed with a product sold on said second website.

17. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

storing a list that associates between (i) a particular video, (ii) a particular time-segment of said particular video, (iii) an associated product that is depicted in said video during said particular time-segment;

during playback of said video, at said particular time-segment, and independently of any user interaction, modifying a content that is displayed outside said video to show a dynamically-generated combined user-and-product image that depicts said user wearing said product.

18. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

at a first website, receiving a self-image from said user;

tracking online operations of said user across multiple websites;

subsequently, during a first-ever visit of said user to a second website, generating and displaying to said user a user-specific landing page which comprises multiple images, wherein each image is a combined image of said user being virtually dressed with a different product that is sold on said second website.

19. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

at a first website, receiving a self-image from said user;

tracking online operations of said user across multiple websites;

subsequently, receiving from said user a search query;

obtaining a set of image search results to said search query;

modifying each image search result into a combined user-and-product image; and

displaying to said user in response to said search query a set of modified image search results wherein each modified image search result depicts said user being virtually dressed with a different product.

20. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

receiving multiple product-identifiers of multiple respective products;

based on said multiple product-identifiers, extracting a group of images corresponding to said products;

generating a group of combined user-and-product images, wherein each combined image depicts said user being virtually dressed by a different product;

generating program code that, when executed by an electronic device, causes said electronic device to dynamically generate an on-screen interface that enables a user to interact with at least one image of said group of combined user-and-product images.

21. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

receiving multiple product-identifiers of multiple respective user-selected products that were selected by a first user via a first electronic device;

based on said multiple product-identifiers, extracting a group of images corresponding to said products;

generating a group of combined user-and-product images, wherein each combined image depicts said first user being virtually dressed by a different product;

causing a second electronic device, operated by a second user, to display said group of combined user-and-product images;

receiving from said second user, via said second electronic device, an indication of which combined image said second user prefers; and

causing said first electronic device to notify said first user which combined image said second user prefers.

22. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

receiving multiple product-identifiers of multiple respective user-selected products that were selected by a first user via a first electronic device;

based on said multiple product-identifiers, extracting a group of images corresponding to said products;

generating a group of combined user-and-product images, wherein each combined image depicts said first user being virtually dressed by a different product;

causing multiple other electronic devices, operated by multiple other users, to display said group of combined user-and-product images;

receiving from said multiple other users, via said multiple other electronic devices, indication of which combined image each one of said other users prefers;

aggregating indications of users who prefer same combined image, and generating and displaying voting results with regard to said group of combined user-and-product images.

23. A computerized method comprising:

obtaining a user-image of a user of an electronic device;

generating a user-mask image that corresponds to said user-image processed to exclude background image-data of said user-image;

receiving a product-image of a particular product, and generating from said product-image a product-mask image that corresponds to said product-image processed to exclude background image-data of said product-image;

receiving said product-mask image and said user-mask image, and generating from them a combined user-and-product image that depicts said user engaging with said product;

receiving multiple product-identifiers of multiple respective user-selected products that were selected by a first user via a first electronic device;

based on said multiple product-identifiers, extracting a group of images corresponding to said products;

generating a group of combined user-and-product images, wherein each combined image depicts said first user being virtually dressed by a different product;

generating a wish-list of products associated with said first user, wherein the wish-list comprises said group of combined user-and-product images;

transferring to a second electronic device a shortcut that causes said second electronic device to display to a second user said wish-list of the first user which comprises said group of combined user-and-product images.