Method and system for presenting personalized products based on digital signage for electronic commerce

• US 10,672,055 B2
• Filed: 05/23/2016
• Issued: 06/02/2020
• Est. Priority Date: 05/23/2016
• Status: Active Grant

• Current Assignee: Yahoo AD Tech LLC
• Original Assignee: Oath Inc. (Verizon Communications Inc.)
• Inventors: Hsiao, Jen-Hao, Chen, Shih-Ying
• Primary Examiner(s): Airapetian, Mila
• Application Number:
  US15/162,022
• Publication Number:
  US 20170337611A1
• Time in Patent Office: 1,471 Days
• Field of Search: 705 261
• CPC Class Codes: G06F 18/24133 Distances to prototypes
• CPC Class Codes: G06N 3/02 Neural networks
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G06Q 30/0631 Item recommendations
• CPC Class Codes: G06V 10/454 Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN]
• CPC Class Codes: G06V 10/764 using classification, e.g. of video objects
• CPC Class Codes: G06V 10/82 using neural networks
• CPC Class Codes: G06V 40/168 Feature extraction; Face representation
• CPC Class Codes: G07F 17/323 wherein the player is informed, e.g. advertisements, odds, instructions

Abstract

Method and system for electronic commerce are provided. An image of a user is obtained. A plurality of features based on the image of the user is determined. A group of products based on the plurality of features are selected. A recommendation to the user is provided based on the group of products.

Claims

1. A method, implemented on a machine having at least one processor, storage, and a communication platform connected to a network for recommending products, comprising:

obtaining, from a recommendation device, visual information related to a user;

determining at least one facial feature of the user and at least one non-facial feature of the user based on the visual information;

estimating an age and/or gender of the user based on the at least one facial feature;

determining one or more other users of the age and/or gender;

selecting (i) a first group of products the one or more other users previously exhibited interest in while interacting with the recommendation device and (ii) a second group of products based on the at least one non-facial feature; and

providing a recommendation to the user based on the first group of products and the second group of products.

2. The method of claim 1, wherein the at least one facial feature comprises a plurality of facial features, the at least one non-facial feature comprises a plurality of non-facial features, and the visual information comprises an image of the user, estimating the age and/or gender of the user comprises:

determining the plurality of facial features based on the image, the age and/or gender being estimated based on the plurality of facial features, and wherein the plurality of facial features and the plurality of non-facial features are determined by;

decomposing the image into a facial part of the image and at least one non-facial part of the image;

computing the plurality of facial features based on the facial part of the image; and

computing the plurality of non-facial features based on the at least one non-facial part of the image.

3. The method of claim 2, wherein the second group of products is selected based on the plurality of non-facial features.

4. The method of claim 2, wherein computing the plurality of non-facial features based on the at least one non-facial part of the image comprises:

computing the plurality of non-facial features based on each of the at least one non-facial part of the image using at least a convolutional neural network.

5. The method of claim 1, wherein selecting the first group of products comprises:

determining at least a subgroup of products from a purchase history database, the subgroup of products being previously purchased by a specific group of customers comprising the one or more other users;

calculating a score for each product of the subgroup of products based at least in part on the age and/or gender;

generating a ranking of the subgroup of products based on the score for each product of the subgroup of products, wherein the first group of products is selected from the subgroup of products based on the ranking.

6. The method of claim 5, wherein selecting the first group of products further comprises:

removing one or more products from the first group of products, the one or more products being purchased by previous customers of all genders and at least a predetermined range of ages.

7. The method of claim 5, wherein:

the visual information comprises an image of the user,

the at least one facial feature comprises a plurality of facial features determined based on the image, and

the plurality of facial features is used to estimate the age of the user, the gender of the user, or the age and the gender of the user.

8. The method of claim 7, wherein each of the specific group of customers has a same gender as the user and an age within a range of the age of the user.

9. The method of claim 3, wherein selecting the second group of products based on the plurality of non-facial features comprises:

selecting the second group of products from a product database, wherein a plurality of features of the second group of products matching the plurality of non-facial features are determined based on the product database.

10. The method of claim 1, further comprising:

executing a game program so that the user can play a game via the recommendation device, wherein in response to the user selecting one or more products from the first group of products, the second group of products, or both the first group of products and the second group of products for purchase, a discount is provided to the user via the recommendation device upon determining that the user won the game, and wherein executing the game comprises;

generating a virtual representation of the user to be presented on a display of the recommendation device, wherein in response to the user making a movement, the virtual representation also makes the movement which is presented on the display of the recommendation device.

11. A machine-readable tangible and non-transitory medium having information for recommending products, wherein the information, when read by the machine, causes the machine to perform the following:

obtaining, from a recommendation device, visual information related to of a user;

determining at least one facial feature of the user and at least one non-facial feature of the user based on the visual information;

estimating an age and/or gender of the user based on the at least one facial feature;

determining one or more other users of the age and/or gender;

selecting (i) a first group of products the one or more other users previously exhibited interest in while interacting with the recommendation device and (ii) a second group based on the at least one non-facial feature; and

providing a recommendation to the user based on the first group of products and the second group of products.

12. The machine-readable tangible and non-transitory medium of claim 11, wherein the at least one facial feature comprises a plurality of facial features, the at least one non-facial feature comprises a plurality of non-facial features, and the visual information comprises an image of the user, and wherein the information, when read by the machine, causes the machine to further perform the following:

determining the plurality of facial features based on the image, the age and/or gender being estimated based on the plurality of facial features, and wherein the plurality of facial features and the plurality of non-facial features are determined by;

decomposing the image into a facial part of the image and at least one non-facial part of the image;

computing the plurality of facial features based on the facial part of the image; and

computing the plurality of non-facial features based on the at least one non-facial part of the image.

13. The machine-readable tangible and non-transitory medium of claim 12, wherein the second group of products is selected based on the plurality of non-facial features.

14. The machine-readable tangible and non-transitory medium of claim 12, wherein the information associated with computing the plurality of non-facial features based on the at least one non-facial part of the image, when read by the machine, causes the machine to perform the following:

computing the plurality of non-facial features based on each of the at least one non-facial part of the image using at least a convolutional neural network.

15. The machine-readable tangible and non-transitory medium of claim 11, wherein the information associated with selecting the group of products, when read by the machine, causes the machine to perform the following:

determining at least a subgroup of products from a purchase history database, the at least a subgroup of products being previously purchased by a specific group of customers comprising the one or more other users;

calculating a score for each product of the subgroup of products based at least in part on the age and/or gender;

generating a ranking of the subgroup of products based on the score for each product of the subgroup of products, wherein the first group of products is selected from the subgroup of products based on the ranking.

16. The machine-readable tangible and non-transitory medium of claim 15, wherein the information associated with selecting the first group of products, when read by the machine, causes the machine to perform the following:

removing one or more products from the first group of products, the one or more products being purchased by previous customers of all genders and at least a predetermined range of ages.

17. The machine-readable tangible and non-transitory medium of claim 15, wherein the visual information comprises an image of the;

the at least one facial feature comprises a plurality of facial features determined based on the image; and

the plurality of facial features is used to estimate the age of the user, the gender of the user, or the age and the gender of the user.

18. The machine-readable tangible and non-transitory medium of claim 17, wherein each of the specific group of customers has a same gender as the user and an age within a range of the age of the user.

19. The machine-readable tangible and non-transitory medium of claim 13, wherein the information associated with selecting the second group of products based on the plurality of non-facial features, when read by the machine, causes the machine to perform the following:

selecting the second group of products from a product database, wherein a plurality of features of the second group of products matching the plurality of non-facial features are determined based on the product database.

20. The machine-readable tangible and non-transitory medium of claim 11, wherein the information, when read by the machine, further causes the machine to perform the following:

executing a game program so that the user can play a game via the recommendation device, wherein in response to the user selecting one or more products from the first group of products, the second group of products, or both the first group of products and the second group of products for purchase, a discount is provided to the user via the recommendation device upon determining that the user won the game, and wherein executing the game comprises;

generating a virtual representation of the user to be presented on a display of the recommendation device, wherein in response to the user making a movement, the virtual representation also makes the movement which is presented on the display of the recommendation device.

21. A system having at least one processor, storage, and a communication platform connected to a network for recommending products, comprising:

a recommendation device comprising;

a camera configured for obtaining visual information related to a user;

an image processor configured for;

determining at least one facial feature of the user and at least one non-facial feature of the user based on the visual information;

estimating an age and/or gender of the user based on the at least one facial feature;

determining one or more other users of the age and/or gender; and

obtaining a selection of (i) a first group of products the one or more other users previously exhibited interest in while interacting with the recommendation device and (ii) a second group of products based on the at least one non-facial feature; and

a display configured for providing a recommendation to the user based on the first group of products and the second group of products.

22. The system of claim 21, wherein the visual information comprises an image of the user, the at least one facial feature comprises a plurality of facial features, and the at least one non-facial feature comprises a plurality of non-facial features, wherein estimating the age and/or gender of the user comprises:

determining the plurality of facial features based on the image, the age and/or gender being estimated based on the plurality of facial features, and the image processor comprises;

a photo cropper configured for decomposing the image of the user into a facial part of the image and at least one non-facial part of the image;

a face-based recommender configured for selecting the group of products based on the facial part of the image; and

a body-based recommender configured for selecting an additional group of products based on the at least one non-facial part of the image.

23. The system of claim 22, wherein the face-based recommender comprises:

an age determiner configured for determining the age of the user based on the facial part of the image;

a gender determiner configured for determining the gender of the user based on at least one of the facial part of the image and the at least one non-facial part of the image; and

a product selector configured for selecting the first group of products based on the age and the gender of the user.

24. The system of claim 23, wherein the product selector comprises:

a product extractor configured for determining at least a subgroup of products from a purchase history database, the subgroup of products being purchased by a specific group of customers comprising the one or more other users;

a score calculator configured for calculating a score for each product of the subgroup of products based at least in part on the age and gender of the user;

a product ranker configured for generating a ranking of the subgroup of products based on the score for each product of the subgroup of products; and

a product determiner configured for selecting the first group of products from the subgroup of products based on the ranking.

25. The system of claim 23, further comprising:

a product filter configured for;

removing one or more products from the first group of products, the one or more products being previously purchased by previous customers of all genders and at least a predetermined range of ages; and

outputting the group of products after removing the one or more products from the group of products.

26. The system of claim 24, wherein each of the specific group of customers has a same gender as the user and an age within a range of the age of the user.

27. The system of claim 22, wherein the body-based recommender comprises:

a feature extractor configured for determining the plurality of non-facial features for each of the at least one non-facial part of the image;

an attribute classifier configured for determining one or more attributes for at least a portion of the plurality of non-facial features for each of the at least one non-facial part of the image; and

a product matcher configured for;

selecting the second group of products from a product database based on the one or more attributes; and

outputting the second group of products.

28. The system of claim 27, wherein the feature extractor is a convolutional neural network.

29. The system of claim 21, further comprising:

a gaming unit configured for executing a game program so that the user can play a game via the recommendation device, wherein in response to the user selecting one or more products from the first group of products, the second group of products, or both the first group of products and the second group of products for purchase, a discount is provided to the user via the recommendation device upon determining that the user won the game, and wherein executing the game comprises;

generating a virtual representation of the user to be presented on the display, wherein in response to the user making a movement, the virtual representation also makes the movement which is presented on the display.