System and method for displaying contextual supplemental content based on image content

• US 9,324,006 B2
• Filed: 04/27/2015
• Issued: 04/26/2016
• Est. Priority Date: 07/29/2007
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Vanhoucke, Vincent O., Gokturk, Salih Burak, Shah, Munjal, Baumgartner, Julie
• Primary Examiner(s): Fitzpatrick, Atiba O
• Application Number:
  US14/697,190
• Publication Number:
  US 20150227813A1
• Time in Patent Office: 365 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06F 18/00 Pattern recognition
• CPC Class Codes: G06F 18/24 Classification techniques
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 30/0251 Targeted advertisements
• CPC Class Codes: G06T 7/00 Image analysis

Abstract

An image-based content item is analyzed to determine one or more interests of a viewer of the content item. The analysis may include performing image analysis on the content item to determine geographic information that is relevant to an image of the content item. The one or more interests may be determined based on an assumption or probabilistic conclusion about a subject of the content item. Further, the one or more interests may be determined by applying one or more rules that utilize the geographic information. For some embodiments, a supplemental content item may be provided to the viewer based on the one or more interests.

Claims

1. A method performed by data processing apparatus, the method comprising:

identifying, in a set of images, two or more people that are each depicted in one or more images in the set of images;

determining, based on content depicted in the images in the set, a relationship between the two or more people;

determining, based at least on the determined relationship, one or more interests of a user viewing the set of images;

selecting, for the user, a content item based on the one or more interests; and

providing, to a user device of the user, data that cause presentation of the content item at the user device.

2. The method of claim 1, further comprising determining an event at which at least one image of the set of images was captured, wherein the content item is selected further based on the determined event.

3. The method of claim 2, wherein determining the event comprises determining a type of location at which the at least one image was captured based on visual characteristics of the at least one image.

4. The method of claim 1, wherein:

identifying the two or more people that are each depicted in the one or more images comprises identifying a baby in multiple images of the set of images;

determining the relationship between the two or more people comprises determining that at least one of the two or more people is a parent of the baby; and

determining the one or more interests of the user comprises determining that the user is interested in baby related content.

5. The method of claim 1, wherein:

identifying the two or more people that are each depicted in the one or more images comprises identifying a group of people that have similar ages and that are depicted in the one or more images; and

determining the relationship between the two or more people comprises determining that the group of people includes two or more friends based on the group of people having similar ages.

6. The method of claim 1, further comprising identifying an object that is depicted by two or more images in the set of images, wherein the content item is selected further based on the identified object.

7. The method of claim 1, wherein the set of images is a private set of images of the user or a person depicted in the set of images.

8. A system comprising:

a data processing apparatus; and

a computer storage medium encoded with a computer program, the program comprising data processing apparatus instructions that when executed by the data processing apparatus cause the data processing apparatus to perform operations comprising;

identifying, in a set of images, two or more people that are each depicted in one or more images in the set of images;

determining, based on content depicted in the images in the set, a relationship between the two or more people;

determining, based at least on the determined relationship, one or more interests of a user viewing the set of images;

selecting, for the user, a content item based on the one or more interests; and

providing, to a user device of the user, data that cause presentation of the content item at the user device.

9. The system of claim 8, wherein the operations further comprise determining an event at which at least one image of the set of images was captured, wherein the content item is selected further based on the determined event.

10. The system of claim 9, wherein determining the event comprises determining a type of location at which the at least one image was captured based on visual characteristics of the at least one image.

11. The system of claim 8, wherein:

identifying the two or more people that are each depicted in the one or more images comprises identifying a baby in multiple images of the set of images;

determining the relationship between the two or more people comprises determining that at least one of the two or more people is a parent of the baby; and

determining the one or more interests of the user comprises determining that the user is interested in baby related content.

12. The system of claim 8, wherein:

identifying the two or more people that are each depicted in the one or more images comprises identifying a group of people that have similar ages and that are depicted in the one or more images; and

determining the relationship between the two or more people comprises determining that the group of people includes two or more friends based on the group of people having similar ages.

13. The system of claim 8, wherein the operations further comprise identifying an object that is depicted by two or more images in the set of images, wherein the content item is selected further based on the identified object.

14. The system of claim 8, wherein the set of images is a private set of images of the user or a person depicted in the set of images.

15. A non-transitory computer storage medium encoded with a computer program, the program comprising instructions that when executed by one or more data processing apparatus cause the data processing apparatus to perform operations comprising:

identifying, in a set of images, two or more people that are each depicted in one or more images in the set of images;

determining, based on content depicted in the images in the set, a relationship between the two or more people;

determining, based at least on the determined relationship, one or more interests of a user viewing the set of images;

selecting, for the user, a content item based on the one or more interests; and

providing, to a user device of the user, data that cause presentation of the content item at the user device.

16. The non-transitory computer storage medium of claim 15, wherein the operations further comprise determining an event at which at least one image of the set of images was captured, wherein the content item is selected further based on the determined event.

17. The non-transitory computer storage medium of claim 16, wherein determining the event comprises determining a type of location at which the at least one image was captured based on visual characteristics of the at least one image.

18. The non-transitory computer storage medium of claim 15, wherein:

identifying the two or more people that are each depicted in the one or more images comprises identifying a baby in multiple images of the set of images;

determining the relationship between the two or more people comprises determining that at least one of the two or more people is a parent of the baby; and

determining the one or more interests of the user comprises determining that the user is interested in baby related content.

19. The non-transitory computer storage medium of claim 15, wherein:

identifying the two or more people that are each depicted in the one or more images comprises identifying a group of people that have similar ages and that are depicted in the one or more images; and

determining the relationship between the two or more people comprises determining that the group of people includes two or more friends based on the group of people having similar ages.

20. The non-transitory computer storage medium of claim 15, wherein the operations further comprise identifying an object that is depicted by two or more images in the set of images, wherein the content item is selected further based on the identified object.