Media object metadata engine configured to determine relationships between persons

• US 8,831,276 B2
• Filed: 01/13/2009
• Issued: 09/09/2014
• Est. Priority Date: 01/13/2009
• Status: Active Grant

• Current Assignee: Slack Technologies, Inc. (Salesforce.com, Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Martinez, Ronald, Higgins, Christopher, Davis, Marc E., O'Sullivan, Joseph, Paretti, Christopher T.
• Primary Examiner(s): AZARIAN, SEYED H
• Application Number:
  US12/353,055
• Publication Number:
  US 20100177938A1
• Time in Patent Office: 2,065 Days
• Field of Search: 382/100, 382/103, 382106-107, 382/155, 382/162, 382/168, 382/173, 382/181, 382190-194, 382/199, 382/206, 382/209, 382/219, 382/232, 382/254, 382/274, 382/276, 382/182, 382286-298, 382/305, 382/312, 707/10, 707/780, 455/556.1, 455/433, 705/14.53, 705/10, 396/56, 709/207
• US Class Current: 382/103
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video

Abstract

A media object, such as an image file, a video file, or an audio file, is analyzed to determine relationships between persons associated with the media object, which may include persons captured in the media object and/or a person that captured the media object. A representation of a first person captured in a media object is detected. The media object is analyzed to determine at least one indicator of a relation between the first person and a second person associated with the media object. A relationship between the first person and the second person is predicted based at least on the determined at least one relation indicator. The media object may be monetized in various ways, such as by directing advertisements to persons associated with the media object, and/or to persons having social connections to the persons associated with the media object.

Claims

1. A computer-implemented method, comprising:

detecting, via a computing device, a representation of a first person captured in a media object that includes an image;

analyzing, via the computing device, the media object to determine at least one indicator of a relation between the first person captured in the media object and a second person associated with the media object, said at least one indicator of relation based, in part, upon analysis of the representation of the first person in the media object, said at least one indicator of relation comprising a factor of closeness representing a distance between the first person and second person, said second person not captured in the media object; and

predicting, via the computing device, a relationship between the first person and the second person based at least on the determined at least one relation indicator.

2. The method of claim 1, wherein said analyzing comprises:

analyzing the image to determine at least one of the distance between the first person and an image capturing device used by the second person to capture the image, a facial expression of the first person in the image, a body expression of the first person in the image, clothing worn by the first person in the image, an activity of the first person in the image, a portion of the first person visible in the image, or a total number of persons in the image.

3. The method of claim 1, wherein said analyzing includes at least one of performing image recognition on the media object, or performing facial recognition on the media object.

4. The method of claim 1, wherein said predicting comprises:

predicting the relationship between the first person and the second person based at least on the determined at least one relation indicator and at least one of a time at which the media object was captured, a location at which the media object was captured, post-capture interaction data associated with the media object, or post-capture annotation of the media object.

5. The method of claim 1, further comprising:

associating data representative of the predicted relationship with the media object.

6. The method of claim 5, wherein said associating comprises:

instrumenting the media object with a contact link for at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

7. The method of claim 1, further comprising:

selecting an advertisement based at least partially on the predicted relationship; and

associating the advertisement with the media object.

8. The method of claim 1, further comprising:

selecting an advertisement based at least partially on the predicted relationship; and

providing the advertisement for display to at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

9. The method of claim 1, wherein said identifying comprises detecting representations of a plurality of persons captured in a media object;

wherein said analyzing comprises analyzing the media object to determine indicators of relations between the plurality of persons; and

wherein said predicting comprises predicting relationships between the plurality of persons based at least on the determined relations indicators.

10. The method of claim 9, further comprising:

generating a social relations graph based on the predicted relationships that includes a node corresponding to each person of the plurality of persons.

11. The method of claim 10, further comprising:

comparing the generated social relations graph with a plurality of network-based social relations graphs to determine a matching network-based social relations graph; and

determining an identity of at least one person of the plurality of persons from the determined matching network-based social relations graph.

12. A system, comprising:

at least one processor;

a media object metadata engine implemented by said at least one processor, said media object metadata engine comprising a human representation detector, a relation determiner, and a relationship predictor;

wherein the human representation detector is configured to detect a representation of a first person captured in a media object, the media object being an audio object;

wherein the relation determiner is configured to analyze the media object to determine at least one indicator of a relation between the first person captured in the media object and a second person associated with the media object, said at least one indicator of relation based, in part, upon analysis of the representation of the first person in the media object, said at least one indicator of relation comprising a factor of closeness representing a relationship distance between the first person and second person, said second person not captured in the media object; and

wherein the relationship predictor is configured to predict a relationship between the first person and the second person based at least on the determined at least one relation indicator.

13. The system of claim 12, wherein the audio object includes recorded voice of the first person and recorded voice of the second person.

14. The system of claim 13, wherein the relation determiner is configured to analyze the audio object to determine an attitude of the first person, an attitude of the second person, an activity of the first person, or an activity of the second person.

15. The system of claim 12, wherein the relation determiner is configured to perform voice recognition on the media object.

16. The system of claim 12, wherein the relationship predictor is configured to predict the relationship between the first person and the second person based at least on the determined at least one relation indicator and at least one of a time at which the media object was captured, a location at which the media object was captured, post-capture interaction data associated with the media object, or post-capture annotation of the media object.

17. The system of claim 12, wherein the media object metadata engine further includes a media object packager configured to associate data representative of the predicted relationship with the media object.

18. The system of claim 17, wherein the media object packager is configured to instrument the media object with a contact link for at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

19. The system of claim 17, further comprising:

a media object monetization engine configured to select an advertisement based at least partially on the predicted relationship;

wherein the media object packager is configured to associate the advertisement with the media object.

20. The system of claim 17, further comprising:

a media object monetization engine configured to select an advertisement based at least partially on the predicted relationship;

wherein the advertisement is provided for display to at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

21. The system of claim 12, wherein the human representation detector is configured to detect representations of a plurality of persons captured in the media object;

wherein the relation determiner is configured to analyze the media object to determine indicators of relations between the plurality of persons;

wherein the relationship predictor is configured to predict relationships between the plurality of persons based at least on the determined relations indicators; and

wherein the relationship predictor includes a social relation graph generator configured to generate a social relations graph based on the predicted relationships that includes a node corresponding to each person of the plurality of persons.

22. The system of claim 21, wherein the relationship predictor further includes a social relation graph comparator configured to compare the generated social relations graph with a plurality of network-based social relations graphs to determine a matching network-based social relations graph;

and

wherein an identity of at least person of the plurality of persons is determined from the determined matching network-based social relations graph.

23. A computer-readable storage medium having computer program logic tangibly recorded thereon for enabling a processor, upon execution of the computer-program logic, to process a media object, comprising:

computer instructions, upon execution by said processor, that detect a representation of a first person captured in a media object that includes an image;

computer, instructions, upon execution by said processor, that analyze the media object to determine at least one indicator of a relation between the first person captured in the media object and a second person associated with the media object, said at least one indicator of relation based, in part, upon analysis of the representation of the first person in the media object, said at least one indicator of relation comprising a factor of closeness representing a distance between the first person and second person, said second person not captured in the media object; and

computer instructions, upon execution by said processor, that predict a relationship between the first person and the second person based at least on the determined at least one relation indicator.