MEDIA OBJECT METADATA ENGINE CONFIGURED TO DETERMINE RELATIONSHIPS BETWEEN PERSONS

• US 20100177938A1
• Filed: 01/13/2009
• Published: 07/15/2010
• Est. Priority Date: 01/13/2009
• Status: Active Grant

• Current Assignee: Slack Technologies, Inc. (Salesforce.com, Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Higgins, Christopher, Davis, Marc E., O'Sullivan, Joseph, Martinez, Ronald, Paretti, Christopher T.
• Granted Patent: US 8,831,276 B2
• Time in Patent Office: Days
• US Class Current: 382/118
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/30 in albums, collections or shared content, e.g. social network photos or video

Abstract

A media object, such as an image file, a video file, or an audio file, is analyzed to determine relationships between persons associated with the media object, which may include persons captured in the media object and/or a person that captured the media object. A representation of a first person captured in a media object is detected. The media object is analyzed to determine at least one indicator of a relation between the first person and a second person associated with the media object. A relationship between the first person and the second person is predicted based at least on the determined at least one relation indicator. The media object may be monetized in various ways, such as by directing advertisements to persons associated with the media object, and/or to persons having social connections to the persons associated with the media object.

Claims

1. A computer-implemented method, comprising:

detecting a representation of a first person captured in a media object;

analyzing the media object to determine at least one indicator of a relation between the first person and a second person associated with the media object; and

predicting a relationship between the first person and the second person based at least on the determined at least one relation indicator.

2. The method of claim 1, wherein the media object includes an image, and a representation of the second person is captured in the image.

3. The method of claim 2, wherein said analyzing comprises:

analyzing the image to determine at least one of a distance between the first person and the second person in the image, a facial expression of the first person in the image, a facial expression of the second person in the image, an amount of contact between the first person and the second person in the image, a type of contact between the first person and the second person in the image, a body expression of the first person in the image, a body expression of the second person in the image, clothing worn by the first person in the image, clothing worn by the second person in the image, an activity of the first person in the image, an activity of the second person in the image, or a total number of persons in the image.

4. The method of claim 1, wherein the media object includes an image captured by the second person.

5. The method of claim 4, wherein said analyzing comprises:

analyzing the image to determine at least one of a distance between the first person and an image capturing device used by the second person to capture the image, a facial expression of the first person in the image, a body expression of the first person in the image, clothing worn by the first person in the image, an activity of the first person in the image, a portion of the first person visible in the image, or a total number of persons in the image.

6. The method of claim 1, wherein the media object includes an audio object, the audio object including recorded voice of the first person and recorded voice of the second person.

7. The method of claim 6, wherein said analyzing comprises:

analyzing the audio object to determine an attitude of the first person, an attitude of the second person, an activity of the first person, or an activity of the second person.

8. The method of claim 1, wherein said analyzing includes at least one of performing image recognition on the media object, performing facial recognition on the media object, or performing voice recognition on the media object.

9. The method of claim 1, wherein said predicting comprises:

predicting the relationship between the first person and the second person based at least on the determined at least one relation indicator and at least one of a time at which the media object was captured, a location at which the media object was captured, post-capture interaction data associated with the media object, or post-capture annotation of the media object.

10. The method of claim 1, further comprising:

associating data representative of the predicted relationship with the media object.

11. The method of claim 10, wherein said associating comprises:

instrumenting the media object with a contact link for at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

12. The method of claim 1, further comprising:

selecting an advertisement based at least partially on the predicted relationship; and

associating the advertisement with the media object.

13. The method of claim 1, further comprising:

selecting an advertisement based at least partially on the predicted relationship; and

providing the advertisement for display to at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

14. The method of claim 1, wherein said identifying comprises

detecting representations of a plurality of persons captured in a media object;

wherein said analyzing comprises

analyzing the media object to determine indicators of relations between the plurality of persons; and

wherein said predicting comprises

predicting relationships between the plurality of persons based at least on the determined relations indicators.

15. The method of claim 14, further comprising:

generating a social relations graph based on the predicted relationships that includes a node corresponding to each person of the plurality of persons.

16. The method of claim 15, further comprising:

comparing the generated social relations graph with a plurality of network-based social relations graphs to determine a matching network-based social relations graph; and

determining an identity of at least person of the plurality of persons from the determined matching network-based social relations graph.

17. A system, comprising:

a media object metadata engine that includes a human representation detector, a relation determiner, and a relationship predictor;

wherein the human representation detector is configured to detect a representation of a first person captured in a media object;

wherein the relation determiner is configured to analyze the media object to determine at least one indicator of a relation between the first person and a second person associated with the media object; and

wherein the relationship predictor is configured to predict a relationship between the first person and the second person based at least on the determined at least one relation indicator.

18. The system of claim 17, wherein the media object includes an image, and a representation of the second person is captured in the image.

19. The system of claim 18, wherein the relation determiner is configured to analyze the image to determine at least one of a distance between the first person and the second person in the image, a facial expression of the first person in the image, a facial expression of the second person in the image, an amount of contact between the first person and the second person in the image, a type of contact between the first person and the second person in the image, a body expression of the first person in the image, a body expression of the second person in the image, clothing worn by the first person in the image, clothing worn by the second person in the image, an activity of the first person in the image, an activity of the second person in the image, or a total number of persons in the image.

20. The system of claim 17, wherein the media object includes an image captured by the second person.

21. The system of claim 20, wherein the relation determiner is configured to analyze the image to determine at least one of a distance between the first person and an image capturing device used by the second person to capture the image, a facial expression of the first person in the image, a body expression of the first person in the image, clothing worn by the first person in the image, an activity of the first person in the image, a portion of the first person visible in the image, or a total number of persons in the image.

22. The system of claim 17, wherein the media object includes an audio object, the audio object including recorded voice of the first person and recorded voice of the second person.

23. The system of claim 22, wherein the relation determiner is configured to analyze the audio object to determine an attitude of the first person, an attitude of the second person, an activity of the first person, or an activity of the second person.

24. The system of claim 17, wherein the relation determiner is configured to perform at least one of image recognition, facial recognition, or voice recognition on the media object.

25. The system of claim 17, wherein the relationship predictor is configured to predict the relationship between the first person and the second person based at least on the determined at least one relation indicator and at least one of a time at which the media object was captured, a location at which the media object was captured, post-capture interaction data associated with the media object, or post-capture annotation of the media object.

26. The system of claim 17, wherein the media object metadata engine further includes a media object packager configured to associate data representative of the predicted relationship with the media object.

27. The system of claim 26, wherein the media object packager is configured to instrument the media object with a contact link for at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

28. The system of claim 26, further comprising:

a media object monetization engine configured to select an advertisement based at least partially on the predicted relationship;

wherein the media object packager is configured to associate the advertisement with the media object.

29. The system of claim 26, further comprising:

a media object monetization engine configured to select an advertisement based at least partially on the predicted relationship;

wherein the advertisement is provided for display to at least one of the first person, the second person, or a third person associated with at least one of the first person or the second person.

30. The system of claim 17, wherein the human representation detector is configured to detect representations of a plurality of persons captured in the media object;

wherein the relation determiner is configured to analyze the media object to determine indicators of relations between the plurality of persons;

wherein the relationship predictor is configured to predict relationships between the plurality of persons based at least on the determined relations indicators; and

wherein the relationship predictor includes a social relation graph generator configured to generate a social relations graph based on the predicted relationships that includes a node corresponding to each person of the plurality of persons.

31. The system of claim 30, wherein the relationship predictor further includes a social relation graph comparator configured to compare the generated social relations graph with a plurality of network-based social relations graphs to determine a matching network-based social relations graph;

and

wherein an identity of at least person of the plurality of persons is determined from the determined matching network-based social relations graph.

32. A computer program product comprising a computer-readable medium having computer program logic recorded thereon for enabling a processor to process a media object, comprising:

first means for enabling the processor to detect a representation of a first person captured in a media object;

second means for enabling the processor to analyze the media object to determine at least one indicator of a relation between the first person and a second person associated with the media object; and

third means for enabling the processor to predict a relationship between the first person and the second person based at least on the determined at least one relation indicator.