Computer-Vision Content Detection for Connecting Objects in Media to Users

• US 20130046823A1
• Filed: 08/18/2011
• Published: 02/21/2013
• Est. Priority Date: 08/18/2011
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Mitchell, Justin, Odio, Samuel, Garcia, David Harry
• Granted Patent: US 9,672,496 B2
• Time in Patent Office: Days
• US Class Current: 709/204
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 50/01 Social networking

Abstract

Methods, apparatuses and systems directed to detecting objects in user-uploaded multimedia such as photos and videos, determining the location at which the media was captured, inferring a set of users of a social network who were physically present at the time and place of capture, and pushing remarketing content to the set of inferred users for the detected objects, or alternatively, the competitors of the detected concepts.

Claims

1. A method comprising, by one or more computing systems:

receiving a multimedia object comprising one or more images from an uploading user, wherein the uploading user is represented by a node on a social network;

detecting one or more concepts associated with the multimedia object; and

inferring an exposure between one or more individuals associated with the multimedia object and the concept.

2. The method of claim 1, wherein inferring an exposure comprises:

determining a physical location associated with the multimedia object;

detecting users associated with the physical location,

calculating an exposure probability for each user and the concept; and

inferring an exposure between the user and the concept if the exposure probability is above a predetermined threshold.

3. The method of claim 2, further comprising pushing content to the users with inferred exposures for the detected one or more concepts.

4. The method of claim 2, further comprising pushing content to the users with inferred exposures for the competitors of the detected one or more concepts.

5. The method of claim 2, wherein the exposure probability increases if the user is determined to be within a predetermined spatial-temporal distance from the place and time the multimedia object was captured.

6. The method of claim 2, detecting users associated with the physical location comprising:

querying a social graph for users associated with a node representing the location.

7. The method of claim 6, users associated with a node representing the location comprising:

users who have checked-into the node within a predetermined duration.

8. The method of claim 6, users associated with a node representing the location comprising:

users who have RSVP'"'"'d into an event at the node.

9. The method of claim 6, users associated with a node representing the location comprising:

users who are tagged in a photo uploaded to the node hub.

10. The method of claim 1, detecting one or more concepts associated with the multimedia object comprising:

applying one or more image object recognition algorithms to the multimedia object;

comparing at least a portion of the multimedia object to a plurality of stored objects in a database, each of the plurality of stored objects associated with a concept;

based on the comparison, identifying one or more objects in the multimedia object associated with one or more concepts; and

attaching meta-data identifying the one or more concepts to the multimedia object.

11. The method of claim 2, determining a physical location associated with the multimedia object comprising querying a social graph for locations explicitly associated with the received multimedia object.

12. The method of claim 10, wherein the multimedia object comprises an image, and the plurality of stored objects comprises a plurality of images.

13. The method of claim 10, wherein the multimedia object comprises a video, and wherein comparing at least a portion of the multimedia object comprises:

extracting one or more frames from the video; and

comparing the extracted one or more frames to the plurality of stored multimedia objects in the database, wherein the plurality of stored multimedia objects comprises a plurality of stored images.

14. The method of claim 1, wherein the multimedia object comprises a video, and detecting a concept comprises:

extracting one or more audio waveforms from the video; and

comparing the extracted one or more waveforms to a plurality of stored objects, each object associated with a concept, in a database, wherein the plurality of stored multimedia objects comprises plurality of stored audio waveforms;

based on the comparison, identifying one or more objects in the multimedia object associated with one or more concepts; and

attaching meta-data identifying the one or more concepts to the multimedia object.

15. The method of claim 2, determining a physical location associated with the multimedia object comprising:

applying one or more object image recognition algorithms to the multimedia object;

comparing at least a portion of the multimedia object to set of a plurality of stored images in a database, each of the plurality of stored images associated with a physical location; and

based on the comparison, determining a physical location of the client.

16. A non-transitory, computer-readable media comprising instructions, the instructions operable, when executed by one or more computing devices, to:

receive a multimedia object comprising one or more images from an uploading user, wherein the uploading user is represented by a node on a social network;

detect one or more concepts associated with the multimedia object; and

infer an exposure between one or more individuals associated with the multimedia object and the concept.

17. The media of claim 16, wherein inferring an exposure comprises:

determining a physical location associated with the multimedia object;

detecting users associated with the physical location,

calculating an exposure probability for each user and the concept; and

inferring an exposure between the user and the concept if the exposure probability is above a predetermined threshold.

18. The media of claim 17, the instructions further operable, when executed, to push content to the users with inferred exposures for the detected one or more concepts.

19. The media of claim 17, the instructions further operable, when executed, to push content to the users with inferred exposures for the competitors of the detected one or more concepts.

20. The media of claim 17, wherein the exposure probability increases if the user is determined to be within a predetermined spatial-temporal distance from the place and time the multimedia object was captured.