Computer-vision content detection for sponsored stories

• US 9,135,631 B2
• Filed: 08/18/2011
• Issued: 09/15/2015
• Est. Priority Date: 08/18/2011
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Mitchell, Justin, Odio, Samuel, Garcia, David Harry
• Primary Examiner(s): Werner, Brian P
• Application Number:
  US13/212,356
• Publication Number:
  US 20130044959A1
• Time in Patent Office: 1,489 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G06Q 30/02 Marketing; Price estimation or determination; Fundraising
• CPC Class Codes: G06Q 50/01 Social networking
• CPC Class Codes: H04L 67/52 specially adapted for the location of the user terminal
• CPC Class Codes: H04L 67/535 Tracking the activity of the user network monitoring arrangements H04L43/00; recording of computer activity G06F11/34

Abstract

Methods, apparatuses and systems directed to sponsored story generation from an photo upload in an organic activity stream in a social networking site. A social networking system may apply computer image algorithms to detect image objects in user-uploaded images and videos, and promote them as sponsored stories.

Claims

1. A method comprising:

receiving, by one or more computer systems, sponsor specifications designating one or more concept nodes in a social network and a temporal factor to apply to sponsored stories;

monitoring, by one or more computer systems, an activity stream for upload by a user of the social network of one or more multimedia objects comprising one or more image objects associated with one or more of the concept nodes, respectively;

upon finding a matching multimedia object, by one or more computer systems;

determining, based at least in part on one or more privacy settings of the social network set by the user, one or more other users of the social network who are allowed to view the multimedia objects; and

exporting the matching multimedia object to a sponsored-story system;

calculating, for one or more sponsored stories, each sponsored story comprising at least part of one or more of the image objects corresponding to one or more of the exported multimedia objects, a score for the sponsored story based at least in part on a time the respective exported matching multimedia object was uploaded to the activity stream by the user and the temporal factor designated by the sponsor specifications; and

sending, to a client system of one or more of the other users for display, one or more of the sponsored stories based at least in part on the calculated scores.

2. The method of claim 1, wherein monitoring the activity stream comprises:

applying, by one or more computer systems, one or more image recognition algorithms to the multimedia objects uploaded by the user;

comparing, by one or more computer systems, at least a portion of the multimedia objects to a plurality of stored images in a database, each of the plurality of stored images associated with one or more of the concept nodes;

based at least in part on the comparison, by one or more computer systems, identifying one or more of the image objects as being associated with one or more of the concept nodes; and

attaching, by one or more computer systems, meta-data identifying one or more of the associated concept nodes to the multimedia object.

3. The method of claim 2, wherein one or more of the multimedia objects comprises a video and one or more frames of the video are one or more of the image objects, and wherein comparing at least a portion of the multimedia objects comprises:

extracting, by one or more computer systems one or more of the frames from the video; and

comparing, by one or more computer systems, one or more of the extracted frames to one or more of the stored images.

4. The method of claim 1, wherein the activity stream is a news feed of actions of one or more users of the social network.

5. The method of claim 2, wherein the sponsor specifications include a visual specification, and

wherein the matching multimedia object is formatted into the one or more of the sponsored stories in accordance with the visual specification.

6. The method of claim 5, wherein the other users comprise all users of the social network connected to the uploading user.

7. The method of claim 5, wherein the other users comprise a portion of users of the social network connected to the uploading user.

8. The method of claim 7, wherein the portion of users is determined at least in part by an ad selection algorithm.

9. One or more non-transitory, computer-readable storage media embodying software that is operable when executed to:

receive sponsor specifications designating one or more concept nodes in a social network and a temporal factor to apply to sponsored stories;

monitor an activity stream for upload by a user of the social network of one or more multimedia objects comprising one or more image objects associated with one or more of the concept nodes, respectively;

upon finding a matching multimedia object;

determine, based at least in part on one or more privacy settings of the social network set by the user, one or more other users of the social network who are allowed to view the multimedia objects; and

export the matching multimedia object to a sponsored-story system;

calculate, for one or more sponsored stories, each sponsored story comprising at least part of one or more of the image objects corresponding to one or more of the exported multimedia objects, a score for the sponsored story based at least in part on a time the respective exported matching multimedia object was uploaded to the activity stream by the user and the temporal factor designated by the sponsor specifications; and

send, to a client system of one or more of the other users for display, one or more of the sponsored stories based at least in part on the calculated scores.

10. The media of claim 9, wherein the software is further operable when monitoring the activity stream to:

apply one or more image recognition algorithms to the multimedia objects uploaded by the user;

compare at least a portion of the multimedia objects to a plurality of stored images in a database, each of the plurality of stored images associated with one or more of the concept nodes;

based at least in part on the comparison, identify one or more of the image objects of the multimedia object as being associated with one or more of the concept nodes; and

attach meta-data identifying one or more of the associated concept nodes to the multimedia object.

11. The media of claim 10, wherein one or more of the multimedia objects comprises a video and one or more frames of the video are one or more of the image objects, and wherein the software is further operable when comparing at least a portion of the multimedia objects to:

extract one or more of the frames from the video; and

compare one or more of the extracted frames to one or more of the stored images.

12. The media of claim 9, wherein the activity stream is a news feed of actions of one or more users of the social network.

13. The media of claim 10, wherein the sponsor specifications include a visual specification, and

wherein the matching multimedia object is formatted into the one or more of the sponsored stories in accordance with the visual specification.

14. The media of claim 13, wherein the other users comprise all users of the social network connected to the uploading user.

15. The media of claim 13, wherein the other users comprise a portion of users of the social network connected to the uploading user.

16. The media of claim 15, wherein the portion of users is determined at least in part by an ad selection algorithm.

17. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

receive sponsor specifications designating one or more concept nodes in a social network and a temporal factor to apply to sponsored stories;

monitor an activity stream for upload by a user of the social network of one or more multimedia objects comprising one or more image objects associated with one or more of the concept nodes, respectively;

upon finding a matching multimedia object;

determine, based at least in part on one or more privacy settings of the social network set by the user, one or more other users of the social network who are allowed to view the multimedia objects; and

export the matching multimedia object to a sponsored-story system;

calculate, for one or more sponsored stories, each sponsored story comprising at least part of one or more of the image objects corresponding to one or more of the exported multimedia objects, a score for the sponsored story based at least in part on a time the respective exported matching multimedia object was uploaded to the activity stream by the user and the temporal factor designated by the sponsor specifications; and

send, to a client system of one or more of the other users for display, one or more of the sponsored stories based at least in part on the calculated scores.

18. The system of claim 17, wherein the processors are further operable when monitoring the activity stream to:

apply one or more image recognition algorithms to the multimedia objects uploaded by the user;

compare at least a portion of the multimedia objects to a plurality of stored images in a database, each of the plurality of stored images associated with one or more of the concept node nodes;

based at least in part on the comparison, identify one or more of the image objects of the multimedia object as being associated with one or more of the concept nodes; and

attach meta-data identifying one or more of the associated concept nodes to the multimedia object.

19. The system of claim 17, wherein the activity stream is a news feed of actions of one or more users of the social network.

20. The system of claim 17, wherein the sponsor specifications include a visual specification, and

wherein the matching multimedia object is formatted into one or more of the sponsored stories in accordance with the visual specification.