Computer-Vision Content Detection for Connecting Objects in Media to Users

• US 20170221158A1
• Filed: 04/14/2017
• Published: 08/03/2017
• Est. Priority Date: 08/18/2011
• Status: Active Grant

• Current Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Original Assignee: Meta Platforms, Inc. (f/k/a Facebook, Inc.)
• Inventors: Mitchell, Justin, Odio, Samuel, Garcia, David Harry
• Granted Patent: US 10,346,929 B2
• Time in Patent Office: Days
• CPC Class Codes: G06Q 10/107 Computer-aided management of electronic mailing [e-mailing]
• CPC Class Codes: G06Q 30/0201 Market modelling; Market analysis; Collecting market data
• CPC Class Codes: G06Q 30/0269 based on user profile or attribute
• CPC Class Codes: G06Q 50/01 Social networking

Abstract

Methods, apparatuses and systems directed to detecting objects in user-uploaded multimedia such as photos and videos, determining the location at which the media was captured, inferring a set of users of a social network who were physically present at the time and place of capture, and pushing remarketing content to the set of inferred users for the detected objects, or alternatively, the competitors of the detected concepts.

Claims

1-20. -20. (canceled)

21. A method comprising:

accessing, by one or more computer systems, a multimedia object received from a first user;

detecting, by one or more computer systems, one or more concepts associated with the multimedia object; and

identifying, by one or more computer systems, one or more second users as being exposed to the one or more concepts associated with the multimedia object, each second user being identified based at least in part on one or more social actions associated with the second user with respect to the multimedia object.

22. The method of claim 21, wherein identifying the second users who were exposed to the concept comprises:

determining, by one or more computer systems, a physical location that is associated with the multimedia object;

determining, by one or more computer systems, a time period during which the multimedia object was captured;

identifying the social actions, wherein the social actions are identified based at least in part on a calculated probability that the social actions occurred both at the physical location and within the time period; and

identifying, by one or more computer systems, the second users associated with the identified social actions.

23. The method of claim 22, wherein:

the first user is associated with an online social network; and

determining the physical location comprises querying the social network for one or more locations associated with the multimedia object.

24. The method of claim 22, wherein determining a physical location comprises:

applying, by one or more computer systems, one or more object image recognition algorithms to the multimedia object;

comparing, by one or more computer systems, at least a portion of the multimedia object to a plurality of stored images in a database, each of the plurality of stored images associated with one or more physical locations; and

based on the comparison, by one or more computer systems, determining the physical location.

25. The method of claim 21, further comprising pushing content associated with one or more of the detected concepts to one or more of the second users.

26. The method of claim 25, wherein:

at least one of the concepts is associated with a concept node of a social network;

the method further comprises determining, for the at least one concept node, an entity associated with that concept node; and

the content pushed to the second users is associated with a competitor of the entity.

27. The method of claim 21, wherein the social actions comprise one or more of:

a check-in to a location;

an RSVP for an event;

or

a tag in one or more other multimedia objects associated with an event.

28. The method of claim 21, wherein detecting one or more concepts associated with the multimedia object comprises:

applying, by one or more computer systems, one or more image object recognition algorithms to the multimedia object;

comparing, by one or more computer systems, at least a portion of the multimedia object to a plurality of stored objects in a database, each of the plurality of stored objects associated with one or more concepts;

based on the comparison, identifying, by one or more computer systems, one or more concepts associated with the multimedia object; and

attaching, by one or more computer systems, meta-data identifying one or more of the identified concepts to the multimedia object.

29. The method of claim 28, wherein:

the multimedia object comprises an image; and

the plurality of stored objects comprises a plurality of stored images.

30. The method of claim 28, wherein the multimedia object comprises a video and comparing at least a portion of the multimedia object comprises:

extracting, by one or more computer systems, one or more frames from the video; and

comparing, by one or more computer systems, one or more of the extracted frames to the plurality of stored objects, wherein the plurality of stored objects comprises a plurality of stored images.

31. The method of claim 21, wherein the multimedia object comprises a video and detecting one or more concepts comprises:

extracting, by one or more computer systems, one or more audio waveforms from the video; and

comparing, by one or more computer systems, one or more of the extracted waveforms to a plurality of stored objects in a database, each of the plurality of stored objects associated with one or more concepts, wherein the plurality of stored objects comprises a plurality of stored audio waveforms;

based on the comparison, by one or more computer systems, identifying one or more concepts associated with the multimedia object; and

attaching, by one or more computer systems, meta-data identifying one or more of the identified concepts to the multimedia object.

32. The method of claim 21, wherein:

at least one of the concepts is associated with a concept node of a social network;

the concept node corresponds to a brand, product, or company.

33. The method of claim 21, further comprising:

accessing a social graph associated with a social network, wherein the social graph comprises a plurality of nodes and a plurality of edges connecting the nodes, each connection between two nodes establishing a single degree of separation between the two nodes, wherein the plurality of nodes comprises;

a first user node corresponding to the first user;

a plurality of second user nodes, each representing one of the second users and each within a specified degree of separation from the first user node; and

a plurality of concept nodes, each representing one of the concepts.

34. The method of claim 33, wherein the specified degree of separation is one, two, three, or all.

35. One or more non-transitory computer-readable storage media embodying software that is operable when executed to:

access a multimedia object received from a first user;

detect one or more concepts associated with the multimedia object; and

identify one or more second users as being exposed to the one or more concepts associated with the multimedia object, each second user being identified based at least in part on one or more social actions associated with the second user with respect to the multimedia object.

36. The media of claim 35, wherein the software is further operable when identifying the second users who were exposed to the concept, to:

determine a physical location that is associated with the multimedia object;

determine a time period during which the multimedia object was captured;

identify the social actions, wherein the social actions are identified based at least in part on a calculated probability that the social actions occurred both at the physical location and within the time period; and

identify the second users associated with the identified social actions.

37. The media of claim 35, wherein the instructions are further operable when executed to push content associated with one or more of the detected concepts to one or more of the second users.

38. The media of claim 37, wherein:

at least one of the concepts is associated with a concept node of a social network;

the instructions are further operable when executed to determine, for the at least one concept node, an entity associated with that concept node; and

the content pushed to the second users is associated with a competitor of the entity.

39. The media of claim 35, wherein the social actions comprise one or more of:

a check-in to a location;

an RSVP for an event;

or

a tag in one or more other multimedia objects associated with an event.

40. A system comprising:

one or more processors; and

a memory coupled to the processors comprising instructions executable by the processors, the processors being operable when executing the instructions to;

access a multimedia object received from a first user;

detect one or more concepts associated with the multimedia object; and

identify one or more second users as being exposed to the one or more concepts associated with the multimedia object, each second user being identified based at least in part on one or more social actions associated with the second user with respect to the multimedia object.