Photo and video search

• US 9,727,565 B2
• Filed: 05/29/2014
• Issued: 08/08/2017
• Est. Priority Date: 06/03/2013
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Yahoo Holdings, Inc. (Verizon Communications Inc.)
• Inventors: Nguyen, Huy X., Pesavento, Gerry
• Primary Examiner(s): MOYER, ANDREW M
• Application Number:
  US14/290,214
• Publication Number:
  US 20140355907A1
• Time in Patent Office: 1,167 Days
• Field of Search: 382305, 382306
• CPC Class Codes: G06F 16/48 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/487 using geographical or spatial information, e.g. location
• CPC Class Codes: G06F 16/489 using time information
• CPC Class Codes: G06F 16/583 using metadata automatically derived from the content
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/78 Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
• CPC Class Codes: G06F 16/7837 using objects detected or recognised in the video content
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 2201/10 Recognition assisted with metadata

Abstract

In one embodiment, a set of tags that has been generated by performing computer vision analysis of image content of a visual media item may be obtained, where each tag of the set of tags has a corresponding probability. In addition, a set of information that is independent from the image content of the visual media item may be obtained. The probability of at least a portion of the set of tags may be modified based, at least in part, upon the set of information.

Claims

1. A method, comprising:

ascertaining, from a database, a set of tags that has been generated by performing computer vision analysis of image content of a visual media item, each tag of the set of tags having a corresponding probability, wherein the probability indicates a likelihood that the tag accurately describes the image content of the visual media item;

obtaining, by a processor, a set of information that is independent from the image content of the visual media item, wherein obtaining the set of information includes performing a look up in a calendar to obtain at least a portion of the set of information using at least one of a date or time at which the visual media item was captured via a mobile device, wherein the portion of the set of information identifies an event that occurred on at least one of the date or time at which the visual media item was captured via the mobile device; and

modifying in the database the probability of at least a portion of the set of tags based, at least in part, upon the set of information, wherein modifying the probability of at least a portion of the set of tags includes increasing or decreasing the probability of a tag in the set of tags based, at least in part, upon whether the event corresponds to the tag.

2. The method as recited in claim 1, further comprising:

capturing the visual media item by a mobile device;

detecting, via one or more sensors of the mobile device, an identity of an object or individual in a vicinity of the mobile device;

wherein the set of information includes an indication of the identity of the object or individual in the vicinity of the mobile device at the time that the mobile device captured the visual media item.

3. The method as recited in claim 1, wherein the set of information comprises an indication of an identity of an object or individual in a vicinity of a mobile device, wherein the identity of the object or individual is detected by the mobile device at the time that the mobile device captured the visual media item.

4. The method as recited in claim 1, further comprising:

performing similarity analysis with respect to the visual media item and other visual media items to identify a set of visual media items that are similar to the visual media item;

wherein obtaining a set of information that is independent from the image content of the visual media item includes ascertaining a set of tags associated with each similar visual media item in the set of media items, each tag of the set of tags associated with each similar visual media item having a corresponding probability.

5. The method as recited in claim 1, further comprising:

ascertaining a date or time at which the visual media item was captured;

wherein the at least a portion of the set of information comprises a set of tags associated with at least one other visual media item captured within a threshold period of time from the date or time at which the visual media item was captured.

6. The method as recited in claim 1, wherein the set of information comprises a comment that has been posted with respect to the visual media item.

17. The method as recited in claim 1, further comprising:

receiving a search query;

identifying a plurality of visual media items that are relevant to the search query based, at least in part, upon probabilities associated with tags of the plurality of visual media items; and

providing the plurality of visual media items for display via a mobile device;

wherein the plurality of visual media items include the visual media item.

18. The method as recited in claim 1, further comprising:

identifying a plurality of visual media items stored on a mobile device;

organizing, by a processor of the mobile device, the plurality of visual media items in a plurality of folders based, at least in part, on tags associated with the plurality of visual media items and corresponding probabilities;

wherein the plurality of visual media items include the visual media item.

19. The method as recited in claim 1, wherein the calendar is associated with a particular user or is publicly available.

20. The method as recited in claim 3, wherein obtaining the set of information comprises detecting the identity of the object or individual via one or more sensors of a mobile device via which the visual media item was captured, wherein the sensors include a Radio Frequency Identifier (RFID) sensor or an Infra-red (IR) sensor.

21. The method as recited in claim 18, further comprising:

receiving a selection, via the mobile device, of one of the plurality of folders.

22. The method as recited in claim 1, wherein the visual media item is a photograph.

23. The method as recited in claim 1, further comprising:

capturing the visual media item by a mobile device;

detecting, via one or more sensors of the mobile device, an identity of an object in a vicinity of the mobile device;

wherein the set of information includes an indication of the identity of the object in the vicinity of the mobile device at the time that the mobile device captured the visual media item.

7. A system, comprising:

one or more processors; and

one or more memories, at least one of processors or the memories being adapted for;

ascertaining, from a database, a set of tags that has been generated by performing computer vision analysis of image content of a visual media item, each tag of the set of tags having a corresponding probability, wherein the probability indicates a likelihood that the tag accurately describes the image content of the visual media item;

obtaining a set of information that is independent from the image content of the visual media item, wherein obtaining the set of information includes performing a look up in a calendar to obtain at least a portion of the set of information using at least one of a date or time at which the visual media item was captured via a mobile device, wherein the portion of the set of information identifies an event that occurred on at least one of the date or time at which the visual media item was captured via the mobile device; and

modifying in the database, the probability of at least a portion of the set of tags based, at least in part, upon the set of information, wherein modifying the probability of at least a portion of the set of tags includes increasing or decreasing the probability of a tag in the set of tags based, at least in part, upon whether the event corresponds to the tag.

8. The system as recited in claim 7, wherein the set of information comprises:

an indication of an identity of an object or individual in a vicinity of a mobile device at a time that the mobile device captured the visual media item.

9. The system as recited in claim 8, wherein the system is the mobile device.

10. The system as recited in claim 7, at least one of the processors or the memories being configured for performing operations, further comprising:

performing similarity analysis with respect to the visual media item and other visual media items to identify a set of visual media items that are similar to the visual media item;

wherein obtaining a set of information that is independent from the image content of the visual media item includes ascertaining a set of tags associated with each similar visual media item in the set of media items, each tag of the set of tags associated with each similar visual media item having a corresponding probability.

11. The system as recited in claim 7, wherein the set of information comprises a comment that has been posted with respect to the visual media item.

12. A non-transitory computer-readable storage medium storing thereon computer-readable instructions for performing operations, comprising:

ascertaining, from a database a set of tags that has been generated by performing computer vision analysis of image content of a visual media item, each tag of the set of tags having a corresponding probability, wherein the probability indicates a likelihood that the tag accurately describes the image content of the visual media item;

obtaining a set of information that is independent from the image content of the visual media item, wherein obtaining the set of information includes performing a look up in a calendar to obtain at least a portion of the set of information using at least one of a date or time at which the visual media item was captured via a mobile device, wherein the portion of the set of information identifies an event that occurred on at least one of the date or time at which the visual media item was captured via the mobile device; and

modifying, in the database, the probability of at least a portion of the set of tags based, at least in part, upon the set of information, wherein modifying the probability of at least a portion of the set of tags includes increasing or decreasing the probability of a tag in the set of tags based, at least in part, upon whether the event corresponds to the tag.

13. The non-transitory computer-readable storage medium as recited in claim 12, further comprising:

detecting, via one or more sensors of a mobile device, an identity of an object or individual in a vicinity of the mobile device;

wherein the set of information includes an indication of the identity of the object or individual in the vicinity of the mobile device at a time that the mobile device captured the visual media item.

14. The non-transitory computer-readable storage medium as recited in claim 12, wherein the set of information comprises an indication of an object or individual in a vicinity of a mobile device at the time that the mobile device captured the visual media item.

15. The non-transitory computer-readable storage medium as recited in claim 12, further comprising:

performing similarity analysis with respect to the visual media item and other visual media items to identify a set of visual media items that are similar to the visual media item;

wherein obtaining a set of information that is independent from the image content of the visual media item includes ascertaining a set of tags associated with each similar visual media item in the set of media items, each tag of the set of tags associated with each similar visual media item having a corresponding probability.

16. The non-transitory computer-readable storage medium as recited in claim 12, wherein the set of information comprises a comment that has been posted with respect to the visual media item.