Monitoring an any-image labeling engine

• US 9,218,364 B1
• Filed: 01/28/2011
• Issued: 12/22/2015
• Est. Priority Date: 01/28/2011
• Status: Active Grant

• Current Assignee: Verizon Patent and Licensing Incorporated (Verizon Communications Inc.)
• Original Assignee: Yahoo! Inc. (Apollo Global Management, Inc.)
• Inventors: Garrigues, Pierre, Cadieu, Charles, Culpepper, Benjamin Jackson, Koepsell, Killian, Nguyen, Huy X., Stadlen, Andrew, Pesavento, Gerry, Warland, David
• Primary Examiner(s): KUDDUS, DANIEL A
• Application Number:
  US13/016,779
• Time in Patent Office: 1,789 Days
• Field of Search: 707/769, 707/915, 707/999.107, 707/772
• US Class Current: 1/1
• CPC Class Codes: G06F 16/50 of still image data
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
• CPC Class Codes: G06F 16/951 Indexing; Web crawling techniques

Abstract

A system for monitoring an image tagging system comprises a processor and a memory. The processor is configured to provide a user display for a type of monitoring of an image tagging system. The user display includes image tagging monitoring for: a) in the event that a computer vision match is found, a computer vision match and an associated computer vision tag and, b) in the event that no computer vision match is found, a human vision match and an associated human vision tag. A memory coupled to the processor and configured to provide the processor with instructions.

Claims

1. A system, comprising:

a processor; and

a memory, at least one of the processor or the memory being configured to;

tag an image using an image tagging system;

in an event that a computer vision analysis system of the image tagging system is able to identify the image, retrieving one or more computer vision tags associated with the image that have been obtained via computer vision analysis and providing the one or more computer vision tags via a user display for monitoring of the image tagging system; and

in an event that the computer vision analysis system of the image tagging system is not able to identify the image,

providing the image and a tagging guideline to the one or more human taggers; and

providing one or more human vision tags received from the one or more human taggers in association with the image via the user display for monitoring of the image tagging system, the one or more human taggers being in communication with a human computation module of the image tagging system; and

remove, by a garbage collection module, an object including an image from a database.

2. The system as in claim 1, further comprising a web interface, wherein the web interface is configured to receive an image query with the image.

3. The system as in claim 1, further comprising one or more databases, wherein the one or more databases comprises one or more of the following:

an instance image database, a class database, a model database, a 3D object database, an optical character database, a face database, a barcode database, a shape database, and a color database.

4. The system as in claim 1, further comprising a results processor, wherein the results processor selects a set of tags to present in response to an image query from a plurality of tags determined by a computer vision system or one or more human taggers.

5. The system as in claim 1, wherein the context comprises a user'"'"'s age, the user'"'"'s social information, the user'"'"'s location, the user'"'"'s employer, or the user'"'"'s primary information.

6. The system as in claim 1, wherein providing further comprises providing any computer vision tags that are found for the image to the one or more human taggers, wherein the one or more tags received from the one or more human taggers comprise one or more computer vision tags that have been verified by the one or more human taggers.

7. The system as in claim 1, at least one of the processor or the memory being further configured to:

provide information pertaining to an identity of the one or more human taggers in association with results of tagging images by the one or more human taggers.

8. The system as in claim 1, at least one of the processor or the memory being further configured to:

determine a validity of the tags received from the one or more human taggers using a status of each of the one or more human taggers from which the tags were received.

9. The system as in claim 1, at least one of the processor or the memory being further configured to:

check accuracy of the one or more human taggers by comparing the one or more tags received from the one or more human taggers against a set of known tags associated with the image.

10. A method, comprising:

obtaining an image;

determining, by a computer vision analysis system, that no computer vision matches are found for the image or the computer vision tags that are found for the image are inadequate;

after determining, by the computer vision analysis system, that no computer vision matches are found for the image or the computer vision tags that are found for the image are inadequate, providing the image and a tagging guideline to one or more human taggers;

receiving one or more tags associated with the image from the one or more human taggers;

storing the image in association with the one or more tags in one or more databases;

remove, by a garbage collection module, an object including the image from the databases.

11. The method as in claim 10, receiving an image query to provide tags for the image.

12. The method as in claim 10, further comprising:

providing the one or more human taggers with context information in association with the image.

13. The method as recited in claim 12, wherein the context information comprises a user query submitted in association with the image.

14. The method as in claim 10, wherein the one or more databases comprises one or more of the following:

an instance image database, a class database, a model database, a 3D object database, an optical character database, a face database, a barcode database, a shape database, and a color database.

15. The method as in claim 10, further comprising selecting a set of tags to present in response to an image query including the image from a plurality of tags determined for the image, and providing the set of tags in response to the image query.

16. The method as in claim 10, further comprising adding an image and one or more associated tags to a database.

17. The method as in claim 10, further comprising removing an object from a database, the object including an image.

18. The method as recited in claim 10, further comprising:

deleting images and associated tags from the one or more databases upon determining they have not been matched by subsequent image queries.

19. The method as recited in claim 10, further comprising:

determining a record of success of at least one of the one or more human taggers.

20. The method as recited in claim 19, further comprising:

providing the record of success of the at least one of the one or more human taggers.

21. The method as recited in claim 10, wherein providing comprises:

providing the image to a crowdsourcing network or a social network.

22. A system, comprising:

a processor; and

a memory, at least one of the processor or the memory being configured to;

analyze images by a computer vision analysis system and provide tags for the analyzed images;

remove, by a garbage collection module, an object including an image from a database;

communicate with one or more human taggers to obtain tags associated with images to be analyzed;

provide a display that presents results of tagging performed by the computer vision analysis system and the one or more human taggers; and

provide a tagger management screen that includes information pertaining to each of the human taggers;

wherein the information pertaining to each of the human taggers indicates whether the corresponding one of the human taggers is currently active.

23. The system as in claim 22, at least one of the processor or the memory being further configured to provide the images to be analyzed to the one or more human taggers via a crowdsourcing network or a social network.