Tagging images by determining a set of similar pre-tagged images and extracting prominent tags from that set

• US 8,411,953 B2
• Filed: 09/30/2008
• Issued: 04/02/2013
• Est. Priority Date: 09/30/2008
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Deluca, Lisa Seacat, Do, Lydia M.
• Primary Examiner(s): Johns, Andrew W
• Assistant Examiner(s): Dunphy, David F
• Application Number:
  US12/241,717
• Publication Number:
  US 20100080470A1
• Time in Patent Office: 1,645 Days
• Field of Search: None
• US Class Current: 382/181
• CPC Class Codes: G06V 20/10 Terrestrial scenes scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59
• CPC Class Codes: G06V 20/70 Labelling scene content, e.g. deriving syntactic or semantic representations

Abstract

An image can be compared with a set of images, each including pre-existing tags. A similar image set can be determined from results of the comparing. Pre-existing tags can be extracted from the similar image set. Prominent tags can be determined from the extracted pre-existing tags. At least one of the determined prominent tags can be added to a tag set associated with the image.

Claims

1. A method for adding tags to images comprising:

identifying an image-to-be-tagged;

determining an initial tag list comprising initial tags associated with the image-to-be-tagged;

determining a similar image set of images comprising tags extracting pre-existing tags from the similar image set to form a set of combined tags;

determining a set of prominent tags, wherein the set of prominent tags comprises tags of the set of combined tags having a prominence score greater than a threshold score; and

adjusting the initial tag list by,

moving a prominent tag to the beginning of the initial tag list when the prominent tag is in the initial tag list, and

appending a prominent tag to the end of the initial tag list, when the prominent tag is not in the initial tag list.

2. The method of claim 1, further comprising:

using photogrammetry, determining the similar set of images from a larger set of images stored in an image repository;

extracting pre-existing tags from the similar image set to form a set of combined tags;

computing prominence scores for each tag of the combined set of tags, wherein the prominence score is based at least in part on persistence, meaning a number of times a term or tag appears within different images of the set of similar images;

ordering the tags of the combined set of tags by prominence score;

determining a threshold score for adding tags to the image-to-be tagged; and

adding each of the combined tags having a prominence score greater than the threshold score, which together are the determined set of tags, to a tag set of the image-to-be-tagged.

3. The method of claim 1, further comprising:

mapping a plurality of images, each of which is a two dimensional image, into a three dimensional space, wherein the mapping considers camera viewpoints of the images; and

determining the similar image set, which is a subset of the plurality of images based upon which of the images share a geographic mapping in the three dimensional space with the image-to-be-tagged.

4. The method of claim 1, wherein the set of prominent tags is added to the initial tag list when the initial tag list is empty.

5. The method of claim 4, further comprising:

before the adding the tags to a previously untagged image, restricting the tags to be added to a semantic topic area.

6. The method of claim 1, further comprising:

performing the determining of the set of tags, and the adding of the determined prominent set of tags as a software service.

7. The method of claim 1, wherein synonyms between tags extracted from different ones of the images of the similar image set is considered when computing the prominence scores for each tag.

8. The method of claim 1, further comprising:

applying additional weights to the prominence scores of the combined tags for tags referring to geographical locations or specific proper nouns.

9. The method of claim 1, wherein as a quantity of images increases in the similar image set, and as a quantity of tags of the set of combined tags increases, a corresponding increase occurs in an accuracy of automatically added tags to the image-to-be-tagged.

10. A computer program product for adding tags to images comprising:

a non-transitory computer usable storage medium having computer usable program code embodied therewith, the computer usable program code comprising;

computer usable program code configured to identify an image-to-be-tagged;

computer usable program code configured to determine an initial tag list comprising initial tags associated with the image-to-be-tagged;

computer usable program code configured to, using photogrammetry, determine a similar image set of images from a larger set of images stored in an image repository, wherein each of the images in the similar image set are tagged with one or more textual tags;

computer usable program code configured to extract pre-existing tags from the similar image set to form a set of combined tags;

computer usable program code configured to compute prominence scores for each tag of the combined set of tags, wherein the prominence score is based at least in part on persistence, meaning a number of times a term or tag appears within different images of the set of similar images;

computer usable program code configured to determine a set of prominent tags, wherein the set of prominent tags comprises tags of the set of combined tags having a prominence score greater than a threshold score; and

computer usable program code configured to adjust the initial tag list by,

moving a prominent tag to the beginning of the initial tag list when the prominent tag is in the initial tag list, and

appending a prominent tag to the end of the initial tag list, when the prominent tag is not in the initial tag list.

11. The computer program product of claim 10, wherein the photogrammetry technique is a photo tourism technique, wherein each of the images in the similar image set comprise multiple different objects within their images.

12. The computer program product of claim 10, further comprising:

computer usable program code configured to map the images being compared, each of which is a two dimensional image, into a three dimensional space, wherein the mapping considers camera viewpoints of the images; and

computer usable program code configured to determine a set of similar images based upon which of the images share a geographic mapping in the three dimensional space with the image to which the prominent tags are added.

13. The computer program product of claim 10, wherein the set of prominent tags is added to the initial tag list when the initial tag list is empty.

14. The computer program product of claim 10, further comprising:

computer usable program code configured to perform the comparing of the images, the determining of the similar image set, the extracting of pre-existing tags, the determining of prominent tags, and the adding of the determined prominent tags as a software service.

15. A method comprising:

identifying an image-to-be-tagged;

accessing an image repository, wherein the image repository includes a set of images each having a set of associated tags;

comparing the image-to-be-tagged with the set of images in the image repository;

identifying one or more images from the set of images as being similar to the image-to-be-tagged based on the comparison;

determining a set of tags associated with the set of images identified as being similar to the image-to-be-tagged;

assigning prominence scores to the set of tags to determine a set of prominent tags; and

updating a list of tags associated with the image-to-be-tagged with the set of prominent tags, wherein the updating comprises adjusting the list of tags by,

moving a prominent tag of the set of prominent tags to the beginning of the list of tags when the prominent tag is in the list of tags, and

appending a prominent tag of the set of prominent tags to the end of the list of tags when the prominent tag is not in the list of tags.