Tag association with image regions

• US 9,047,319 B2
• Filed: 12/17/2010
• Issued: 06/02/2015
• Est. Priority Date: 12/17/2010
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Inventors: Hua, Xian-Sheng, Yang, Kuiyuan, Wang, Meng, Zhang, Hong-Jiang
• Primary Examiner(s): Waldron, Scott A
• Application Number:
  US12/972,203
• Publication Number:
  US 20120158721A1
• Time in Patent Office: 1,628 Days
• Field of Search: 707/756, 382/170
• US Class Current: 1/1
• CPC Class Codes: G06F 16/5838 using colour
• CPC Class Codes: G06F 16/5866 using information manually generated, e.g. tags, keywords, comments, manually generated location and time information

Abstract

A computing device configured to determine that one or more regions of an image are associated with a tag of the image is described herein. The computing device is further configured to determine one or more attribute tags describing at least one of the content or context of the one or more regions. Upon determining the attribute tags, the computing device associates the attribute tags with the tag to enable image searching based on the tag and attribute tags.

Claims

1. A method comprising:

receiving, by a computing device, an image having one or more image tags, wherein each image tag is associated with the image as a whole;

identifying a plurality of regions of the image;

associating, by the computing device, a particular region of the plurality of regions with a particular image tag of the one or more image tags;

determining, by the computing device, that one or more attribute tags describe at least one of the content or context of the particular region based at least in part on determining that the one or more attribute tags are associated with a highest magnitude measurement of diverse density of the particular region; and

associating, by the computing device, the one or more attribute tags with the particular image tag to enable image searching based on the image tag and the attribute tags.

2. The method of claim 1, wherein associating the particular region with the particular image tag includes calculating, for combinations of the particular region with individual image tags of the one or more image tags, a measurement of diverse density in individual feature spaces of a plurality of feature spaces, wherein the plurality of feature spaces represent one or more low level features of the particular regions.

3. The method of claim 1, wherein associating the particular region with the particular image tag includes calculating, for each combinations of the particular region and individual image tags of the one or more image tags, a measurement of diverse density in individual feature spaces of a plurality of feature spaces, wherein the plurality of feature spaces represent one or more low level features of the particular region;

and further includes selecting, based at least in part on the plurality of feature spaces, a highest magnitude measurement of diverse density calculated for combinations of the particular region with individual image tags of the one or more image tags as a diverse density measurement for a combination of the particular region and the particular image tag of the one or more image tags.

4. The method of claim 3, further comprising associating the particular image tag with the particular region based at least in part on the measurement of diverse density for combinations of the particular region with the one or more image tags.

5. The method of claim 1, further comprising calculating, for combinations of the particular region and individual image tags of the one or more image tags, a measurement of diverse density in individual feature spaces of a plurality of feature spaces, wherein the plurality of feature spaces represent one or more low level features of the particular region;

and associating one or more other regions of the plurality of regions of the image with a default image tag in response to determining that a highest magnitude measurement of diverse density for the one or more other regions is below a threshold.

6. The method of claim 1, further comprising merging adjacent regions of the image into a single region of the image when the adjacent regions share an image tag.

7. The method of claim 1, further comprising

associating an image tag with each region of the plurality of regions.

8. The method of claim 1, wherein the attribute tags include a location tag for the particular region and the method further comprises determining as the location tag a position of the particular region relative to positions of one or more other regions of the plurality of regions of the image.

9. The method of claim 1, wherein the attribute tags include a size tag for the particular region and the method further comprises determining the size tag based on an area-ratio of the particular region.

10. The method of claim 1, wherein the attribute tags include a dominance tag for the particular region and the method further comprises determining the dominance tag based at least in part on a measurement of diverse density of the particular region and an area-ratio of the particular region.

11. The method of claim 1, wherein the attribute tags include a color tag for the particular region and the method further comprises determining the color tag based at least in part on diverse density measurements for the particular region in a color space.

12. The method of claim 1, wherein the attribute tags include a texture tag for the particular region and the method further comprises determining the texture tag based at least in part on diverse density measurements for the particular region in a texture space.

13. The method of claim 1, wherein the attribute tags include a shape tag for the particular region and the method further comprises determining the shape tag based at least in part on diverse density measurements for the particular region in a shape space.

14. The method of claim 1, further comprising setting a particular attribute tag of the one or more attribute tags to a default value in response to determining based on thresholding that the particular attribute tag is not relevant.

15. A computer-readable physical memory device comprising executable instructions stored on the computer-readable memory device and configured to program a computing device to perform operations including:

identifying an image having a plurality of image tags, wherein each image tag is associated with the image as a whole;

identifying a plurality of regions of the image;

calculating, for each combination of a region of the image with one of the plurality of image tags, a measurement of diverse density in each of a plurality of feature spaces, wherein the plurality of feature spaces represent one or more low level features of the region;

selecting a highest magnitude measurement of diverse density calculated for each combination of the region and one of the plurality of image tags as a diverse density measurement for the combination of the region and the one of the plurality of image tags; and

associating one of the plurality of image tags with the region based at least in part on the diverse density measurements for the combinations of the region with the plurality of image tags.

16. The computer-readable physical memory device of claim 15, wherein the plurality of feature spaces include a color space, a texture space, and a shape space.

17. The computer-readable physical memory device of claim 15, wherein associating one of the plurality of image tags with the region comprises:

selecting an image tag that corresponds to the combination of the region and the image tag having the highest magnitude measurement of diverse density; and

associating the image tag that is selected with the region.

18. A system comprising:

a processor;

a tag-to-region module comprising a plurality of instructions configured to be executed by the processor to perform operations including;

identifying an image having one or more image tags, wherein at least one of the one or more image tags is associated with the image as a whole;

identifying a plurality of regions of the image; and

associating a particular region of the image with a particular image tag of the one or more image tags based at least in part on metrics for combinations of particular regions and image tags of the one or more image tags; and

an attribute tag module comprising a plurality of instructions configured to be executed by the processor to perform operations including;

determining one or more attribute tags describing at least one of the content or context of the particular region associated with the image tag, the attribute tags being determined to be at least in part associated with a highest magnitude measurement of diverse density of the particular region; and

associating the one or more attribute tags with the image tag to enable image searching based on the image tag and the attribute tags.

19. The system of claim 18, wherein the attribute tags further include one or more of a motion tag, a view tag, or an affection tag.

20. The system of claim 18, wherein the attribute tags include at least one of a location tag, a size tag, a dominance tag, a color tag, a texture tag, or a shape tag.