Textual attribute-based image categorization and search

• US 8,503,767 B2
• Filed: 09/16/2009
• Issued: 08/06/2013
• Est. Priority Date: 09/16/2009
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Jian, Tao, Litian, Yuan, Lu
• Primary Examiner(s): Wang, Claire X
• Assistant Examiner(s): Thirugnanam, Gandhi
• Application Number:
  US12/560,658
• Publication Number:
  US 20110064301A1
• Time in Patent Office: 1,420 Days
• Field of Search: 382/159
• US Class Current: 382/159
• CPC Class Codes: G06F 3/0482 Interaction with lists of selectable items, e.g. menus
• CPC Class Codes: G06V 10/422 for representing the structure of the pattern or shape of an object therefor

Abstract

Techniques and systems for providing textual attribute-based image categorization and search are disclosed herein. In some aspects, images may be analyzed to identify a category of an image, or portion thereof. Additional textual attributes may be identified and associated with the image. In various aspects, the categories may be types of sky sceneries. Categorized images may be searched based on the categories and/or attributes. In further aspects, a user interface may provide an intuitive arrangement of the images for user navigation and selection. The user interface may also provide a simplified presentation and search of the categorized images. Images selected from user interface may be used to replace or modify features of an existing target image.

Claims

1. A method of analyzing an image, the method comprising:

identifying a selected portion of the image as a focus portion to be categorized within one of a plurality of textual categories;

generating a score for patches of the image by comparing each patch to a codeword from a codebook, the patches being subdivisions of the image;

associating a category of the focus portion with the image as one of the plurality of textual categories based on the score;

determining a layout of the image based on at least one of a size or position of the focus portion of the image; and

storing the image in a computer readable storage medium, the image stored with metadata that includes the category and layout.

2. The method as recited in claim 1, wherein the focus portion is a sky region of the image.

3. The method as recited in claim 2, wherein the plurality of textual categories include at least a blue sky category, a cloudy sky category, and a sunset category.

4. The method as recited in claim 1, wherein the determining the layout of the image includes determining a horizon by moving a horizontal line upwards from a bottom of the image until a number of focus portion pixels below the horizontal line is greater than a predetermined percent.

5. The method as recited in claim 4, wherein the layout is based on an existence of the horizon and further based upon a percentage of the pixels of the focus portion with respect to a number of pixels in the image when the horizon exists in the image.

6. The method as recited in claim 1, further comprising detecting an existence of a sun in the image and determining a location of the sun when the sun is detected in the image.

7. The method as recited in claim 1, further comprising computing a richness factor for the focus portion based on a number of edges identified in the focus portion.

8. An image analysis system, comprising:

one or more processors; and

memory to store computer readable instructions executable by the one or more processors, the instructions operable to;

identify a focus portion of an image to be categorized within one of a plurality of textual categories;

segment the image into a plurality of patches, the patches being subdivisions of the image;

determine a score for each patch of the plurality of patches by comparing each patch to a codeword from a codebook;

associate a category of the focus portion with the image as one of the plurality of textual categories based on the score;

determine a layout of the image based at least in part on a percentage of pixels of the focus portion with respect to a number of pixels in the image; and

store the image in the memory with metadata that includes the category and the layout.

9. The system as recited in claim 8, wherein the focus portion is a sky region of the image.

10. The system as recited in claim 9, wherein the plurality of textual categories include at least a blue sky category, a cloudy sky category, and a sunset category.

11. The system as recited in claim 8, wherein the determining the layout of the image includes determining a horizon by moving a horizontal line upwards from a bottom of the image until a number of focus portion pixels below the horizontal line is greater than a predetermined percent.

12. The system as recited in claim 11, wherein the layout is based on an existence of the horizon and further based upon a percentage of the pixels of the focus portion with respect to the number of pixels in the image when the horizon exists in the image.

13. The system as recited in claim 8, the instructions further operable to detect an existence of a sun in the image and determine a location of the sun when the sun is detected in the image.

14. The system as recited in claim 8, the instructions further operable to compute a richness factor for the focus portion based on a number of edges identified in the focus portion.

15. One or more memory devices storing computer-executable instructions that, when executed, cause one or more processors to perform acts comprising:

identifying a focus portion of an image to be categorized within one of a plurality of textual categories;

segmenting the image into a plurality of patches, the patches being subdivisions of the image;

determining a score for each patch of the plurality of patches by comparing each patch to a codeword from a codebook;

associating a category of the focus portion with the image as one of the plurality of textual categories based on the score;

determining a layout of the image based at least in part on a percentage of pixels of the focus portion with respect to a number of pixels in the image; and

storing the image in the one or more memory devices with metadata that includes the category and the layout.

16. The one or more memory devices as recited in claim 15, wherein the focus portion is a sky region of the image.

17. The one or more memory devices as recited in claim 16, wherein the plurality of textual categories include at least a blue sky category, a cloudy sky category, and a sunset category.

18. The one or more memory devices as recited in claim 15, wherein the determining the layout of the image includes determining a horizon by moving a horizontal line upwards from a bottom of the image until a number of focus portion pixels below the horizontal line is greater than a predetermined percent.

19. The one or more memory devices as recited in claim 18, wherein the layout is based on an existence of the horizon and further based upon a percentage of the pixels of the focus portion with respect to the number of pixels in the image when the horizon exists in the image.

20. The one or more memory devices as recited in claim 15, the acts further comprising detecting an existence of a sun in the image and determining a location of the sun when the sun is detected in the image.