Contrast-based image attention analysis framework

• US 7,400,761 B2
• Filed: 09/30/2003
• Issued: 07/15/2008
• Est. Priority Date: 09/30/2003
• Status: Expired due to Fees

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Hong-Jiang, Ma, Yu-Fei
• Primary Examiner(s): Mehta; Bhavesh
• Assistant Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US10/676,519
• Publication Number:
  US 20050069206A1
• Time in Patent Office: 1,750 Days
• Field of Search: None
• US Class Current: 382/162
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

Systems and methods for image attention analysis are described. In one aspect, image attention is modeled by preprocessing an image to generate a quantized set of image blocks. A contrast-based saliency map for modeling one-to-three levels of image attention is then generated from the quantized image blocks.

Claims

1. A method for modeling image attention, the method comprising:

preprocessing an image to generate a quantized set of image blocks; and

generating a contrast-based saliency map for modeling one-to-three levels of image attention from the quantized image blocks; and

performing a fuzzy growing operation to extract attended areas from the contrast-based saliency map, the fuzzy growing operation comprising;

partitioning the contrast-based saliency map into two mutually exclusive areas as a function of classes of pixels comprising attended and unattended pixel areas;

selecting seeds for the fuzzy growing operation according to a set of criteria such that a seed has a local maximum contrast with respect to other regional perception units and the seed belongs to an attended area;

grouping pixels in the contrast-based saliency map with gray levels that satisfy criteria that indicate attended as compared to unattended areas; and

iteratively growing the attended area by using grouped pixel as seeds in subsequent fuzzy growth operations until no candidates of the perception units can be grouped.

2. The method of claim 1, wherein preprocessing further comprises:

resizing the image such that an aspect ratio of the image is maintained;

transforming the image from a first color space to a second color space that is consistent with human perception; and

making color in texture areas of the image coarser.

3. The method of claim 1, wherein generating the contrast-based saliency map further comprises:

dividing the image subsequent to quantization into multiple perception units; and

calculating a respective contrast of color components for each perception unit; and

normalizing calculated contrasts to smooth the contrasts.

4. The method of claim 1, further comprising extracting attended points from the contrast-based saliency map.

5. The method of claim 1, further comprising extracting an attended area from the contrast-based saliency map.

6. The method of claim 1, further comprising extracting an attended view from the contrast-based saliency map.

7. A computer-readable medium storing computer-program instructions executable by a processor for modeling image attention, the computer-program instructions when executed by the processor performing operations comprising:

preprocessing an image to generate a quantized set of image blocks; and

generating a contrast-based saliency map for three-level contrast-based image attention analysis from the quantized image blocks; and

performing a fuzzy growing operation to extract attended areas from the contrast-based saliency map, the fuzzy growing operation comprising;

partitioning the contrast-based saliency map into two mutually exclusive areas as a function of classes of pixels comprising attended and unattended pixel areas;

selecting seeds for the fuzzy growing operation according to a set of criteria such that a seed has a local maximum contrast with respect to other regional perception units and the seed belongs to an attended area;

grouping pixels in the contrast-based saliency map with gray levels that satisfy criteria that indicate attended as compared to unattended areas; and

iteratively growing the attended area by using grouped pixel as seeds in subsequent fuzzy growth operations until no candidates of the perception units can be grouped.

8. A computer-readable medium storing computer-program instructions executable by a processor, the computer-program instructions when executed by the processor for modeling image attention by operations comprising:

generating a preprocessed image by;

resizing the image such that an aspect ratio of the image is maintained; and

if the image is not already in a color space that is consistent with human perception, transforming the image from a first color space to a second color space that is consistent with human perception;

quantizing the preprocessed image to generate quantized image perception units such that color in texture areas across the quantized image perception units are coarser as compared to the image;

generating a contrast-based saliency map from the quantized image blocks, the contrast-based saliency map comprising a respective contrast of color components for each perception unit; and

performing a fuzzy growing operation to extract attended areas from the contrast-based saliency map, the fuzzy growing operation comprising;

partitioning the contrast-based saliency map into two mutually exclusive areas as a function of classes of pixels comprising attended and unattended pixel areas;

selecting seeds for the fuzzy growing operation according to a set of criteria such that a seed has a local maximum contrast with respect to other regional perception units and the seed belongs to an attended area;

grouping pixels in the contrast-based saliency map with gray levels that satisfy criteria that indicate attended as compared to unattended areas; and

iteratively growing the attended area by using grouped pixel as seeds in subsequent fuzzy growth operations until no candidates of the perception units can be grouped.

9. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for extracting attended points from the contrast-based saliency map.

10. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for extracting an attended view from the contrast-based saliency map.

11. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for extracting an attended view from the contrast-based saliency map, the attended view being a rectangle V(C, W, H), where C denotes an attention center, and W and H are the width and height of rectangle respectively, the attention center being a centroid of the contrast-based saliency map.

12. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for determining a size of an attended view in the contrast-based saliency map, the size being related to a 1st order central moment of the contrast-based saliency map.

13. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for extracting an attended area from the contrast-based saliency map.

14. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for extracting attended areas from the contrast-based saliency map by performing the fuzzy growing operation on the contrast-based saliency map as a function of two classes of pixels to partition the contrast-based saliency map into two mutually exclusive areas, the two classes of pixels comprising attended and unattended pixel areas.

15. The computer-readable medium of claim 8, wherein the computer-program instructions further comprise instructions for representing the contrast-based saliency map as a fuzzy event in probability space to extract attended areas.

16. The computer-readable medium of claim 15, wherein the contrast-based saliency map 216 has L gray levels from g₀ to g_L-1 and the histogram of saliency map is h_k, k=0, . . . , L-1, and wherein the computer-program instructions further for representing the contrast-based saliency map as a fuzzy event in probability space further comprise instructions for:

modeling the contrast-based saliency map by a triplet (Ω

, k, P), where Ω

={g₀, g₁, . . . , g_L-1} and P is a probability measure of the occurrence of gray levels, i.e., Pr{g_k}=h_k/Σ

h_k;

denoting a membership function μ

_S(g_k) of a fuzzy set Sε

Ω

indicating a degree of properties comprising attended areas possessed by gray level g_k; and

representing the properties as a fuzzy event as follows;

$S=\underset{g_k\in <br><br>\mathrm{\Omega <br><br>}}{\overset{}{\sum <br><br>}}<br><br><br><br>{\mathrm{\mu <br><br>}}_S<br><br>\left(g_k\right)/g_k;<br><br>\mathrm{and}$

computing a probability of the fuzzy event by;

$P<br><br>\left(S\right)=\underset{k=0}{\overset{L-1}{\sum <br><br>}}<br><br><br><br>{\mathrm{\mu <br><br>}}_S<br><br>\left(g_k\right)<br><br>P_r<br><br>\left(g_k\right).$

17. A computing device for modeling image attention, the computing device comprising a processor coupled to a memory, the memory comprising computer computer-program instructions executable by the processor for:

quantizing a preprocessed image to generate quantized image perception units such that color in texture areas across the quantized image perception units are coarser as compared to the image, the preprocessed image being a resized version of the image with an original aspect ratio and in a color space consistent with human perception;

generating a contrast-based saliency map from the quantized image blocks, the contrast-based saliency map comprising a respective contrast of color components for each perception unit; and

performing a fuzzy growing operation to extract attended areas from the contrast-based saliency map, the fuzzy growing operation comprising;

partitioning the contrast-based saliency map into two mutually exclusive areas as a function of classes of pixels comprising attended and unattended pixel areas;

selecting seeds for the fuzzy growing operation according to a set of criteria such that a seed has a local maximum contrast with respect to other regional perception units and the seed belongs to an attended area;

grouping pixels in the contrast-based saliency map with gray levels that satisfy criteria that indicate attended as compared to unattended areas; and

iteratively growing the attended area by using grouped pixel as seeds in subsequent fuzzy growth operations until no candidates of the perception units can be grouped.

18. The computing device of claim 17, wherein the computer-program instructions further comprise instructions for extracting attended points from the contrast-based saliency map.

19. The computing device of claim 17, wherein the computer-program instructions further comprise instructions for extracting an attended area from the contrast-based saliency map.

20. The computing device of claim 17, wherein the computer-program instructions further comprise instructions for extracting an attended view from the contrast-based saliency map.

21. A computing device comprising:

means for preprocessing an image to generate a quantized set of image blocks;

means for generating a contrast-based saliency map for modeling three-levels of image attentions from the quantized image blocks; and

means for performing a fuzzy growing operation to extract attended areas from the contrast-based saliency map, the fuzzy growing operation comprising;

partitioning the contrast-based saliency map into two mutually exclusive areas as a function of classes of pixels comprising attended and unattended pixel areas;

selecting seeds for the fuzzy growing operation according to a set of criteria such that a seed has a local maximum contrast with respect to other regional perception units and the seed belongs to an attended area;

grouping pixels in the contrast-based saliency map with gray levels that satisfy criteria that indicate attended as compared to unattended areas; and

iteratively growing the attended area by using grouped pixel as seeds in subsequent fuzzy growth operations until no candidates of the perception units can be grouped.

22. The computing device of claim 21, wherein the means for generating the contrast-based saliency map further comprises:

means for dividing the image subsequent to quantization into multiple perception units; and

means for calculating a respective contrast of color components for each perception unit; and

means for normalizing calculated contrasts to smooth the contrasts.

23. The computing device of claim 21, further comprising means for extracting attended points from the contrast-based saliency map.

24. The computing device of claim 21, further comprising means for extracting an attended area from the contrast-based saliency map.

25. The computing device of claim 21, further comprising means for extracting an attended view from the contrast-based saliency map.