Contrast-based image attention analysis framework

• US 20050069206A1
• Filed: 09/30/2003
• Published: 03/31/2005
• Est. Priority Date: 09/30/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Zhang, Hong-Jiang, Ma, Yu-Fei
• Granted Patent: US 7,400,761 B2
• Time in Patent Office: Days
• US Class Current: 382/190
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

Systems and methods for image attention analysis are described. In one aspect, image attention is modeled by preprocessing an image to generate a quantized set of image blocks. A contrast-based saliency map for modeling one-to-three levels of image attention is then generated from the quantized image blocks.

Claims

1. A method for modeling image attention, the method comprising:

preprocessing an image to generate a quantized set of image blocks; and

generating a contrast-based saliency map for modeling one-to-three levels of image attention from the quantized image blocks.

2. A method as recited in claim 1, wherein preprocessing further comprises:

resizing the image such that an aspect ratio of the image is maintained;

transforming the image from a first color space to a second color space that is consistent with human perception; and

making color in texture areas of the image coarser.

3. A method as recited in claim 1, wherein generating the contrast-based saliency map further comprises:

dividing the image subsequent to quantization into multiple perception units; and

calculating a respective contrast of color components for each perception unit; and

normalizing calculated contrasts to smooth the contrasts.

4. A method as recited in claim 1, further comprising extracting attended points from the contrast-based saliency map.

5. A method as recited in claim 1, further comprising extracting an attended area from the contrast-based saliency map.

6. A method as recited in claim 1, further comprising extracting an attended view from the contrast-based saliency map.

7. A computer-readable medium comprising computer-program instructions executable by a processor for:

preprocessing an image to generate a quantized set of image blocks; and

generating a contrast-based saliency map for three-level contrast-based image attention analysis from the quantized image blocks.

8. A computer-readable medium comprising computer-program instructions for modeling image attention, the computer-program instructions being executable by a processor for modeling image attention by:

generating a preprocessed image by;

resizing the image such that an aspect ratio of the image is maintained; and

if the image is not already in a color space that is consistent with human perception, transforming the image from a first color space to a second color space that is consistent with human perception;

quantizing the preprocessed image to generate quantized image perception units such that color in texture areas across the quantized image perception units are coarser and normalized as compared to the image; and

generating a contrast-based saliency map from the quantized image blocks, the contrast-based saliency map comprising a respective contrast of color components for each perception unit.

9. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for extracting attended points from the contrast-based saliency map.

10. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for extracting an attended view from the contrast-based saliency map.

11. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for extracting an attended view from the contrast-based saliency map, the attended view being a rectangle V(C, W, H), where C denotes an attention center, and W and H are the width and height of rectangle respectively, the attention center being a centroid of the contrast-based saliency map.

12. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for determining a size of an attended view in the contrast-based saliency map, the size being related to a 1^st order central moment of the contrast-based saliency map.

13. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for extracting an attended area from the contrast-based saliency map.

14. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for extracting attended areas from the contrast-based saliency map by performing a fuzzy growing operation on the contrast-based saliency map as a function of two classes of pixels to partition the contrast-based saliency map into two mutually exclusive areas, the two classes of pixels comprising attended and unattended pixel areas.

15. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for:

performing a fuzzy growing operation to extract attended areas from the contrast-based saliency map, the fuzzy growing operation comprising;

partitioning the contrast-based saliency map into two mutually exclusive areas as a function of classes of pixels comprising attended and unattended pixel areas;

selecting seeds to for the fuzzy growing operation according to a set of criteria such that a seed has a local maximum contrast with respect to other regional perception units and the seed belongs to an attended area;

grouping pixels in the contrast-based saliency map with gray levels that satisfy criteria that indicate attended as compared to unattended areas; and

iteratively growing the attended area by using grouped pixel as seeds in subsequent fuzzy growth operations until no candidates of the perception units can be grouped.

16. A computer-readable medium as recited in claim 8, wherein the computer-program instructions further comprise instructions for representing the contrast-based saliency map as a fuzzy event in probability space to extract attended areas.

17. A computer-readable medium as recited in claim 16, wherein the contrast-based saliency map 216 has L gray levels from g₀ to g_L-1 and the histogram of saliency map is h_k, k=0, . . . , L-1, and wherein the computer-program instructions further for representing the contrast-based saliency map as a fuzzy event in probability space further comprise instructions for:

modeling the contrast-based saliency map by a triplet (Ω

, k, P), where Ω

={g₀, g₁, . . . , g_L-1} and P is a probability measure of the occurrence of gray levels, i.e., Pr{gk}=h_k/Σ

h_k.;

denoting a membership function μ

_S(g_k) of a fuzzy set Sε

Ω

indicating a degree of properties comprising attended areas possessed by gray level g_k; and

representing the properties as a fuzzy event as follows;

$S=\underset{g_k\in <br><br>\mathrm{\Omega <br><br>}}{\overset{\text{\hspace{1em}<br><br>}}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\mu <br><br>}}_S<br><br>\left(g_k\right)/g_k;<br><br>\mathrm{and}$

computing a probability of the fuzzy event by;

$P<br><br>\left(S\right)=\underset{k=0}{\overset{L-1}{\sum <br><br>}}<br><br>\text{\hspace{1em}<br><br>}<br><br>{\mathrm{\mu <br><br>}}_S<br><br>\left(g_k\right)<br><br>P_r<br><br>\left(g_k\right).$

18. A computing device for modeling image attention, the computing device comprising a processor coupled to a memory, the memory comprising computer computer-program instructions executable by the processor for:

quantizing a preprocessed image to generate quantized image perception units such that color in texture areas across the quantized image perception units are coarser and normalized as compared to the image, the preprocessed image being a resized version of the image with an original aspect ratio and in a color space consistent with human perception; and

generating a contrast-based saliency map from the quantized image blocks, the contrast-based saliency map comprising a respective contrast of color components for each perception unit.

19. A computing device as recited in claim 18, wherein the computer-program instructions further comprise instructions for extracting attended points from the contrast-based saliency map.

20. A computing device as recited in claim 18, wherein the computer-program instructions further comprise instructions for extracting an attended area from the contrast-based saliency map.

21. A computing device as recited in claim 18, wherein the computer-program instructions further comprise instructions for extracting an attended view from the contrast-based saliency map.

22. A computing device comprising:

means for preprocessing an image to generate a quantized set of image blocks; and

means for generating a contrast-based saliency map for modeling three-levels of image attentions from the quantized image blocks.

23. A computing device as recited in claim 22, wherein the means for generating the contrast-based saliency map further comprises:

means for dividing the image subsequent to quantization into multiple perception units; and

means for calculating a respective contrast of color components for each perception unit; and

means for normalizing calculated contrasts to smooth the contrasts.

24. A computing device as recited in claim 22, further comprising means for extracting attended points from the contrast-based saliency map.

25. A computing device as recited in claim 22, further comprising means for extracting an attended area from the contrast-based saliency map.

26. A computing device as recited in claim 22, further comprising means for extracting an attended view from the contrast-based saliency map.