IMAGE PROCESSING METHOD AND IMAGE PROCESSING DEVICE

• US 20120288189A1
• Filed: 05/11/2012
• Published: 11/15/2012
• Est. Priority Date: 05/13/2011
• Status: Active Grant

• Current Assignee: Omron Corporation, Tsinghua University
• Original Assignee: Omron Corporation, Tsinghua University
• Inventors: Hu, Shi-Min, Cheng, Ming-Ming, Zhang, Guo-Xin, Mitra, Niloy J., Ruan, Xiang
• Granted Patent: US 9,014,467 B2
• Time in Patent Office: Days
• US Class Current: 382/164
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

An image processing method includes a segmentation step that segments an input image into a plurality of regions by using an automatic segmentation algorithm, and a computation step that calculates a saliency value of one region of the plurality of segmented regions by using a weighted sum of color differences between the one region and all other regions. Accordingly, it is possible to automatically analyze visual saliency regions in an image, and a result of analysis can be used in application areas including significant object segmentation, object recognition, adaptive image compression, content-aware image resizing, and image retrieval.

Claims

1. An image processing method for detecting visual saliency of an image based on regional contrast, the method comprising:

a segmentation step that segments an input image into a plurality of regions by using an automatic segmentation algorithm; and

a computation step that calculates a saliency value of one region of the plurality of segmented regions by using a weighted sum of color differences between the one region and all other regions.

2. The image processing method according to claim 1,

wherein in the computation step, a saliency S(r_k) of a region r_k of the plurality of segmented regions is defined by the following expression:

3. The image processing method according to claim 2, wherein the parameter σ

_s² is a value in a range from 0.1 to 1.0 when spatial coordinates of a pixel in the image are normalized to values in a range from 0 to 1.

4. The image processing method according to claim 2, wherein a number of pixels included in the region r_i is deemed as the weighted value w(r_i) of the region r_i.

5. The image processing method according to claim 2,

wherein the color difference D_c(r_k,r_i) between the region r_k and the region r_i is defined by the following expression:

D_c(r_k,r_i)=Σ

_i=1ⁿ¹Σ

_j=1ⁿ²f(c₁,i)f(c₂,j)D(c_1,i,c_2,j), and

wherein f(c_k,i) is an appearance frequency of i-th c_k of n_k types of colors contained in the region r_k, D(c_1,i,c_2,j) is an Euclidean distance between two colors and c_1,i and c_2,j in a color space, and k={1,2}.

6. The image processing method according to claim 5, wherein the computation is performed with 8 to 128 classes of colors in each color channel.

7. The image processing method according to claim 1, further comprising at least one of:

an output step that outputs the calculated saliency value;

a display step that displays the calculated saliency value; and

a storage step that stores the calculated saliency value.

15. The image processing method according to claim 2, further comprising at least one of:

an output step that outputs the calculated saliency value;

a display step that displays the calculated saliency value; and

a storage step that stores the calculated saliency value.

16. The image processing method according to claim 3, further comprising at least one of:

an output step that outputs the calculated saliency value;

a display step that displays the calculated saliency value; and

a storage step that stores the calculated saliency value.

17. The image processing method according to claim 4, further comprising at least one of:

an output step that outputs the calculated saliency value;

a display step that displays the calculated saliency value; and

a storage step that stores the calculated saliency value.

18. The image processing method according to claim 5, further comprising at least one of:

an output step that outputs the calculated saliency value;

a display step that displays the calculated saliency value; and

a storage step that stores the calculated saliency value.

19. The image processing method according to claim 6, further comprising at least one of:

an output step that outputs the calculated saliency value;

a display step that displays the calculated saliency value; and

a storage step that stores the calculated saliency value.

8. An image processing device for detecting visual saliency of an image based on regional contrast, the device comprising:

a segmentation unit that segments an input image into a plurality of regions by using an automatic segmentation algorithm; and

a computation unit that calculates a saliency value of one region of the plurality of segmented regions by using a weighted sum of color differences between the one region and all other regions.

9. The image processing device according to claim 8,

wherein a saliency S(r_k) of a region r_k of the plurality of segmented regions is defined by the following expression:

10. The image processing device according to claim 9,

wherein the parameter σ

_s² is a value in a range from 0.1 to 1.0 when spatial coordinates of a pixel in the image are normalized to values in a range from 0 to 1.

11. The image processing device according to claim 9, wherein a number of pixels included in the region r_i is deemed as the weighted value w(r_i) of the region r_i.

12. The image processing device according to claim 9,

wherein the color difference D_c(r_k,r_i) between the region r_k and the region r_i is defined by the following expression:

D_c(r_k,r_i)=Σ

_i=1ⁿ¹Σ

_j=1ⁿ²f(c₁,i)f(c₂,j)D(c_1,i,c_2,j), and

wherein f(c_k,i) is an appearance frequency of i-th c_k of n_k types of colors contained in region r_k, D(c_1,i,c_2,j) is an Euclidean distance between two colors c_1,i and c_2,j in a color space, and k={1,2}.

13. The image processing device according to claim 12, wherein the computation is performed with 8 to 128 classes of colors in each color channel

14. The image processing device according to claim 8, further comprising at least one of:

an output unit that outputs the calculated saliency value;

a display unit that displays the calculated saliency value; and

a storage unit that stores the calculated saliency value.

20. The image processing device according to claim 9, further comprising at least one of:

an output unit that outputs the calculated saliency value;

a display unit that displays the calculated saliency value; and

a storage unit that stores the calculated saliency value.