Image processing method and image processing device

• US 9,014,467 B2
• Filed: 05/11/2012
• Issued: 04/21/2015
• Est. Priority Date: 05/13/2011
• Status: Active Grant

• Current Assignee: Omron Corporation, Tsinghua University
• Original Assignee: Omron Corporation, Tsinghua University
• Inventors: Hu, Shi-Min, Cheng, Ming-Ming, Zhang, Guo-Xin, Mitra, Niloy J., Ruan, Xiang
• Primary Examiner(s): MOTSINGER, SEAN T
• Application Number:
  US13/469,578
• Publication Number:
  US 20120288189A1
• Time in Patent Office: 1,075 Days
• Field of Search: None
• US Class Current: 382/164
• CPC Class Codes: G06T 2207/10024 Color image
• CPC Class Codes: G06T 2207/20164 Salient point detection; Corner detection
• CPC Class Codes: G06T 2207/30196 Human being; Person
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/90 Determination of colour characteristics
• CPC Class Codes: G06V 10/462 Salient features, e.g. scale invariant feature transforms [SIFT]

Abstract

An image processing method includes a segmentation step that segments an input image into a plurality of regions by using an automatic segmentation algorithm, and a computation step that calculates a saliency value of one region of the plurality of segmented regions by using a weighted sum of color differences between the one region and all other regions. Accordingly, it is possible to automatically analyze visual saliency regions in an image, and a result of analysis can be used in application areas including significant object segmentation, object recognition, adaptive image compression, content-aware image resizing, and image retrieval.

Claims

1. An image processing method for detecting visual saliency of an image based on regional contrast, the method comprising:

segmenting an input image into a plurality of regions by using an automatic segmentation algorithm; and

calculating a saliency value of one region of the plurality of segmented regions by using a weighted sum of color differences between the one region and all other regions and by using spatial distances between the one region and all other regions,

wherein, in the segmenting, a saliency S(r_k) of a region r_k of the plurality of segmented regions is defined by the following expression;

2. The image processing method according to claim 1, wherein the parameter σ

_s² is a value in a range from 0.1 to 1.0 when spatial coordinates of a pixel in the image are normalized to values in a range from 0 to 1.

3. The image processing method according to claim 1, wherein a number of pixels included in the region r, is deemed as the weighted value w(r_i) of the region r_i.

4. The image processing method according to claim 1,

wherein the color difference D_c (r_k,r_i) between the region r_k and the region r_i is defined by the following expression:

D_c(r_k,r_i)=Σ

_i=1ⁿ¹Σ

_j=1ⁿ²f (c₁,i)f(c₂,j)D(c_1,i,c_2,j), and

wherein f(c_k,i) is an appearance frequency of i-th c_k of n_k types of colors contained in the region r_k, D(c_1,i,c_2,j) is an Euclidean distance between two colors c_1,i and c_2,j in a color space, and k={1,2}.

5. The image processing method according to claim 4, wherein the calculating is performed with 8 to 128 classes of colors in each color channel.

11. The image processing method according to claim 1, further comprising at least one of:

outputting the calculated saliency value;

displaying the calculated saliency value; and

storing the calculated saliency value.

12. The image processing method according to claim 2, further comprising at least one of:

outputting the calculated saliency value;

displaying the calculated saliency value; and

storing the calculated saliency value.

13. The image processing method according to claim 3, further comprising at least one of:

outputting the calculated saliency value;

displaying the calculated saliency value; and

storing the calculated saliency value.

14. The image processing method according to claim 4, further comprising at least one of:

outputting the calculated saliency value;

displaying the calculated saliency value; and

storing the calculated saliency value.

15. The image processing method according to claim 5, further comprising at least one of:

outputting the calculated saliency value;

displaying the calculated saliency value; and

storing the calculated saliency value.

6. An image processing device for detecting visual saliency of an image based on regional contrast, the device comprising:

a segmentation processor that segments an input image into a plurality of regions by using an automatic segmentation algorithm; and

a computation processor that calculates a saliency value of one region of the plurality of segmented regions by using a weighted sum of color differences between the one region and all other regions and by spatial distances between the one region and all other regions,

wherein a saliency S(r_k) of a region r_k of the plurality of segmented regions is defined by the following expression;

7. The image processing device according to claim 6, wherein the parameter σ

_s² is a value in a range from 0.1 to 1.0 when spatial coordinates of a pixel in the image are normalized to values in a range from 0 to 1.

8. The image processing device according to claim 6, wherein a number of pixels included in the region r, is deemed as the weighted value w(r_i) of the region r_i.

9. The image processing device according to claim 6,

wherein the color difference D_c (r_k,r_i) between the region r_k and the region r_i is defined by the following expression:

D_c(r_k,r_i)=Σ

_i=1ⁿ¹Σ

_j=1ⁿ²f(c₁,i)f(c₂,j)D(c_1,i,c_2,j), and

wherein f(c_k,i) is an appearance frequency of i-th c_k of n_k types of colors contained in region r_k, D(c_1,i,c_2,j) is an Euclidean distance between two colors c_1,i and c_2,j in a color space, and k={1,2}.

10. The image processing device according to claim 9, wherein the computation processor calculates with 8 to 128 classes of colors in each color channel.

16. The image processing device according to claim 6, further comprising at least one of:

an output processor that outputs the calculated saliency value;

a display processor that displays the calculated saliency value; and

a storage processor that stores the calculated saliency value.