Salient Object Detection

• US 20080304740A1
• Filed: 06/06/2007
• Published: 12/11/2008
• Est. Priority Date: 06/06/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sun, Jian, Shum, Heung-Yeung, Tang, Xiaoou, Liu, Tie
• Granted Patent: US 7,940,985 B2
• Time in Patent Office: Days
• US Class Current: 382/168
• CPC Class Codes: G06V 10/25 Determination of region of interest [ROI] or a volume of interest [VOI]
• CPC Class Codes: G06V 10/457 by analysing connectivity, e.g. edge linking, connected component analysis or slices

Abstract

Methods for detecting a salient object in an input image are described. For this, the salient object in an image may be defined using a set of local, regional, and global features including multi-scale contrast, center-surround histogram, and color spatial distribution. These features are optimally combined through conditional random field learning. The learned conditional random field is then used to locate the salient object in the image. The methods can also use image segmentation, where the salient object is separated from the image background.

Claims

1. A method for salient object detection in an image comprising:

receiving the image that includes the salient object and a background;

defining the salient object using various feature maps;

combining the various feature maps of the salient object; and

detecting the salient object using the combined features.

2. The method of claim 1, wherein the features maps define the salient object locally, regionally, and globally.

3. The method of claim 2, wherein the feature map that defines the salient object locally is defined by a multi-scale contrast feature map.

4. The method of claim 2, wherein the feature map that defines the salient object regionally is defined by a center-surround histogram feature map.

5. The method of claim 2, wherein the feature map that defines the salient object globally is defined by a color-spatial variance feature map.

6. The method of claim 1, wherein the combining is performed by conditional random field learning.

7. The method of claim 6, wherein an optimal linear combination of the feature maps is obtained under a maximized linear criteria through the conditional random field learning.

8. A method for conditional random field learning for salient object detection comprising:

collecting a number of images of salient objects, from a variety of sources, wherein each of the images are labeled;

computing a saliency probability map for each of the images;

computing labeling consistency statistics for each of the images;

selecting consistent images from the labeled images;

determining feature maps for selected consistent images; and

training conditional random fields for the conditional random field learning, using the feature maps.

9. The method of claim 8, wherein the collecting is from one or more of the following sources:

an image database, an image forum, and a personal collection.

10. The method of claim 8, wherein the collecting is perform using an image search engine.

11. The method of claim 8, wherein the images are labeled by multiple users, identifying a salient object in the image.

12. The method of claim 8, wherein the computing the saliency probability map is defined as G={g_x|g_xε

[0,1]}, and the salient object is computed for each label images by the following equation;

$g_x=\frac{1}{M}<br><br>\underset{m=1}{\overset{M}{\sum <br><br>}}<br><br>a_x^m$

13. The method of claim 8, wherein the computing labeling consistency statistics is defined by C^t for each image, and computes by the following equations:

$C_t=\frac{\underset{x\in <br><br>\left\{g_x><br><br>t\right\}}{\sum <br><br>}<br><br>g_x}{\underset{x}{\sum <br><br>}<br><br>g_x}$

14. The method of claim 8, wherein the selecting includes region-based and boundary-based measurements.

15. The method of claim 14, wherein region-based measurement uses precision, recall, and F-measure to determine the accuracy of a salient object.

16. The method of claim 14, wherein boundary-based measurement measure an average displacement error of corresponding boundaries of two rectangles that identify the salient object in an image set.

17. The method of claim 8, wherein the determining feature maps includes creating feature maps that define local, global, and global features of a salient object.

18. The method of claim 8, wherein the training conditional random fields uses a conditional random field model wherein probability of a label A={a_x} in an image I is modeled as a conditional distribution P(A|I)=

$\frac{1}{Z}$

exp 

( - E 

( A | I ) ) ,

where Z is a partition function, and E(A|I) is defined as a linear combination of a number of K salient features F_k(a_x, I) and a pairwise feature S(a_x, a_x′

, I), E(A|I) is defined by the following equation;

$E<br><br>\left(A|I\right)=\underset{x}{\sum <br><br>}<br><br>\underset{k=1}{\overset{K}{\sum <br><br>}}<br><br>{\mathrm{\lambda <br><br>}}_k<br><br>F_k<br><br>\left(a_x,I\right)+\underset{x,x^{\mathrm{\prime <br><br>}}}{\sum <br><br>}<br><br>S<br><br>\left(a_x,a_{x^{\mathrm{\prime <br><br>}}},I\right)$

19. A method for salient object detection in an image comprising:

receiving the image that includes the salient object;

resealing the image to standard size;

defining local, regional, and global features of the image; and

detecting the salient object by learned conditional random field.

20. The method of claim 19, wherein the defining local features of the image uses a multi-scale contrast feature map, regional features of the image uses a center-surround histogram feature map, and global features of the image uses a color spatial feature map.