Active segmentation for groups of images

• US 8,045,800 B2
• Filed: 02/04/2008
• Issued: 10/25/2011
• Est. Priority Date: 06/11/2007
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Tang, Xiaoou, Yang, Qiong
• Primary Examiner(s): Le; Brian Q
• Assistant Examiner(s): Bayat; Ali
• Application Number:
  US12/025,703
• Publication Number:
  US 20080304743A1
• Time in Patent Office: 1,359 Days
• Field of Search: 382/173, 382/162, 382/164, 382/170, 382/171, 382/159, 382/190, 382/178, 382/180, 382/203, 382/224, 358/453, 358/462, 714/38, 714/9, 714/10, 717/140, 717/120
• US Class Current: 382/173
• CPC Class Codes: G06V 10/248 by interactive preprocessing or interactive shape modelling, e.g. feature points assigned by a user
• CPC Class Codes: G06V 10/267 by performing operations on regions, e.g. growing, shrinking or watersheds
• CPC Class Codes: G06V 10/28 Quantising the image, e.g. histogram thresholding for discrimination between background and foreground patterns

Abstract

Systems and methods of segmenting images are disclosed herein. The similarity of images in a set of images is compared. A group of images is selected from the set of images. The images in the group of images are selected based on compared similarities among the images. An informative image is selected from the group of images. User-defined semantic information of the informative image is received. The group of images as a graph is modeled as a graph. Each image in the group of images denotes a node in the graph. Edges of the graph denote a foreground relationship between images or a background relationship between images. One or more images in the group of images are automatically segmented by propagating the semantic information of the informative image to images in the group of images having a corresponding graph node that is related to a graph node corresponding to the informative image. Segmentation results can be refined according to user provided image semantics.

Claims

1. A computer-implemented method of segmenting images, comprising:

comparing similarity of images in a set of images;

selecting a group of images from the set of images, wherein the images in the group of images are selected based on compared similarities among the images;

selecting from the group of images an informative image;

receiving user-defined semantic information of the informative image;

modeling the group of images as a graph, wherein each image in the group of images denotes a node in the graph, and wherein edges of the graph denote a foreground relationship between images or a background relationship between images; and

automatically segmenting one or more images in the group of images by propagating the semantic information of the informative image to images in the group of images having a corresponding graph node that is related to a graph node corresponding to the informative image.

2. The method of claim 1, wherein comparing similarity of images in a set of images includes constructing a code book based on words of the images in the set of images.

3. The method of claim 2, wherein selecting a group of images comprises selecting using similarities of the words of each image.

4. The method of claim 2, wherein selecting the informative image comprises selecting an image in the group of images with the higher number of non-overlapping words in relation to other images in the group of images.

5. The method of claim 1, further comprising:

detecting errors in a segmented image from the one or more images segmented from the group of images;

receiving user-defined refined semantic information regarding the segmented image;

automatically segmenting other images in the group of images that are related to the segmented image by propagating the refined semantic information of the segmented image to the other images having a corresponding node that is related to a node corresponding to the segmented image via at least one edge.

6. The method of claim 1, wherein the corresponding nodes of the images in the group of images are related to the graph node corresponding to the informative image if the graph contains at least one edge between the corresponding nodes and the graph node.

7. The method of claim 1, further comprising classifying an image in the set of images using background and foreground classifications.

8. A non-signal computer-readable storage medium having computer-executable instructions for causing a computer to perform acts comprising:

selecting a group of images from a set of images, wherein the images in the group of images are selected based on similarities among the images in the set of images;

selecting from the group of images an informative image;

receiving user-defined semantic information of the informative image;

modeling the group of images as a graph, wherein each image in the group of images denotes a node in the graph, and wherein edges of the graph denote a foreground relationship between images or a background relationship between images; and

automatically segmenting one or more images in the group of images by propagating the semantic information of the informative image to images in the group of images having a corresponding graph node that is related to a graph node corresponding to the informative image.

9. The computer-readable storage medium of claim 8, wherein comparing similarity of images in a set of images includes constructing a code book based on words of the images in the set of images.

10. The computer-readable storage medium of claim 9, wherein selecting a group of images comprises selecting using similarities of the words of each image.

11. The computer-readable storage medium of claim 9, wherein selecting the informative image comprises selecting an image in the group of images with the higher number of non-overlapping words in relation to other images in the group of images.

12. The computer-readable storage medium of claim 8, the acts further comprising:

detecting errors in a segmented image from the one or more images segmented from the group of images;

receiving user-defined refined semantic information regarding the segmented image;

automatically segmenting other images in the group of images that are related to the segmented image by propagating the refined semantic information of the segmented image to the other images having a corresponding node that is related to a node corresponding to the segmented image via at least one edge.

13. The computer-readable storage medium of claim 8, wherein the corresponding nodes of the images in the group of images are related to the graph node corresponding to the informative image if the graph contains at least one edge between the corresponding nodes and the graph node.

14. The computer-readable storage medium of claim 8, the acts further comprising classifying an image in the set of images using background and foreground classifications.