Image extraction apparatus and method

• US 6,757,444 B2
• Filed: 05/01/2000
• Issued: 06/29/2004
• Est. Priority Date: 07/05/1996
• Status: Expired due to Fees

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Takiguchi, Hideo, Hatanaka, Koji
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): Sherali, Ishrat I
• Application Number:
  US09/561,780
• Publication Number:
  US 20030099397A1
• Time in Patent Office: 1,520 Days
• Field of Search: 382/282-284, 382/131, 382/197-199, 382/173, 382/180, 382/203, 382/220, 382/162-165, 382/103, 382/107, 348/251, 348/586-587, 348/597-599, 348/239, 348/241, 348/169, 348/699-700, 348/14.15, 358/450, 358/540, 358/453, 358/537, 358/538, 358/428, 358/447, 358/504, 358/517-518, 345/626
• US Class Current: 382/283
• CPC Class Codes: H04N 19/20 using video object coding
• CPC Class Codes: H04N 19/503 involving temporal prediction adaptive coding with adaptive selection between spatial and temporal predictive coding H04N19/107; adaptive coding with adaptive selection among a plurality of temporal predictive coding modes H04N19/109

Abstract

There is disclosed an image extraction method for extracting, from a subject image that records both the background and the object to be extracted, image data of the object using a mask. An initial mask used for extracting the subject region is generated on the basis of difference data between the subject image and the background image that records the background alone, the region of the initial mask is grown on the basis of the similarity between the features of a first region and its neighboring second region in the subject image corresponding to the initial mask, and the image data of the object is extracted from the first image on the basis of the grown mask region.

Claims

1. An image extraction method comprising:

the partial region extraction step of extracting a partial region as a portion of a subject to be extracted from an input image;

the region growing step of growing the extracted partial region using the extracted partial region as an initial mask by obtaining a similarity by comparing the partial region with the neighboring region with regard to a predetermined feature distribution and comparing the similarity with a predetermined threshold value which is set on the basis of a distribution of edge information; and

the extraction step of extracting an image of the subject on the basis of the region-grown region.

2. The method according to claim 1, wherein the partial region extraction step includes the step of extracting the partial region on the basis of a difference between a background image excluding the subject, and a subject image including the subject.

3. The method according to claim 1, wherein the feature distribution is an edge distribution of the subject.

4. The method according to claim 1, wherein the feature distribution is a distribution within a maximum growing range set based on the partial region.

5. The method according to claim 1, wherein the threshold value is set to assume a value that suppresses growing of the region at an edge position as compared to a non-edge position.

6. The method according to claim 4, wherein the threshold value is set to assume a value that promotes growing of the region in a region within the maximum growing range, and to assume a value that suppresses growing of the region outside the maximum growing region.

7. The method according to claim 4, wherein the maximum growing range is obtained as an output when a shape of the partial region is smoothed using a smoothing filter having a predetermined size.

8. The method according to claim 1, wherein the input image includes time-serial images, and the partial region extraction step includes the step of extracting the partial region on the basis of difference data between image frames at different times of the input image.

9. The method according to claim 1, wherein the input image includes a plurality of images from a plurality of different view point positions, and the partial region extraction step includes the step of extracting the partial region on the basis of a disparity distribution between the input images.

10. An image extraction apparatus for extracting, from a first image including both a background and an object to be extracted, image data of the object using a mask, comprising:

temporary storage means for receiving and temporarily storing the first image and a second image that records the background;

initial mask generation means for generating an initial mask of an extraction region on the basis of difference data between the stored first and second images;

region growing means for growing a region of the initial mask by obtaining a feature similarity by comparing the partial region with the neighboring region with regard to a predetermined feature information and comparing the similarity with a predetermined threshold value which is set on the basis of a distribution of edge information; and

first image extraction means for extracting the image data of the object from the first image on the basis of the grown mask region.

11. The apparatus according to claim 10, wherein said initial mask generation means comprises:

first seed extraction means for extracting a color difference seed by performing a threshold value process of a difference between color components at individual points of the first and second images;

second seed extraction means for respectively extracting first and second edge intensity images from the first and second images, and extracting an edge difference seed by performing a threshold value process of difference data between the first and second edge intensity images; and

generation means for generating an initial seed on the basis of outputs from said first and second seed extraction means.

12. The apparatus according to claim 11, wherein said first image extraction means comprises:

means for extracting difference edge data between the first and second images; and

means for shaping a contour line of the object on the basis of the extracted difference edge data.

13. The apparatus according to claim 12, wherein said shaping means comprises:

threshold value process means for performing a threshold value process of the difference edge data or edge data of the first image;

first continuity evaluation means for evaluating shape continuity for edge candidates that remain after the threshold value process;

second continuity evaluation means for evaluating continuity of image features of the edge candidates;

edge selection means for selecting one of the edge candidates on the basis of outputs from said first and second continuity evaluation means; and

smoothing means for smoothing an extracted contour line or a correction mask region including the contour line.

14. The apparatus according to claim 12, wherein said region growing means comprises:

means for determining a condition for suppressing a growing process; and

means for determining a similarity.

15. An image extraction apparatus comprising:

partial region extraction means for extracting a partial region as a portion of a subject to be extracted from an input image;

region growing means for growing the extracted partial region using the extracted partial region as an initial mask by obtaining a similarity by comparing the partial region with the neighboring region with regard to a predetermined feature distribution and comparing the similarity with a predetermined threshold value which is set on the basis of a distribution of edge information; and

extraction means for extracting an image of the subject on the basis of the region-grown region.