Method of extracting image from input image using reference image

• US 6,453,069 B1
• Filed: 11/17/1997
• Issued: 09/17/2002
• Est. Priority Date: 11/20/1996
• Status: Expired due to Term

• Current Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Original Assignee: Canon Kabushiki Kaisha (Canon Inc.)
• Inventors: Matsugu, Masakazu, Katayama, Tatsushi, Hatanaka, Koji
• Primary Examiner(s): Boudreau, Leo
• Assistant Examiner(s): SHERALI, ISHRAT I
• Application Number:
  US08/972,166
• Time in Patent Office: 1,765 Days
• Field of Search: 382/199, 382/266, 382/103, 382/291, 382/180, 382/107, 382/164, 382/173, 382/282-283, 382/224, 382/251, 382/165, 382/276, 358/452-453, 358/537-538, 345/626, 348/699-700, 348/14.15
• US Class Current: 382/173
• CPC Class Codes: G06T 2207/10016 Video; Image sequence
• CPC Class Codes: G06T 2207/20156 Automatic seed setting
• CPC Class Codes: G06T 2207/20224 Image subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/174 involving the use of two or more images
• CPC Class Codes: G06T 7/181 involving edge growing; involving edge linking
• CPC Class Codes: G06T 7/187 involving region growing; involving region merging; involving connected component labelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06V 10/46 Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features colour feature extraction G06V10/56
• CPC Class Codes: G06V 10/7515 Shifting the patterns to accommodate for positional errors

Abstract

This invention relates to an image processing method for precisely and automatically recognizing a specific image extraction region to be divided from an input image, and extracting the specific image extraction region. Thresholding is done for intensity differences of edge data obtained from the input data to extract difference edge data. A main extraction region is estimated from the outermost contour line extracted based on the extraction result of the difference edge data. Thresholding or the like is done in units of pixels of the input image to extract an initial region. Extraction region determination processing is done for extracting an accurate target region by combining the main extraction region and initial region.

Claims

1. An image segmentation method for segmenting an object, which is not present in a reference image, from an input image including the object using the reference image, comprising:

the edge intensity distribution extraction step of extracting edge intensity distributions in the reference image and input image;

the direction-classified line detection step of detecting, in each of a plurality of blocks divided on an image screen, line components in terms of predetermined directions for edges in the reference image and input image on the basis of the extracted edge intensity distributions; and

the image region specifying step of specifying an existence range of the object in the input image on the basis of a distribution of differences between the detected line components in each of said directions between the reference image and input image.

2. An image segmentation method for segmenting a predetermined image region from an input image input from an external device such as an image input device using a reference image, comprising:

the edge distribution extraction step of extracting edge distributions in the input image and reference image;

the direction-classified line detection step of detecting, in each of a plurality of blocks divided on an image screen, line distributions in terms of predetermined directions in the input image and reference image on the basis of the extracted edge distributions;

the singular edge extraction step of extracting a singular edge on the basis of a difference between the detected line distributions in each of said directions between the reference image and input image; and

the image extraction step of extracting the predetermined image region in the input image on the basis of the extracted singular edge.

3. An image identification method for identifying an object, which is not present in a reference image, in an input image including the object using a standard model image representing a predetermined object and the reference image, comprising:

the edge intensity distribution extraction step of extracting edge intensity distributions in the reference image and input image;

the direction-classified line detection step of detecting, in each of a plurality of blocks divided on an image screen, line components in terms of predetermined directions for edges in the reference image and input image on the basis of the extracted edge intensity distributions;

the auto-framing step of specifying an existence range of the object in the input image on the basis of a distribution of differences between the detected line components in each of said directions between the reference image and input image; and

the model size estimation step of estimating a size with respect to the standard model image on the basis of the specified existence range of the object,

wherein a size of the standard model image is changed to the estimated size, and thereafter, the object is identified on the basis of similarity between the object image present in the existence range in the input image and the size-changed standard model image.

4. An image segmentation method for extracting an object, which is not present in a reference image, from an input image including the object using the reference image, comprising:

the edge intensity distribution extraction step of extracting edge intensity distributions in the reference image and input image;

the direction-classified line detection step of detecting, in each of a plurality of blocks divided on an image screen, line components in terms of predetermined directions for edges in the reference image and input image on the basis of the extracted edge intensity distributions;

the auto-framing step of specifying an existence range of the object in the input image on the basis of a distribution of differences between the detected line components in each of said directions between the reference image and input image; and

the extraction processing step of performing extraction processing of the object within the specified existence range of the object.

5. An image segmentation method for segmenting an image region to be extracted from an input image using a reference image that represents a region approximating a remaining region excluding the image region, comprising:

the edge intensity distribution extraction step of extracting edge intensity distributions in the reference image and input image;

the direction-classified line detection step of detecting, in each of a plurality of blocks divided on an image screen, line components in terms of predetermined directions for edges in the reference image and input image on the basis of the extracted edge intensity distributions;

the singular contour extraction step of extracting a singular contour portion of the image region to be extracted on the basis of a distribution of differences between the detected line components in each of said directions in the reference image and input image; and

the image extraction step of extracting the image region to be extracted on the basis of distribution data representing the extracted singular contour portion.

6. The method according to claim 5, wherein the image extraction step comprises:

the partial region extraction step of extracting a portion of the image region to be extracted as a partial region from the input image;

the region growing step of performing region growing by thresholding similarities between the extracted partial region as a seed and its neighboring regions; and

the extraction step of extracting a region obtained by the region growth as the image region to be extracted.

7. The method according to claim 5, wherein the singular contour extraction step includes the step of extracting, as the singular contour portion, edges in the input image which have different line labels in terms of directions in identical neighboring regions of the input image and reference image.

8. The method according to claim 5, wherein the singular contour extraction step comprises:

the dominant line map extraction step of dividing the reference image into a plurality of blocks and detecting dominant line direction components in the blocks; and

the line direction comparison step of comparing a label assigned to each of edges of the input image and the dominant direction line component in the block to which that edge belongs, and

when the label assigned to the edge is different from the dominant direction line component in the block to which that edge belongs, the edge is extracted as the singular contour portion of the input image.

9. The method according to claim 5, wherein the image extraction step comprises:

the partial region extraction step of binarizing a portion of the image region to be extracted and extracting the binary data as mask data;

the smoothing step of smoothing the extracted mask data; and

the singular contour restoration step of restoring the singular contour portion to mask data after the mask data is smoothed.

10. The method according to claim 5, wherein the singular contour extraction step includes the step of detecting a occluding boundary line serving as a boundary between the image region to be extracted, and the remaining region, and determining the detected occluding boundary line as the singular contour portion.

11. The method according to claim 10, wherein the singular contour extraction step comprises:

the dominant line map extraction step of segmenting the reference image into a plurality of blocks and detecting dominant line direction components in the blocks,

a boundary point located in the vicinity of a boundary between a block without any dominant direction line component, and a block with the dominant direction line component among the blocks of the input image is extracted, and

an edge of the input image located at a position closest to the boundary point in a predetermined local region including the boundary point is extracted as a portion of the occluding boundary line.

12. The method according to claim 6, wherein the region growing step includes the step of controlling the region growing so that a region growing direction from the edge substantially agrees with a label in units of directions of that edge.

13. An image segmentation method for segmenting an image region to be extracted from an input image using a reference image that represents a region approximating a remaining region excluding the image region, comprising:

the low-resolution image extraction step of extracting low-resolution image portions in the input image and reference image;

the image matching step of performing matching corresponding points between for the input image and reference image;

the dominant line map extraction step of segmenting the input image and reference image into a plurality of blocks an detecting, in each of a plurality of blocks divided on an image screen, dominant line direction components in the blocks; and

the extraction step of extracting the image region on the basis of a degree of mating between a label in each of said directions of each edge of the input image and a label of the dominant line map of the reference image at the edge position.

14. The method according to claim 13, wherein the image matching step comprises:

the corresponding point extraction step of extracting corresponding points between the reference image and input image;

the first transformation step of geometrically transforming one of the input image and reference image on the basis of the extracted corresponding points; and

the second transformation step of performing color correction of one of the input image and reference image, so that corresponding pixels in regions including the corresponding points have substantially equal gray levels after the geometric transformation.

15. The method according to claim 14, wherein the geometric transformation includes entire or local processing associated with at least one of a translation, rotation, magnification transformation, and perspective transformation.

16. An image segmentation apparatus for segmenting an image region to be extracted from an input image using a reference image that represents a region substantially equal to a remaining region excluding the image region, comprising:

storage means for storing the reference image;

edge extraction means for extracting edge distributions in the input image and reference image;

direction-classified line detection means for detecting, in each of a plurality of blocks divided on an image screen, line distributions in terms of directions in the input image and reference image on the basis of the extracted edge distributions;

corresponding point extraction means for extracting corresponding point information between the reference image and input image;

transformation means for geometrically transforming one of the input image and reference image on the basis of the extracted corresponding point information;

singular edge extraction means for extracting a singular edge on the basis of a line distribution difference in each of said directions between the geometrically transformed image, and the other image; and

segmentation means for segmenting the image region to be extracted from the input image on the basis of the extracted singular edge.

17. The apparatus according to claim 16, wherein the geometric transformation includes entire or local processing associated with at least one of a translation, rotation, magnification transformation, and perspective transformation.

18. An image identification apparatus for identifying an object in an input image including the object which is not present in a reference image using a standard model image representing a predetermined object and the reference image, comprising:

edge intensity distribution extraction means for extracting edge intensity distributions in the reference image and input image;

direction-classified line detection means for detecting, in each of a plurality of blocks on an image screen, line components in terms of predetermined directions for edges in the reference image and input image on the basis of the extracted edge intensity distributions;

auto-framing means for specifying an existence range of the object in the input image on the basis of a distribution of differences between the detected line components in each of said directions between the reference image and input image; and

model size estimation means for estimating a size with respect to the standard model image on the basis of the specified existence range of the object,

wherein a size of the standard model image is changed to the estimated size, and thereafter, the object is identified on the basis of similarity between the object image present in the existence range in the input image and the size-changed standard model image.