Object detecting system based on multiple-eye images

• US 5,917,936 A
• Filed: 02/14/1997
• Issued: 06/29/1999
• Est. Priority Date: 02/14/1996
• Status: Expired due to Term

• Current Assignee: NEC Corporation
• Original Assignee: NEC Corporation
• Inventors: Katto, Jiro
• Primary Examiner(s): Mancuso, Joseph
• Assistant Examiner(s): Miller, Martin E.
• Application Number:
  US08/800,402
• Time in Patent Office: 865 Days
• Field of Search: 382/291, 382/225, 382/154, 382/254, 382/263, 382/282, 382/283, 382/149, 382/228
• US Class Current: 382/154
• CPC Class Codes: G06T 2207/10028 Range image; Depth image; 3D point clouds
• CPC Class Codes: G06T 2207/10148 Varying focus
• CPC Class Codes: G06T 5/50 using two or more images, e.g. averaging or subtraction
• CPC Class Codes: G06T 7/11 Region-based segmentation
• CPC Class Codes: G06T 7/143 involving probabilistic approaches, e.g. Markov random field [MRF] modelling
• CPC Class Codes: G06T 7/194 involving foreground-background segmentation
• CPC Class Codes: G06T 7/571 from focus
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06T 7/97 Determining parameters from multiple pictures depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85
• CPC Class Codes: G06V 10/255 Detecting or recognising potential candidate objects based on visual cues, e.g. shapes
• CPC Class Codes: G06V 40/162 using pixel segmentation or colour matching

Abstract

An object detecting system based on multiple-eye images, comprises an image sharpening unit receiving a plurality of images having substantially the same optical axis and different focus positions, for generating one sharpened image, a focus information and a cluster information, a mask generating unit receiving the focus information for generating a first objection detecting information, a region integrating unit receiving the cluster information for generating an integrated cluster information, a region selecting unit receiving the first objection detecting information and the integrated cluster information for generating a second objection detecting information, and an object extracting unit receiving the sharpened image and the second objection detecting information for extracting an object image from the sharpened image in accordance with the second objection detecting information.

Claims

1. An object detecting system based on multiple-eye images for extracting an object image, comprising:

an image sharpening means receiving a plurality of images having substantially the same optical axis and different focus positions, for generating a sharpened image, a focus information and a cluster information;

a mask generating means receiving said focus information for generating a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating means receiving said cluster information for generating an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting means receiving said first object detecting information and said integrated cluster information for generating a second object detecting information; and

an object extracting means receiving said sharpened image and said second object detecting information for extracting an object image from said sharpened image in accordance with said second object detecting information.

2. An object detecting system based on multiple-eye images, comprising:

a disparity detecting means receiving a plurality of images having different optical axes, for generating a reference image, a disparity information and a cluster information;

a mask generating means receiving said disparity information for generating a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating means receiving said cluster information for generating an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting means receiving said first object detecting information and said integrated cluster information for generating a second object detecting information; and

an object extracting means receiving said reference image and said second object detecting information for extracting an object image from said reference image in accordance with said second object detecting information.

3. An object detecting system based on multiple-eye images, comprising:

a first image sharpening means receiving a plurality of images having substantially the same first optical axis and different focus positions, for generating a first sharpened image, a focus information and a cluster information;

a second image sharpening means receiving a plurality of images having substantially the same second optical axis different from said first optical axis and different focus positions, for generating a second sharpened image;

a disparity detecting means receiving said first sharpened image and said second sharpened image having different optical axes, for generating a disparity information;

a mask generating means receiving said focus information and said disparity information for generating a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating means receiving said cluster information for generating an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting means receiving said first object detecting information and said integrated cluster information for generating a second object detecting information; and

an object extracting means receiving said first sharpened image and said second object detecting information for extracting an object image from said first sharpened image in accordance with said second object detecting information.

4. An object detecting system based on multiple-eye images, comprising:

a first image sharpening means receiving a plurality of images having substantially the same first optical axis and different focus positions, for generating a first sharpened image and a focus information;

a second image sharpening means receiving a plurality of images having substantially the same second optical axis different from said first optical axis and different focus positions, for generating a second sharpened image;

a disparity detecting means receiving said first sharpened image and said second sharpened image having different optical axes, for generating a disparity information and a cluster information;

a mask generating means receiving said focus information and said disparity information for generating a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating means receiving said cluster information for generating an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting means receiving said first object detecting information and said integrated cluster information for generating a second object detecting information; and

an object extracting means receiving said first sharpened image and said second object detecting information for extracting an object image from said first sharpened image in accordance with said second object detecting information.

5. An object detecting system based on multiple-eye images for extracting an object image, comprising:

an image sharpening unit configured to receive a plurality of images having substantially the same optical axis and different focus positions, and configured to generate a sharpened image, a focus information and a cluster information;

a mask generating unit communicatively coupled to said image sharpening unit, said mask generating unit configured to receive said focus information, and configured to generate a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating unit communicatively coupled to said image sharpening unit, said region integrating unit configured to receive said cluster information, and configured to generate an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting unit communicatively coupled to said region integrating unit and said mask generating unit, said region selecting unit configured to receive said first object detecting information and said integrated cluster information, and configured to generate a second object detecting information; and

an object extracting unit communicatively coupled to said image sharpening unit and said region selecting unit, said object extracting unit configured to receive said sharpened image and said second object detecting information, and configured to extract an object image from said sharpened image in accordance with said second object detecting information.

6. An object detecting system based on multiple-eye images, comprising:

a disparity detecting unit configured to receive a plurality of images having different optical axes, and configured to generate a reference image, a disparity information, and a cluster information;

a mask generating unit communicatively coupled to said disparity detecting unit, said mask generating unit configured to receive said disparity information, and configured to generate a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating unit communicatively coupled to said disparity detecting unit, said region integrating unit configured to receive said cluster information, and configured to generate an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting unit communicatively coupled to said mask generating unit and said region integrating unit, said region selecting unit configured to receive said first object detecting information and said integrated cluster information, and configured to generate a second object detecting information; and

an object extracting unit communicatively coupled to said region selecting unit and said disparity detecting unit, said object extracting unit configured to receive said reference image and said second object detecting information, and configured to extract an object image from said reference image in accordance with said second object detecting information.

7. An object detecting system based on multiple-eye images, comprising:

a first image sharpening unit configured to receive a plurality of images having substantially the same first optical axis and different focus positions, for generating a first sharpened image, a focus information and a cluster information;

a second image sharpening unit configured to receive a plurality of images having substantially the same second optical axis different from said first optical axis and different focus positions, for generating a second sharpened image;

a disparity detecting unit communicatively coupled to said first image sharpening unit and said second image sharpening unit, said disparity detecting unit configured to receive said first sharpened image and said second sharpened image having different optical axes, for generating a disparity information;

a mask generating unit communicatively coupled to said disparity detecting unit and said first image sharpening unit, said mask generating unit configured to receive said focus information and said disparity information for generating a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating unit communicatively coupled to said first image sharpening unit, said region integrating unit configured to receive said cluster information for generating an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting unit communicatively coupled to said mask generating unit and said region integrating unit, said region selecting unit configured to receive said first object detecting information and said integrated cluster information for generating a second object detecting information; and

an object extracting unit communicatively coupled to said first image sharpening unit and said region selecting unit, said object extracting unit configured to receive said first sharpened image and said second object detecting information for extracting an object image from said first sharpened image in accordance with said second object detecting information.

8. An object detecting system based on multiple-eye images, comprising:

a first image sharpening unit configured to receive a plurality of images having substantially the same first optical axis and different focus positions, for generating a first sharpened image and a focus information;

a second image sharpening unit configured to receive a plurality of images having substantially the same second optical axis different from said first optical axis and different focus positions, for generating a second sharpened image;

a disparity detecting unit communicatively coupled to said first image sharpening unit and said second image sharpening unit, said disparity detecting unit configured to receive said first sharpened image and said second sharpened image having different optical axes, for generating a disparity information and a cluster information;

a mask generating unit communicatively coupled to said first image sharpening unit and said disparity detecting unit, said mask generating unit configured to receive said focus information and said disparity information for generating a first object detecting information, said first object detecting information comprising information corresponding to a rough object extraction;

a region integrating unit communicatively coupled to said disparity detecting unit, said region integrating unit configured to receive said cluster information for generating an integrated cluster information based on information on an integrated cluster obtained by collecting a plurality of clusters having a substantially similar feature;

a region selecting unit communicatively coupled to said mask generating unit and said region integrating unit, said region selecting unit configured to receive said first object detecting information and said integrated cluster information for generating a second object detecting information; and

an object extracting unit communicatively coupled to said first image sharpening unit and said region selecting unit, said object extracting unit configured to receive said first sharpened image and said second object detecting information for extracting an object image from said first sharpened image in accordance with said second object detecting information.