Object detecting system

• US 8,174,563 B2
• Filed: 10/29/2008
• Issued: 05/08/2012
• Est. Priority Date: 10/29/2007
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Saito, Toru
• Primary Examiner(s): Mesfin, Yemane
• Application Number:
  US12/260,640
• Publication Number:
  US 20090237491A1
• Time in Patent Office: 1,287 Days
• Field of Search: 348/47, 348/135, 348169-172, 382/103, 382/104, 382/106, 382/154
• US Class Current: 348/47
• CPC Class Codes: G01C 11/06 by comparison of two or more pictures of the same area
• CPC Class Codes: G06V 10/10 Image acquisition document image scanning and transmission H04N1/00; control of digital cameras H04N23/60
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 2201/12 Acquisition of 3D measurements of objects
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An object detecting system includes stereo-image taking means for taking images of an object and outputting the images as a reference image and a comparative image, stereo matching means for calculating parallaxes by stereo matching, and determination means for setting regions of objects in the reference image on the basis of the parallaxes grouped by grouping means, and performing stereo matching again for an area on the left side of a comparative pixel block specified on an epipolar line in the comparative image corresponding to a reference pixel block in a left end portion of each region. When a comparative pixel block that is different from the specified comparative pixel block and provides the local minimum SAD value less than or equal to a threshold value is detected, the determination means determines that the object in the region is mismatched.

Claims

1. An object detecting system comprising:

stereo-image taking means for simultaneously taking images of an object by a pair of right and left cameras mounted at the same height and outputting the images as a reference image and a comparative image;

stereo matching means for performing stereo matching so that a reference pixel block having a predetermined number of pixels is set in the reference image and differences in luminance pattern between the reference pixel block and comparative pixel blocks are calculated according to a predetermined calculation expression, the comparative pixel blocks having the same shape as that of the reference pixel block on an epipolar line in the comparative image and being corresponding to the reference pixel block, and so that a comparative pixel block having the smallest difference is specified, the stereo matching means calculating a parallax from a position of the specified comparative pixel block in the comparative image and a position of the reference pixel block in the reference image, the parallax being calculated for each reference pixel block;

grouping means for grouping the parallaxes regarded as corresponding to the same object on the basis of the parallaxes of the reference pixel blocks; and

determination means for detecting objects in the reference image on the basis of the grouped parallaxes and setting regions including images of the objects, selecting a reference pixel block belonging to a left end portion or a right end portion of each of the regions, performing stereo matching again for an area on the right or left side of a comparative pixel block, which is specified in the comparative image corresponding to the reference pixel block, on an epipolar line including the specified comparative pixel block, and determining that the object in the region is mismatched when a comparative pixel block that is different from the specified comparative pixel block and provides the minimum difference less than or equal to a preset threshold value is detected.

2. The object detecting system according to claim 1, wherein the predetermined calculation expression is provided to calculate, as the difference, a SAD value of a luminance for all pixels in the reference pixel block and the comparative pixel block according to the following expression:

3. The object detecting system according to claim 1,

wherein the determination means performs stereo matching again for all of or a plurality number of reference pixel blocks, of the reference pixel blocks belonging to the right or left end portion of the region including each of the objects detected in the reference image, and

wherein the determination means judges the object mismatched when positions of the comparative pixel blocks that correspond to the reference pixel blocks and provide the minimum differences less than or equal to the threshold value concentrate in the comparative image.

4. The object detecting system according to claim 1, wherein the determination means removes the object judged mismatched from the detected objects.

5. The object detecting system according to claim 1,

wherein, by the stereo matching performed again on the epipolar line including the comparative pixel block specified in the comparative image corresponding to the reference pixel block belonging to the right or left end portion of the region including the object judged mismatched, the determination means calculates a parallax from a position in the comparative image of the comparative pixel block that is different from the specified comparative pixel block and provides the minimum difference less than or equal to the threshold value and a position in the reference image of the reference pixel block, and

wherein, when the object judged mismatched exists at a distance in real space calculated from the calculated parallax and when the distance coincides with a distance in real space of any other detected object within a predetermined error range, the determination means determines that the object judged mismatched is the same as the any other detected object.

6. The object detecting system according to claim 5, wherein, when a plurality of comparative pixel blocks that are different from the specified comparative pixel block and provide the minimum differences less than or equal to the threshold value exist, the judgment of coincidence is made for all comparative pixel blocks except the specified pixel block.

7. The object detecting system according to claim 5, wherein the any other detected object judged for coincidence is an object detected near the object judged mismatched in the reference image.

8. The object detecting system according to claim 1, wherein, when information about a comparative pixel block that provides the minimum difference less than or equal to the threshold value and is obtained by stereo matching performed again for an object judged mismatched in the current sampling period includes information close to information about a comparative pixel block that provides the minimum difference less than or equal to the threshold value and is obtained by stereo matching performed again for an object detected in the previous sampling period corresponding to the object judged mismatched, the determination means improves reliability of determination in the current sampling period that the object is mismatched.

9. The object detecting system according to claim 1, wherein the determination means increases or decreases the luminance of the reference image and/or the comparative image on the basis of the calculated difference.