Object detection system

• US 8,094,934 B2
• Filed: 03/05/2008
• Issued: 01/10/2012
• Est. Priority Date: 03/07/2007
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Morimitsu, Noboru
• Primary Examiner(s): Liew, Alex
• Application Number:
  US12/042,831
• Publication Number:
  US 20080219505A1
• Time in Patent Office: 1,406 Days
• Field of Search: 382/103, 382168-172, 382173-180
• US Class Current: 382/168
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 20/64 Three-dimensional objects
• CPC Class Codes: G06V 2201/08 Detecting or categorising vehicles

Abstract

An object detection system is provided a plurality of image capture units for capturing images of surroundings of the system, a distance information calculation unit for dividing a captured image which constitutes a reference of captured images captured by the plurality of image capture units into a plurality of pixel blocks, individually retrieving corresponding pixel positions within the other captured image for the pixel blocks, and individually calculating distance information, and a histogram generation module for dividing a range image representing the individual distance information of the pixel blocks calculated by the distance information calculation unit into a plurality of segments having predetermined sizes, providing histograms relating to the distance information for the respective divided segments, and casting the distance information of the pixel blocks to the histograms of the respective segments.

Claims

1. An object detection system comprising:

a plurality of image capture units for capturing images including a reference image and a comparing image;

a distance information calculation unit for dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information;

a histogram generation module for dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments;

an object detection module for detecting an object based on frequencies of classes to each of the histograms; and

a reliability evaluation module for calculating an index of reliability of the distance information of each of the pixel blocks,

wherein the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks, and wherein the histogram generation module changes, as the resolution for generating the histogram, at least one of the size of the segments which divide the range image, a size of class divisions of the histogram, and an extent of a distribution range in voting by distributing the distance information of the pixel blocks to the classes of the histogram and classes which surround the classes of the histogram according to the index of reliability of the distance information of each of the pixel blocks.

7. The object detection system according to claim 1, wherein

the reliability evaluation module sets an evaluation window in a predetermined image area in the range image and calculates an index of reliability of the distance information of each of the pixel blocks based on a statistical value of the distance information of each of the pixel blocks within the evaluation window.

8. The object detection system according to claim 1, wherein

the reliability evaluation module sets an evaluation window in an image area occupied by an object detected by the object detection module on the captured image which constitutes the reference and calculates an index of reliability of the distance information of each of the pixel blocks based on a statistical value of the distance information of each of the pixel blocks within the evaluation window so set.

9. The object detection system according to claim 1, wherein

the object detection module changes a threshold value for detection of the object based on the index of reliability of the distance information of each of the pixel blocks which is calculated by the reliability evaluation module.

2. An object detection system comprising:

a plurality of image capture units for capturing images including a reference image and a comparing image;

a distance information calculation unit for dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information;

a histogram generation module for dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments;

an object detection module for detecting an object based on frequencies of classes to each of the histograms; and

a reliability evaluation module for calculating an index of reliability of the distance information of each of the pixel blocks, wherein

the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks,

the histogram generation module further generates a second histogram for the respective segments based on the frequencies of the classes of the histogram implemented in predetermined filtering operations, and parameters for the predetermined filtering operations are changed according to the indexes of reliability of the distance information of the respective pixel blocks.

3. The object detection system according to claim 2, wherein

the histogram generation module implements, as the predetermined filtering operation, a temporal smoothing operation for calculating a true value in the current sampling cycle by smoothing a temporal variation in the frequency of each class in each class of each histogram, and generates the second histogram by allocating the true values of the classes to corresponding classes of the second histogram.

4. The object detection system according to claim 3, wherein

the parameter for the predetermined filtering operation which is to be changed is a time constant which is used in the temporal smoothing.

5. The object detection system according to claim 2, wherein

when the second histogram is arranged so that the corresponding classes are aligned into a straight line, in allocating the frequencies of the classes of the histogram to the corresponding classes of the second histogram, as the predetermined filtering operation, the histogram generation module implements a spatial smoothing operation for executing the allocation by distributing the frequencies to the class and classes of the second histogram to which the class belongs and the second histogram surrounding the second histogram of the class which surround the class of interest according to a predetermined distribution, so as to generate the second histogram.

6. The object detection system according to claim 5,

wherein the parameter for the predetermined filtering operation which is to be changed is a distribution ratio for the classes which surrounds the classes in the spatial smoothing operation.

10. A method for detecting an object comprising:

capturing a plurality of images with a plurality of image capture units including a reference image and a comparative image;

dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information from a distance information calculation unit;

dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments with a histogram generation module;

detecting an object based on frequencies of classes to each of the histograms with an object detection module; and

calculating an index of reliability of the distance information of each of the pixel blocks with a reliability evaluation module,

wherein, in generating a histogram, the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks, and wherein the histogram generation module changes, as the resolution for generating the histogram, at least one of the size of the segments which divide the range image, a size of class divisions of the histogram, and an extent of a distribution range in voting by distributing the distance information of the pixel blocks to the classes of the histogram and classes which surround the classes of the histogram according to the index of reliability of the distance information of each of the pixel blocks.

11. A method for detecting an object comprising:

capturing a plurality of images with a plurality of image capture units including a reference image and a comparative image;

dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information from a distance information calculation unit;

dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments with a histogram generation module;

detecting an object based on frequencies of classes to each of the histograms with an object detection module; and

calculating an index of reliability of the distance information of each of the pixel blocks with a reliability evaluation module, wherein, in generating a histogram, the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks, and

wherein, the histogram generation module further generates a second histogram for the respective segments based on the frequencies of the classes of the histogram implemented in predetermined filtering operations, and parameters for the predetermined filtering operations are changed according to the indexes of reliability of the distance information of the respective pixel blocks.

12. An object detection system comprising:

a plurality of image capture units for capturing images including a reference image and a comparing image;

a distance information calculation unit for dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information;

a histogram generation module for dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments;

an object detection module for detecting an object based on frequencies of classes to each of the histograms; and

a reliability evaluation module for calculating an index of reliability of the distance information of each of the pixel blocks,

wherein the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks, and

wherein the reliability evaluation module sets an evaluation window in a predetermined image area in the range image and calculates an index of reliability of the distance information of each of the pixel blocks based on a statistical value of the distance information of each of the pixel blocks within the evaluation window.

13. An object detection system comprising:

a plurality of image capture units for capturing images including a reference image and a comparing image;

a distance information calculation unit for dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information;

a histogram generation module for dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments;

an object detection module for detecting an object based on frequencies of classes to each of the histograms; and

a reliability evaluation module for calculating an index of reliability of the distance information of each of the pixel blocks, wherein the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks, and

wherein the reliability evaluation module sets an evaluation window in an image area occupied by an object detected by the object detection module on the captured image which constitutes the reference and calculates an index of reliability of the distance information of each of the pixel blocks based on a statistical value of the distance information of each of the pixel blocks within the evaluation window so set.

14. An object detection system comprising:

a plurality of image capture units for capturing images including a reference image and a comparing image;

a distance information calculation unit for dividing the reference image into a plurality of pixel blocks, searching corresponding pixel positions within the comparative image for each of the pixel blocks, calculating distance information for each of the pixel blocks, and forming a range image representing distance information;

a histogram generation module for dividing the range image into a plurality of segments having predetermined sizes, and generating a histogram relating to the distance information by voting the distance information of the pixel blocks to the histogram of the respective segments;

an object detection module for detecting an object based on frequencies of classes to each of the histograms; and

a reliability evaluation module for calculating an index of reliability of the distance information of each of the pixel blocks, wherein the histogram generation module changes a resolution for generating the histogram according to the index of reliability of the distance information of each of the pixel blocks, and

wherein the object detection module changes a threshold value for detection of the object based on the index of reliability of the distance information of each of the pixel blocks which is calculated by the reliability evaluation module.