Environment recognition system

• US 8,437,536 B2
• Filed: 01/23/2009
• Issued: 05/07/2013
• Est. Priority Date: 01/25/2008
• Status: Active Grant

• Current Assignee: Subaru Corp.
• Original Assignee: Fuji Heavy Industries Limited (Subaru Corp.)
• Inventors: Saito, Toru
• Primary Examiner(s): Liew, Alex
• Application Number:
  US12/320,353
• Publication Number:
  US 20090190827A1
• Time in Patent Office: 1,565 Days
• Field of Search: 382/103, 382/154
• US Class Current: 382/154
• CPC Class Codes: G06T 7/593 from stereo images
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An environment recognition system includes image taking means for taking a pair of images of an object in a surrounding environment with a pair of cameras and outputting the pair of images, stereo matching means for conducting stereo matching on a plurality of pairs of images that are taken by different image taking methods or that are formed by subjecting the pair of taken images to different image processing methods and forming distance images respectively for the pairs of images, selection means for dividing the distance images into a plurality of sections, calculating representative parallaxes respectively for the sections, and selecting any of the representative parallaxes of the corresponding section as a representative parallax of the section, and detection means for detecting the object in the image on the basis of the representative parallaxes of the sections.

Claims

1. An environment recognition system comprising:

image taking means for taking a pair of images of the same object in a surrounding environment with a pair of cameras and outputting the pair of images;

stereo matching means for conducting stereo matching on each of a plurality of pairs of images taken by different image taking methods or each of a plurality of pairs of images formed by subjecting a pair of taken images to different image processing methods, and for forming distance images in a one-to-one correspondence with the plurality of pairs of images by assigning calculated parallaxes to pixel blocks in the images;

selection means for dividing the distance images into a plurality of vertical strip sections, calculating representative parallaxes respectively for the sections, selecting any of the representative parallaxes of corresponding sections in the distance images, and selecting the representative parallax for each of the sections; and

detection means for detecting the object in the images on the basis of the representative parallaxes of the sections.

2. The environment recognition system according to claim 1, wherein the plurality of pairs of images taken by the different image taking methods are a plurality of pairs of images taken by changing an image taking condition of the image taking means.

3. The environment recognition system according to claim 1, wherein the plurality of pairs of images formed by subjecting the pair of taken images to the different image processing methods are pairs of images taken by the image taking means, pairs of images obtained by subjecting the pair of taken images to edge processing, pairs of images formed by making resolutions of the pair of taken images different, or pairs of images formed by making brightnesses of the pair of taken images different.

4. The environment recognition system according to claim 1, wherein the selection means forms the pairs of images by changing a threshold value at which noise is removed from the distance images.

5. The environment recognition system according to claim 1, wherein a pixel width of the sections is different between the distance images formed on the basis of the taken pair of images, and the selection means calculates representative parallaxes respectively for the sections, selects any of the representative parallaxes of the corresponding section as a representative parallax of the section, and selects a representative parallax for each of the sections.

6. The environment recognition system according to claim 1, wherein the selection means forms a histogram for each of the sections of the distance images formed respectively for the pairs of images, and calculates the mode of the histogram as the representative parallax on the basis of frequencies of the parallaxes in the histogram.

7. The environment recognition system according to claim 6, wherein the selection means selects, from the representative parallaxes of the corresponding section in the distance images formed respectively for the pairs of images, one of a representative parallax having the highest frequency, a representative parallax based on a histogram having the smallest variance, a representative parallax having the largest mode, and a representative parallax that provides the shortest distance to the object in a current sampling period estimated from a position of the object detected in a previous sampling period, and sets the selected representative parallax as a representative parallax of the section.

8. The environment recognition system according to claim 6, wherein the selection means forms a plurality of histograms for each of the sections of the distance images, the histograms being different in at least one of the maximum value, the minimum value, and a class width, and calculates the representative parallax for each of the histograms.

10. The environment recognition system according to claim 1, wherein the selection means selects any of the representative parallaxes to which a predetermined weight is assigned.

11. The environment recognition system according to claim 1, wherein the image taking method for obtaining the pair of images, the image processing method for the pair of images, or a method for processing the distance image is switched.

12. The environment recognition system according to claim 11, wherein the image taking method, the image processing method, or the method for processing the distance image is switched in accordance with the surrounding environment.

13. The environment recognition system according to claim 1, wherein the object is a three-dimensional object existing above a datum plane.

14. The environment recognition system according to claim 1, wherein the stereo matching means and the selection means operate by using distances and representative distances uniquely correlated to the parallaxes and the representative parallaxes according to the principle of triangulation, instead of the parallaxes and the representative parallaxes.

15. The environment recognition system according to claim 1, wherein said selection means selects representative parallaxes from each of the corresponding sections in the distance images, and said detection means detects objects based on selections derived from each of said distance images.

16. The environment recognition system according to claim 15, wherein said selection means is configured to switch to different criteria based on different environmental situations faced by said image taking means.

17. The environment recognition system according to claim 1, wherein said stereo matching means conducts stereo matching on each of a plurality of pairs of images, which pairs of images are produced by respective different image taking methods.

18. The environment recognition system according to claim 1, wherein said stereo matching means conducts stereo matching on each of a plurality of pairs of images, which pairs of images are produced by subjecting the pair of taken images to respective, different image processing methods.

19. The environment recognition system according to claim 1, wherein said selection means is configured to select a first kind of representative parallax over a second kind of representative parallax at a common distance image vertical section location based on a determination the first representative parallax is superior to the second representative parallax at that distance image vertical section location under current environment condition of the image taking means.

9. An environment recognition system comprising:

image taking means for taking a pair of images of the same object in a surrounding environment with a pair of cameras and outputting the pair of images;

stereo matching means for conducting stereo matching on the pair of images and for forming a distance image by assigning calculated parallaxes to pixel blocks in the images;

selection means for dividing the distance image into a plurality of vertical strip sections, calculating a plurality of different kinds of representative parallaxes for each of the sections, and selecting any of the representative parallaxes of each of the sections as a representative parallax of the section; and

detection means for detecting the object in the images on the basis of the representative parallaxes of the sections,

wherein the selection means forms a histogram for each of the sections, calculates the mode of the parallaxes in the histogram as the representative parallax, and calculates the plurality of representative parallaxes by using a plurality of histograms that are different in at least one of the maximum value, the minimum value, and a class width, as the histogram for each of the sections.

20. An environment recognition system comprising:

image taking means for taking a pair of images of the same object in a surrounding environment with a pair of cameras and outputting the pair of images;

stereo matching means for conducting stereo matching on the pair of taken images forming one or more distance images by assigning calculated parallaxes to pixel blocks in the taken images;

selections means for dividing the one or more distance images into a plurality of vertical strips sections, calculating a plurality of different kinds of representative parallaxes for common or corresponding distance image sections, and selecting any of the different kinds of representative parallaxes of each of the sections as a representative parallax of the section; and

detection means for detecting the object in the taken images on the basis of the selected representative parallaxes of the section, and

wherein the plurality of representative parallaxes are calculated on the basis of a plurality of the distance images that are different in a threshold value at which noise is removed from the distance images, and wherein the selecting of any of the different kinds of representative parallaxes of each of the sections as a representative parallax of the section includes choosing amongst corresponding distance image sections of two distance images produced by said stereo matching means.