Periodic stationary object detection system and periodic stationary object detection method

• US 8,903,133 B2
• Filed: 02/17/2012
• Issued: 12/02/2014
• Est. Priority Date: 02/21/2011
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Tsuchiya, Chikao, Hayakawa, Yasuhisa, Tanaka, Shinya, Furushou, Hiroyuki, Fukata, Osamu
• Primary Examiner(s): Seth, Manav
• Application Number:
  US13/821,918
• Publication Number:
  US 20130322688A1
• Time in Patent Office: 1,019 Days
• US Class Current: 382/104
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/246 using feature-based methods, e.g. the tracking of corners or segments
• CPC Class Codes: G06V 10/421 by analysing segments intersecting the pattern
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A periodic stationary object detection system extracts a feature point of a three-dimensional object from image data on a predetermined region of a bird'"'"'s eye view image for each of multiple sub regions included in the predetermined region, calculates waveform data corresponding to a distribution of the feature points in the predetermined region on the bird'"'"'s eye view image, and judges whether or not the three-dimensional object having the extracted feature point is a periodic stationary object candidate on the basis of whether or not peak information of the waveform data is equal to or larger than a predetermined threshold value.

Claims

1. A periodic stationary object detection system for detecting a periodic stationary object in surroundings of a moving object, comprising:

an image-capturing device mounted on the moving object and capable of capturing an image of the surroundings of the moving object;

a viewpoint converting unit configured to perform viewpoint conversion on the image captured by the image-capturing device to create a bird'"'"'s eye view image;

a feature point-extracting unit configured to extract a feature point of a three-dimensional object from image data on a predetermined region of the bird'"'"'s eye view image for each of a plurality of sub regions included in the predetermined region;

a waveform data-calculating unit configured to calculate waveform data corresponding to a distribution of the feature points extracted by the feature point-extracting unit in the predetermined region on the bird'"'"'s eye view image;

a peak information detecting unit configured to detect peak information of the waveform data;

a periodic stationary object candidate-detecting unit configured to judge whether or not the three-dimensional object having the feature point extracted by the feature point-extracting unit is a periodic stationary object candidate on a basis of whether or not the peak information is equal to or larger than a predetermined first threshold value; and

a periodic stationary object judging unit configured to determine that the periodic stationary object candidate is a periodic stationary object when the periodic stationary object candidate is detected by the periodic stationary object candidate-detecting unit and the detection is made under a predetermined condition.

2. The periodic stationary object detection system according to claim 1, wherein

the feature point-extracting unit extracts edge elements present along each of a plurality of vertical virtual lines extending in a vertical direction in an actual space,

the waveform data-calculating unit sums the number of the edge elements extracted by the feature point-extracting unit for each of the plurality of vertical virtual lines, and calculates an edge distribution waveform on a basis of the number of the edge elements thus summed,

the peak information detecting unit is a counting unit configured to detect peaks of the edge distribution waveform calculated by the waveform data-calculating unit, and to count the number of the peaks thus detected, and

the periodic stationary object candidate-detecting unit determines that the three-dimensional object having the feature point extracted by the feature point-extracting unit is a periodic stationary object candidate when the number of the peaks counted by the counting unit is equal to or larger than a predetermined second threshold value.

3. The periodic stationary object detection system according to claim 2, wherein the counting unit detects peaks of the edge distribution waveform calculated by the waveform data-calculating unit, and counts the number of peaks aligned at equal intervals among the peaks thus detected.

4. The periodic stationary object detection system according to claim 1, wherein the periodic stationary object judging unit determines that the periodic stationary object candidate is a periodic stationary object when the periodic stationary object candidate is continuously detected for a predetermined time period.

5. The periodic stationary object detection system according to claim 2, further comprising:

a speed detector configured to detect a traveling speed of the moving object;

an alignment unit configured to align a position of an edge distribution waveform at a first time calculated by the waveform data-calculating unit with a position of an edge distribution waveform at a second time different from the first time on a basis of the traveling speed detected by the speed detector; and

a difference calculating unit configured to calculate a difference distribution waveform between the edge distribution waveform at the second time and the edge distribution waveform aligned by the alignment unit, wherein

the periodic stationary object judging unit

integrates the difference distribution waveform calculated by the difference calculating unit to calculate a first integrated value,

integrates any one of the edge distribution waveform at the first time, the edge distribution waveform at the second time, and the edge distribution waveform aligned by the alignment unit to calculate a second integrated value,

calculates a ratio of the first integrated value to the second integrated value, and

determines whether or not the periodic stationary object candidate is stationary on a basis of whether or not a value of the ratio is smaller than a predetermined third threshold value.

6. The periodic stationary object detection system according to claim 2, further comprising:

a speed detector configured to detect a traveling speed of the moving object;

an alignment unit configured to align a position of an edge distribution waveform at a first time calculated by the waveform data-calculating unit with a position of an edge distribution waveform at a second time different from the first time on a basis of the traveling speed detected by the speed detector; and

a difference calculating unit configured to calculate a first difference distribution waveform between the edge distribution waveform at the second time and the edge distribution waveform aligned by the alignment unit, and a second difference distribution waveform between the edge distribution waveform at the first time and the edge distribution waveform at the second time, wherein

the periodic stationary object-judging unit

integrates the first difference distribution waveform to calculate a first integrated value,

integrates the second difference distribution waveform to calculate a second integrated value,

calculates a ratio of the first integrated value to the second integrated value, and

determines whether or not the periodic stationary object candidate is stationary on a basis of whether or not a value of the ratio is smaller than a predetermined fourth threshold value.

7. The periodic stationary object detection system according to claim 5, wherein the periodic stationary object judging unit determines that the periodic stationary object candidate is a periodic stationary object when continuously detecting that the periodic stationary object candidate is stationary for a predetermined time period.

8. The periodic stationary object detection system according to claim 1, further comprising:

a speed detector configured to detect a traveling speed of the moving object; and

a travel range-calculating unit configured to calculate a travel range of periodic stationary objects on the bird'"'"'s eye view image on a basis of a capturing interval of the image-capturing device and the traveling speed detected by the speed detector, wherein

the feature point-extracting unit includes;

an alignment unit configured to align positions of image data at different times created by the viewpoint converting unit;

a three-dimensional object detecting unit configured to detect a plurality of three-dimensional objects on a basis of data on a difference image of the image data at the different times aligned by the alignment unit; and

a travel amount candidate-calculating unit configured to calculate travel amount candidates of the plurality of three-dimensional objects detected by the three-dimensional object detecting unit,

the waveform data-calculating unit includes a counting unit configured to count the travel amount candidates calculated by the travel amount candidate-calculating unit and to generate a histogram as the waveform data,

when values of peaks of the histogram generated by the counting unit are equal to or larger than a predetermined fifth threshold value, the periodic stationary object candidate-detecting unit determines that the three-dimensional objects linked to travel amount candidates corresponding to the peaks are periodic stationary object candidates, and

the periodic stationary object-judging unit judges that the periodic stationary object candidates are periodic stationary objects when the peaks of the histogram are present within the travel range calculated by the travel range-calculating unit.

9. The periodic stationary object detection system according to claim 8, wherein the travel amount candidate-calculating unit calculates a plurality of the travel amount candidates for each of the three-dimensional objects.

10. The periodic stationary object detection system according to claim 8, wherein the periodic stationary object candidate-detecting unit sets the predetermined fifth threshold value obtained from a maximum value of the peaks.

11. The periodic stationary object detection system according to claim 8, further comprising a periodicity determining unit configured to judge if there is a periodicity from positions where the peaks appear, and to lower the predetermined fifth threshold value when judging that there is a periodicity.

12. The periodic stationary object detection system according to claim 11, wherein the periodicity determining unit judges if there is a periodicity from positions where peaks appear, the peaks having values equal to or larger than the predetermined fifth threshold value set from the maximum value of the peaks.

13. The periodic stationary object detection system according to claim 11, further comprising a lateral movement-detecting unit configured to detect a lateral movement of the moving object, and

when the lateral movement-detecting unit detects a lateral movement of a specified distance or longer and the predetermined fifth threshold value has been lowered, the periodicity determining unit initializes the lowered fifth threshold value.

14. The periodic stationary object detection system according to claim 8, wherein

the periodic stationary object-judging unit

stores positions where peaks appear and peak values thereof when it is judged that there is a periodicity in previous processing,

judges that the periodic stationary object candidates are periodic stationary objects in a case of detecting a peak having a peak value equal to or larger than the predetermined fifth threshold value other than the peaks judged to have a periodicity in current processing, when an average value of the peak values judged to have the periodicity in the current processing is not smaller than an average value of the peak values judged to have the periodicity in the previous processing by a predetermined value or larger, and

judges that moving objects are present when the average value of the peak values judged to have the periodicity in the current processing is smaller than the average value of the peak values judged to have the periodicity in the previous processing by the predetermined value or larger.

15. A periodic stationary object detection method for detecting periodic stationary objects in surrounding of a moving object, comprising:

an image-capturing step of capturing an image of the surroundings of the moving object using an image-capturing device mounted on the moving object;

a viewpoint converting step of performing viewpoint conversion on the image captured by the image-capturing device to create a bird'"'"'s eye view image;

a feature point-extracting step of extracting a feature point of a three-dimensional object from image data on a predetermined region of the bird'"'"'s eye view image for each of a plurality of sub regions included in the predetermined region;

a waveform data-calculation step of calculating waveform data corresponding to a distribution of the feature points extracted in the feature point-extracting step in the predetermined region on the bird'"'"'s eye view image;

a peak information-detecting step of detecting peak information of the waveform data;

a periodic stationary object candidate-detecting step of judging whether or not the three-dimensional object having the feature point extracted in the feature point-extracting step is a periodic stationary object candidate on a basis of whether or not the peak information is equal to or larger than a predetermined first threshold value; and

a periodic stationary object-judging step of determining that the periodic stationary object candidate is a periodic stationary object when the periodic stationary object candidate is detected in the periodic stationary object candidate-detecting step and the detection is made under a predetermined condition.

16. A periodic stationary object detection system for detecting a periodic stationary object in surroundings of a moving object, comprising:

image-capturing means for capturing an image of the surroundings of the moving object, the image-capturing means being mounted on the moving object;

viewpoint converting means for performing viewpoint conversion on the image captured by the image-capturing means to create a bird'"'"'s eye view image;

feature point-extracting means for extracting a feature point of a three-dimensional object from image data on a predetermined region of the bird'"'"'s eye view image for each of a plurality of sub regions included in the predetermined region;

waveform data-calculating means for calculating waveform data corresponding to a distribution of the feature points extracted by the feature point-extracting means in the predetermined region on the bird'"'"'s eye view image;

peak information detecting means for detecting peak information of the waveform data;

periodic stationary object candidate-detecting means for judging whether or not the three-dimensional object having the feature point extracted by the feature point-extracting means is a periodic stationary object candidate on a basis of whether or not the peak information is equal to or larger than a predetermined first threshold value; and

periodic stationary object judging means for determining that the periodic stationary object candidate is a periodic stationary object when the periodic stationary object candidate is detected by the periodic stationary object candidate-detecting means and the detection is made under a predetermined condition.