Moving obstacle detecting device

• US 20060013480A1
• Filed: 07/13/2005
• Published: 01/19/2006
• Est. Priority Date: 07/13/2004
• Status: Active Grant

• Current Assignee: Nissan Motor Co., Ltd.
• Original Assignee: Nissan Motor Co., Ltd.
• Inventors: Sano, Yasuhito
• Granted Patent: US 7,346,191 B2
• Time in Patent Office: Days
• US Class Current: 382/168
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

An aspect of the present invention provides a moving obstacle detecting device installed in or on a moving body for detecting moving obstacles in the vicinity of the moving body, the moving obstacle detecting device that includes an image capturing unit configured and arranged to capture an image of the surroundings of the moving body, an optical flow calculating unit configured to find an optical flow corresponding to characteristic points in the image captured with the camera, a focus of expansion calculating unit configured to calculate focus or foci of expansion of the motion vectors in the optical flow found by the optical flow calculating unit, a histogram creating unit configured to create a histogram of the distribution of the focus or foci of expansion calculated by the focus of expansion calculating unit, a peak test unit configured to detect at least one moving obstacle to test peaks in the histogram, the test determining whether or not the peaks correspond to the same object if two or more peaks exist in the histogram created by the histogram creating unit.

Claims

1. A moving obstacle detecting device installed in or on a moving body for detecting moving obstacles in the vicinity of the moving body, the moving obstacle detecting device comprising:

an image capturing unit configured and arranged to capture an image of the surroundings of the moving body;

an optical flow calculating unit configured to find an optical flow corresponding to characteristic points in the image captured with the camera;

a focus of expansion calculating unit configured to calculate focus or foci of expansion of the motion vectors in the optical flow found by the optical flow calculating unit;

a histogram creating unit configured to create a histogram of the distribution of the focus or foci of expansion calculated by the focus of expansion calculating unit;

a peak test unit configured to detect at least one moving obstacle to test peaks in the histogram, said test determining whether or not the peaks correspond to the same object if two or more peaks exist in the histogram created by the histogram creating unit.

2. The moving obstacle detecting device as claimed in claim 1, wherein the histogram creating unit is configured to create the histogram of the distribution of the foci of expansion based on a predetermined axis.

3. The moving obstacle detecting device as claimed in claim 2, wherein the direction of the axis along which the histogram is created is changed in accordance with the location on the moving body where the image capturing unit is installed.

4. The moving obstacle detecting device as claimed in claim 1, wherein the histogram creating unit is configured to create the histogram of the distribution of the foci of expansion along an axis oriented in the transverse direction of the moving body based on the foci of expansion calculated by the focus of expansion calculating unit.

5. The moving obstacle detecting device as claimed in claim 1, wherein the moving body is a vehicle.

6. The moving obstacle detecting device as claimed in claim 5, wherein the optical flow calculating unit is configured to vary the time interval according to which it calculates the motion vectors based on one or more of the following pieces of information:

the angle to which the steering wheel of the vehicle is rotated, the yaw rate detected by a yaw rate sensor, the difference between the rotational speeds of the left and right wheels, the status of the turn signal (directional), and information acquired from a car navigation system comprising a road map and a GPS.

7. The moving obstacle detecting device as claimed in claim 5, wherein the focus of expansion calculating unit is configured to vary the time interval according to which it calculates the foci of expansion based on one or more of the following pieces of information:

the angle to which the steering wheel of the vehicle is rotated, the yaw rate detected by a yaw rate sensor, the difference between the rotational speeds of the left and right wheels, the status of the turn signal (directional), and information acquired from a car navigation system comprising a road map and a GPS.

8. The moving obstacle detecting device as claimed in claim 5, wherein the peak examining unit is configured to:

use a prescribed risk level to determine if the peaks correspond to the same object when two or more peaks are found to exist in the histogram of the foci of expansion created by the histogram creating unit; and

vary the risk level based on one or more of the following pieces of information;

the width of the road on which the vehicle is traveling, the existence or absence of a sidewalk the number of lanes, the time of day during which the vehicle is traveling, the weather at the time when the vehicle is traveling, and information acquired from a car navigation system comprising a road map and a GPS.

9. The moving obstacle detecting device as claimed in claim 8, wherein

the peak examining unit is configured to detect a moving obstacle(s) by determining if the highest peak among the plurality of peaks that exist is of the same distribution as each of the other peak(s).

10. The moving obstacle detecting device as claimed in claim 9, wherein

the peak examining unit is configured to separate the motion vectors of the optical flow into a group of motion vectors whose foci of expansion are included in the highest peak and a group of motion vectors whose foci of expansion are included in other peak(s), generate distributions of the foci of expansion in each of the groups of motion vectors, and calculate the probability that the distributions of the foci of expansion in the two groups of motion vectors occurred within the same population.

11. The moving obstacle detecting device as claimed in claim 10, wherein

the peak examining unit is configured such that when it calculates the probability that the distributions of the foci of expansion in two groups of motion vectors occurred within the same population, it uses the t-test to test the difference between the two groups and determines that the distributions are different when the t-value is larger than a prescribed value.

12. The moving obstacle detecting device as claimed in claim 11, wherein

said prescribed value is set to a smaller value in situations where the number of moving obstacles is large and to a larger value in situations where the number of moving obstacles is small.

13. The moving obstacle detecting device as claimed in claim 12, wherein

the prescribed value is varied based on one or more of the following pieces of information:

the width of the road on which the vehicle is traveling, the existence or absence of a sidewalk the number of lanes, the time of day during which the vehicle is traveling, the weather at the time when the vehicle is traveling, and information acquired from a car navigation system comprising a road map and a GPS.

14. A moving obstacle detecting method for detecting moving obstacles existing in the vicinity of a moving body, comprising:

(1) capturing an image of the surroundings of the moving body;

(2) finding an optical flow corresponding to characteristic points in the captured image;

(3) calculating focus or foci of expansion of the motion vectors in the optical flow;

(4) creating a histogram of the distribution of the focus or foci of expansion calculated; and

(5) detecting at least one moving obstacle to test peaks in the created histogram to determine whether or not the peaks correspond to the same object determine if two or more peaks exist in the histogram of the foci of expansion created.

15. The moving obstacle detecting method as claimed in claim 14, wherein the step (4) creates the histogram of the distribution of the foci of expansion based on a predetermined axis.

16. The moving obstacle detecting method as claimed in claim 14, wherein the step (4) creates the histogram of the distribution of the foci of expansion along an axis oriented in the transverse direction of the moving body based on the foci of expansion calculated.

17. The moving obstacle detecting method as claimed in claim 14, wherein the step (2) varies the time interval according to which it calculates the motion vectors based on one or more of the following pieces of information:

the angle to which the steering wheel of the moving body is rotated, the yaw rate detected by a yaw rate sensor, the difference between the rotational speeds of the left and right wheels, the status of the turn signal (directional), and information acquired from a car navigation system comprising a road map and a GPS.

18. The moving obstacle detecting method as claimed in claim 14, wherein the step (3) varies the time interval according to which it calculates the foci of expansion based on one or more of the following pieces of information:

the angle to which the steering wheel of the moving body is rotated, the yaw rate detected by a yaw rate sensor, the difference between the rotational speeds of the left and right wheels, the status of the turn signal (directional), and information acquired from a car navigation system comprising a road map and a GPS.

19. The moving obstacle detecting method as claimed in claim 14, wherein the step (5) uses a prescribed risk level to determine if the peaks correspond to the same object when two or more peaks are found to exist in the histogram of the foci of expansion created;

and

varies the risk level based on one or more of the following pieces of information;

the width of the road on which the vehicle is traveling, the existence or absence of a sidewalk the number of lanes, the time of day during which the vehicle is traveling, the weather at the time when the vehicle is traveling, and information acquired from a car navigation system comprising a road map and a GPS.

20. The moving obstacle detecting method as claimed in claim 19, wherein step (5) detects a moving obstacle(s) by determining if the highest peak among the plurality of peaks that exist is of the same distribution as each of the other peak(s).