Vehicle environment monitoring system

• US 6,789,015 B2
• Filed: 03/20/2002
• Issued: 09/07/2004
• Est. Priority Date: 03/30/2001
• Status: Expired due to Fees

• Current Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Original Assignee: Honda Giken Kogyo Kabushiki Kaisha (Honda Motor Company)
• Inventors: Tsuji, Takayuki, Nagaoka, Nobuharu, Hattori, Hiroshi, Watanabe, Masahito
• Primary Examiner(s): Black, Thomas G.
• Assistant Examiner(s): Hernandez, Olga
• Application Number:
  US10/102,572
• Publication Number:
  US 20020183929A1
• Time in Patent Office: 902 Days
• Field of Search: 701/301, 701/96, 701/93, 701/36, 180/170, 340/425.5, 340/435, 340/436
• US Class Current: 701/301
• CPC Class Codes: B60R 21/013 including means for detecting collisions, impending collisions or roll-over
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: G01S 11/12 using electromagnetic waves other than radio waves
• CPC Class Codes: G06T 7/20 Analysis of motion motion estimation for coding, decoding, compressing or decompressing digital video signals H04N19/43, H04N19/51
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

A vehicle environment monitoring system for an automotive vehicle, is capable of accurately detecting the movement of an object existing in an external environment of the vehicle, and determining the probability of collision between the object and the vehicle, thereby appropriately warning the driver. A running speed of the vehicle is detected, and a distance between the object and the vehicle is calculated, based on the image obtained by the imaging device. A relative speed between the object and the vehicle is calculated according to the calculated distance. It is determined whether or not there is a significant probability that the vehicle may collide with the object, based on the running speed of the vehicle and the distance therebetween on condition that the relative speed is in the vicinity of the running speed of the vehicle.

Claims

1. A vehicle environment monitoring system for monitoring an external environment proximate to a vehicle, and for detect an object existing in the external environment of said vehicle from an image obtained by imaging means mounted on said vehicle, the vehicle environment monitoring system comprising:

vehicle speed detecting means for detecting a running speed of said vehicle;

distance calculating means for calculating a distance between said abject and said vehicle based on the image obtained by said imaging means;

relative speed calculating means for calculating a relative speed between said object and said vehicle according to the distance calculated by said distance calculating means; and

probability determining means for determining whether or not there is a significant probability that said vehicle will collide with said object, based on the running speed of said vehicle and the distance calculated by said distance calculating means, on condition that an absolute value of a difference between the relative speed and the running speed of said vehicle is less than or equal to half of the running speed of said vehicle.

2. A vehicle environment monitoring system according to claim 1, wherein said probability determining means carries out the determination by applying collision determination conditions that are dependent on a width of said vehicle.

3. A vehicle environment monitoring system according to claim 1, wherein said imaging means comprises two infrared cameras adapted to receive and detect infrared rays.

4. A vehicle environment monitoring system according to claim 1, wherein said imaging means comprises two TV cameras for detecting infrared rays or visible rays,

and wherein said relative position-detecting means includes search area-setting means for setting, based on a position of an object image contained in an image output from one of said two TV cameras, a search area within an image output from the other of said two TV cameras for searching for a corresponding object image contained in the image output from the other of said two TV cameras; and

corresponding object image-identifying means for identifying the corresponding object image by carrying out a correlation operation on data within said search area, said distance-calculating means calculating the distance between said vehicle and said object based on a parallax between the object image and the corresponding object image.

5. A vehicle environment monitoring system according to claim 1, further comprising warning means for warning a driver of said vehicle, when said probability determining means determines that the probability of collision against said object is high, wherein said warning means is inhibited from warning the driver when the driver is carrying out a braking operation and at the same time deceleration caused by the braking operation is larger than a predetermined threshold.

6. A vehicle environment monitoring system according to claim 1, further comprising

relative position-detecting means for detecting a relative position of said object to said vehicle, based on the image obtained by said imaging means and the distance calculated by said distance calculating means, to thereby obtain position data;

and

movement vector-calculating means for calculating positions of said object in a real space based on a plurality of time series items of the position data detected on said object by said relative position-detecting means, and calculating a movement vector of said object based an the positions in said real space;

wherein said probability determining means determines whether or not the probability of collision is high based on the movement vector, when it is determined based on the running speed of said vehicle and the distance that there is a probability of collision.

7. A vehicle environment monitoring system according to claim 6, wherein said movement vector-calculating means includes

approximate straight line-calculating means for calculating an approximate straight line approximating a locus of relative movement of said object, and

position data-correcting means for correcting the time series items of the position data by using the approximate straight line,

said movement vector-calculating means calculating the movement vector based on the corrected time series items of the position data.

8. A vehicle environment monitoring system according to claim 7, wherein said approximate straight line-calculating means calculates the approximate straight line as a three-dimensional straight line including data of a height of said object.

9. A vehicle environment monitoring method for monitoring an external environment proximate to a vehicle, in which an object existing in the external environment of said vehicle is detected from an image obtained by an imaging device mounted on said vehicle, said method comprising the steps of:

a) detecting a running speed of said vehicle;

b) calculating a distance between said object and said vehicle based on the image obtained by said imaging device;

c) calculating a relative speed between said object and said vehicle according to the calculated distance; and

d) determining whether or not there is a significant probability that said vehicle will collide with said object, based on the running speed of said vehicle and the calculated distance, on condition that an absolute value of a difference between the relative speed and the running speed of said vehicle is less than or equal to half of the running speed of said vehicle.

10. A vehicle environment monitoring method according to claim 9, wherein the probability determination is carried out by applying collision determination conditions that are dependent on a width of said vehicle.

11. A vehicle environment monitoring method according to claim 9, wherein said imaging device comprises two infrared cameras adapted to receive and detect infrared rays.

12. A vehicle environment monitoring method according to claim 9, wherein said imaging device comprises two TV cameras for detecting infrared rays or visible rays, and said step of detecting a relative position of said object to said vehicle includes the steps of:

l) setting, based on a position of an object image contained in an image output from one of said two TV cameras, a search area within an image output from the other of said two TV cameras for searching for a corresponding object image contained in the image output from the other of said two TV cameras; and

m) identifying the corresponding object image by carrying out a correlation operation on data within said search area, the distance between said vehicle and said object being calculated based on a parallax between the object image and the corresponding object image.

13. A vehicle environment monitoring method according to claim 9, further comprising the step of warning a driver of said vehicle, when said determining step establishes that the probability of collision against said object is high, wherein the step of warning is omitted when the driver is carrying out a braking operation and at the same time deceleration caused by the braking operation is larger than a predetermined threshold.

14. A vehicle environment monitoring method according to claim 9, further comprising the steps of

e) detecting a relative position of said object to said vehicle based on the image obtained by said imaging device and the calculated distance, to thereby obtain position data;

f) calculating positions of said object in a real space based on a plurality of time series items of the position data detected on said object;

g) calculating a movement vector of said object based on the positions in said real space; and

h) determining whether or not the probability of collision is high based on the movement vector, when it is determined based on the running speed of said vehicle and the distance that there is a probability of collision.

15. A vehicle environment monitoring method according to claim 14, wherein said step of calculating the movement vector includes the steps of:

j) calculating an approximate straight line approximating a locus of relative movement of said object; and

k) correcting the time series items of the position data by using the approximate straight line, said movement vector being calculated based on the corrected time series items of the position data.

16. A vehicle environment monitoring method according to claim 15, wherein the approximate straight line is calculated as a three-dimensional straight line including data of a height of said object.

17. A vehicle environment monitoring system for monitoring an external environment proximate to a vehicle, and for detecting an object existing in an environment of said vehicle from an image obtained by an imaging device mounted on said vehicle, the vehicle environment monitoring system comprising:

a vehicle speed sensor for detecting a running speed of said vehicle;

a distance calculating module for calculating a distance between said object and said vehicle based on the image obtained by said imaging device;

a relative speed calculating module for calculating a relative speed between said object and said vehicle according to the distance calculated by said distance calculating module; and

a probability determining module for determining whether or not there is a significant probability that said vehicle will collide with said object, based on the running speed of said vehicle and the distance calculated by said distance calculating module, on condition that an absolute value of a difference between the relative speed and the running speed of said vehicle is less than or equal to half of the running speed of said vehicle.

18. A vehicle environment monitoring system according to claim 17, wherein said imaging device comprises two infrared cameras adapted to receive and detect infrared rays.

19. A vehicle environment monitoring system according to claim 17, wherein said imaging device comprises two TV cameras for detecting infrared rays or visible rays, and wherein said relative position-detecting module includes a search area-setting module for setting, based on a position of an object image contained in an image output from one of said two TV cameras, a search area within an image output from the other of said two TV cameras for searching for a corresponding object image contained in the image output from the other of said two TV cameras;

and a corresponding object image-identifying module for identifying the corresponding object image by carrying out a correlation operation on data within said search area, said distance-calculating module calculating the distance between said vehicle and said object based on a parallax between the object image and the corresponding object image.

20. A vehicle environment monitoring system according to claim 17, further comprising a warning module for warning a driver of said vehicle, when said probability determining module determines that the probability of collision against said object is high, wherein said warning module is inhibited from warning the driver when the driver is carrying out a braking operation and at the same time deceleration caused by the braking operation is larger than a predetermined threshold.

21. A vehicle environment monitoring system according to claim 17, further comprising a relative position-detecting module for detecting a relative position of said object to said vehicle based on the image obtained by said imaging device and the distance calculated by said distance calculating module, to thereby obtain position data;

and a movement vector-calculating module for calculating positions of said object in a real space based on a plurality of time series items of the position data detected on said object by said relative position-detecting module, and calculating a movement vector of said object based on the positions in said real space;

wherein said probability determining module determines whether or not the probability of collision is high based on the movement vector, when said probability determining module determines, based on the running speed of said vehicle and the distance, that there is a probability of collision.

22. A vehicle environment monitoring system according to claim 21, wherein said movement vector-calculating module includes approximate straight line-calculating module for calculating an approximate straight line approximating a locus of relative movement of said object, and position data-correcting module for correcting the time series items of the position data by using the approximate straight line, said movement vector-calculating module calculating the movement vector based on the corrected time series items of the position data.

23. A vehicle environment monitoring system according to claim 22, wherein said approximate straight line-calculating module calculates the approximate straight line as a three-dimensional straight line including data of a height of said object.

24. A vehicle environment monitoring system according to claim 17, wherein said probability determining module carries out the determination by applying collision determination conditions that are dependent on a width of said vehicle.