Vision system for vehicle

• US 8,665,079 B2
• Filed: 10/15/2012
• Issued: 03/04/2014
• Est. Priority Date: 05/03/2002
• Status: Expired due to Term

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Pawlicki, John A., McMahon, Martha A., Gibson, Joel S., Chinn, Steven G.
• Primary Examiner(s): PHAM, TOAN NGOC
• Application Number:
  US13/651,659
• Publication Number:
  US 20130063600A1
• Time in Patent Office: 505 Days
• Field of Search: 340/435, 340/436, 340/437, 701/209, 701/301
• US Class Current: 340/435
• CPC Class Codes: B60K 31/00 Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator fittings acting on two or more sub-units B60W30/14; speedometers G01P; systems or devices for controlling speed in general G05D13/00 ; in traffic anti-collision system for road vehicles G08G1/16
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60Q 1/525 automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/082 using alarm actuation
• CPC Class Codes: B60T 2201/089 using optical detection
• CPC Class Codes: B60T 2210/34 Blind spots
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2555/20 : Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2710/20 : Steering systems
• CPC Class Codes: B60W 2720/10 : Longitudinal speed
• CPC Class Codes: B60W 30/09 : Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 : Lane keeping
• CPC Class Codes: B60W 30/143 : Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 30/16 : Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 30/18 : Propelling the vehicle
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 15/025 : Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: G01S 11/12 : using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 13/867 : Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 : of land vehicles
• CPC Class Codes: G06T 2207/30244 : Camera pose
• CPC Class Codes: G06T 2207/30256 : Lane; Road marking
• CPC Class Codes: G06T 2207/30261 : Obstacle
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/174 : involving the use of two or more images
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 : of traffic signs
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/10 : Recognition assisted with metadata
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

A vision system for a vehicle includes an imaging sensor operable to capture an image data set representative of a scene occurring in the exterior field of view of the imaging sensor. A processor processes the captured image data set via an edge detection algorithm, which is applied to a reduced image data set of the captured image data set to detect edges of objects present exteriorly of the equipped vehicle and within a target zone within the exterior field of view of the imaging sensor. The reduced data set is processed more than other image data of the captured image data set in order to detect objects present exteriorly of the equipped vehicle and within the target zone. The target zone encompasses a geographic area of the exterior scene where an object of interest to the driver of the equipped vehicle is realistically expected to be present.

Claims

1. A vision system for a vehicle, said vision system comprising:

an imaging sensor disposed at a vehicle equipped with said vision system, wherein said imaging sensor comprises a plurality of photo-sensing pixels, and wherein said imaging sensor has a field of view exteriorly of the equipped vehicle, and wherein said imaging sensor is operable to capture an image data set representative of a scene occurring exteriorly of the equipped vehicle and in the exterior field of view of said imaging sensor;

a processor for processing said captured image data set via an edge detection algorithm;

wherein said edge detection algorithm is applied to a reduced image data set of said captured image data set to detect edges of objects present exteriorly of the equipped vehicle and within a target zone within the exterior field of view of said imaging sensor;

wherein said reduced data set is processed more than other image data of said captured image data set in order to detect objects present exteriorly of the equipped vehicle and within said target zone; and

wherein said target zone encompasses a geographic area of the exterior scene where an object of interest to the driver of the equipped vehicle is realistically expected to be present.

2. The vision system of claim 1, wherein said other image data is representative of other geographic areas of the exterior scene where an object of interest is unrealistically expected to be present or cannot be present.

3. The vision system of claim 1, wherein said target zone is selected in accordance with an adjustment input.

4. The vision system of claim 3, wherein said adjustment input comprises at least one of (i) an output from an ambient light sensor, (ii) a headlamp control, (iii) a manual control, (iv) a daytime driving condition for the equipped vehicle, (v) a night-time driving condition for the equipped vehicle, (vi) the equipped vehicle traveling through or along a curved section of road, (vii) an output from a separate forward facing imaging system, (viii) an external lighting condition and (ix) a mathematical adjustment of the height of the captured image such that the geometry of the target zone is adjusted.

5. The vision system of claim 1, wherein said target zone is selected responsive, at least in part, to at least one of (i) an ambient light sensor, (ii) a headlamp control and (iii) a manual control.

6. The vision system of claim 1, wherein said imaging sensor has its exterior field of view at least partially sidewardly and at least partially rearwardly relative to the equipped vehicle.

7. The vision system of claim 6, wherein said vision system comprises a lane change assist system.

8. The vision system of claim 1, wherein said imaging sensor comprises a forward facing imaging sensor and is operable to capture at least one of oncoming traffic information, leading traffic information and lane marking information.

9. The vision system of claim 1, wherein said processor is operable to provide a warning indication to a driver of the equipped vehicle in response to a condition detected by said vision system.

10. The vision system of claim 9, wherein said processor provides a first warning in response to a detection that the equipped vehicle is on course to leave a lane of a road traveled by the equipped vehicle and travel onto a shoulder of the road traveled by the equipped vehicle and a second warning in response to a detection that the equipped vehicle is on course to leave the shoulder of the road traveled by the equipped vehicle.

11. The vision system of claim 1, wherein said vision system is operable to adjust a steering direction of the equipped vehicle in response to a detection by said vision system of the equipped vehicle drifting or leaving its occupied lane.

12. The vision system of claim 1, wherein said vision system is operable to adjust a cruise control setting of the equipped vehicle in response to road or traffic conditions detected by said vision system.

13. The vision system of claim 1, wherein said vision system is selectively operable to provide image data to a remote device that is remote from the equipped vehicle, and wherein said vision system provides said image data to indicate a location and/or condition of the equipped vehicle.

14. The vision system of claim 13, wherein said remote device comprises at least one of a personal display system, a security display system and an emergency display system.

15. The vision system of claim 1, wherein said target zone is adjusted responsive to a daytime or nighttime driving condition for the equipped vehicle.

16. The vision system of claim 1, wherein said target zone is adjusted in response to a distance to a detected object.

17. The vision system of claim 1, wherein a distance to a portion of at least one detected object is approximated.

18. The vision system of claim 17, wherein said pixel location is determined relative to a target pixel which is directed generally at the horizon and along the direction of travel of the equipped vehicle.

19. The vision system of claim 17, wherein said distance is approximated in response to at least one of a pixel size, a pixel resolution and a field of view of said imaging sensor.

20. The vision system of claim 1, wherein a distance to a portion of at least one detected object is approximated in response to a pixel location of the portion in said captured image.

21. The vision system of claim 1, wherein said processor is operable to process horizontal detected edges and substantially limit processing of vertical detected edges.

22. The vision system of claim 1, wherein said processor is operable to process horizontal detected edges and substantially limit processing of vertical detected edges and reduce the amount of data to be processed downstream.

23. The vision system of claim 1, wherein said processor is operable to process detected edges which have a concentration of edges in a second reduced image data set which is representative of a particular reduced area or zone of said target zone of said captured image.

24. The vision system of claim 1, wherein said processor is operable to substantially limit processing of detected edges having an intensity that is less than a threshold intensity.

25. The vision system of claim 1, wherein said processor is operable to determine whether detected edges constitute a portion of a vehicle.

26. The vision system of claim 25, wherein said processor is operable to determine that detected edges may constitute a portion of a vehicle in response to a determination that said detected edges are associated with an ellipse or partial ellipse, said ellipse or partial ellipse being indicative of a vehicle tire.

27. The vision system of claim 1, wherein said processor is operable to determine whether detected edges constitute a portion of a vehicle in an adjacent lane.

28. The vision system of claim 1, wherein said processor is operable to track at least one detected edge between subsequent frames captured by said imaging sensor.

29. The vision system of claim 1, wherein said processor is operable to primarily process detected edges or substantially limit processing of detected edges in response to at least one of a size, location, intensity and distance of said detected edges relative to the equipped vehicle.

30. The vision system of claim 1, wherein said processor is operable to identify lane or road markings in images captured by said imaging sensor.

31. The vision system of claim 1, wherein said processor is operable to identify lane or road markings in images captured by said imaging sensor to determine road curvature of the road section that the equipped vehicle is traveling along.

32. The vision system of claim 31, wherein said processor is operable to adjust said target zone in response to an identification of a curve in the road section.

33. A vision system for a vehicle, said vision system comprising:

an imaging sensor disposed at a vehicle equipped with said vision system, wherein said imaging sensor comprises a plurality of photo-sensing pixels, and wherein said imaging sensor has a field of view exteriorly of the equipped vehicle, and wherein said imaging sensor is operable to capture an image data set representative of a scene occurring exteriorly of the equipped vehicle and in the exterior field of view of said imaging sensor;

wherein said imaging sensor comprises a CMOS sensor;

a processor for processing said captured image data set via an edge detection algorithm;

wherein said edge detection algorithm is applied to a reduced image data set of said captured image data set to detect edges of objects present exteriorly of the equipped vehicle and within a target zone within the exterior field of view of said imaging sensor;

wherein said reduced data set is processed more than other image data of said captured image data set in order to detect objects present exteriorly of the equipped vehicle and within said target zone;

wherein said target zone encompasses a geographic area of the exterior scene where an object of interest to the driver of the equipped vehicle is realistically expected to be present; and

wherein said imaging sensor comprises a forward facing imaging sensor and is operable to capture at least one of oncoming traffic information, leading traffic information and lane marking information.

34. The vision system of claim 33, wherein said target zone is selected in accordance with an adjustment input, and wherein said adjustment input comprises at least one of (i) an output from an ambient light sensor, (ii) a headlamp control, (iii) a manual control, (iv) a daytime driving condition for the equipped vehicle, (v) a night-time driving condition for the equipped vehicle, (vi) the equipped vehicle traveling through or along a curved section of road, (vii) an output from a separate forward facing imaging system, (viii) an external lighting condition and (ix) a mathematical adjustment of the height of the captured image such that the geometry of the target zone is adjusted.

35. The vision system of claim 33, wherein said processor is operable to provide a warning indication to a driver of the equipped vehicle in response to at least two conditions detected by said vision system, and wherein said processor provides a first warning in response to a detection that the equipped vehicle is on course to leave a lane of a road traveled by the equipped vehicle and travel onto a shoulder of the road traveled by the equipped vehicle and a second warning in response to a detection that the equipped vehicle is on course to leave the shoulder of the road traveled by the equipped vehicle.

36. The vision system of claim 33, wherein said vision system is operable to adjust a steering direction of the equipped vehicle in response to a detection by said vision system of the equipped vehicle drifting or leaving its occupied lane.

37. The vision system of claim 33, wherein said target zone is adjusted responsive to a daytime or nighttime driving condition for the equipped vehicle.

38. The vision system of claim 33, wherein said processor is operable to identify lane or road markings in images captured by said imaging sensor.

39. The vision system of claim 33, wherein said processor is operable to identify lane or road markings in images captured by said imaging sensor to determine road curvature of the road section that the equipped vehicle is traveling along.

40. The vision system of claim 39, wherein said processor is operable to adjust said target zone in response to an identification of a curve in the road section.

41. The vision system of claim 33, wherein at least one of (i) said processor is operable to process horizontal detected edges and substantially limit processing of vertical detected edges, (ii) said processor is operable to process horizontal detected edges and substantially limit processing of vertical detected edges and reduce the amount of data to be processed downstream, (iii) said processor is operable to process detected edges which have a concentration of edges in a second reduced image data set which is representative of a particular reduced area or zone of said target zone of said captured image, (iv) said processor is operable to substantially limit processing of detected edges having an intensity that is less than a threshold intensity, (v) said processor is operable to determine that detected edges may constitute a portion of a vehicle in response to a determination that said detected edges are associated with an ellipse or partial ellipse, said ellipse or partial ellipse being indicative of a vehicle tire, (vi) said processor is operable to track at least one detected edge between subsequent frames captured by said imaging sensor, and (vii) said processor is operable to primarily process detected edges or substantially limit processing of detected edges in response to at least one of a size, location, intensity and distance of said detected edges relative to the equipped vehicle.

42. The vision system of claim 33, wherein said other image data is representative of other geographic areas of the exterior scene where an object of interest is unrealistically expected to be present or cannot be present.

43. A vision system for a vehicle, said vision system comprising:

an imaging sensor disposed at a side of a vehicle equipped with said vision system, wherein said imaging sensor comprises a plurality of photo-sensing pixels, and wherein said imaging sensor has a field of view exteriorly of the equipped vehicle, and wherein said imaging sensor is operable to capture an image data set representative of a scene occurring exteriorly of the equipped vehicle and in the exterior field of view of said imaging sensor;

a processor for processing said captured image data set via an edge detection algorithm;

wherein said edge detection algorithm is applied to a reduced image data set of said captured image data set to detect edges of objects present exteriorly of the equipped vehicle and within a target zone within the exterior field of view of said imaging sensor;

wherein said reduced data set is processed more than other image data of said captured image data set in order to detect objects present exteriorly of the equipped vehicle and within said target zone;

wherein said target zone encompasses a geographic area of the exterior scene where an object of interest to the driver of the equipped vehicle is realistically expected to be present; and

wherein said imaging sensor has its exterior field of view at least partially sidewardly and at least partially rearwardly relative to the equipped vehicle.

44. The vision system of claim 43, wherein said vision system comprises a lane change assist system.

45. The vision system of claim 43, wherein said imaging sensor is accommodated in a side rearview mirror at the side of the equipped vehicle.

46. The vision system of claim 43, wherein said target zone is selected in accordance with an adjustment input, and wherein said adjustment input comprises at least one of (i) an output from an ambient light sensor, (ii) a headlamp control, (iii) a manual control, (iv) a daytime driving condition for the equipped vehicle, (v) a night-time driving condition for the equipped vehicle, (vi) the equipped vehicle traveling through or along a curved section of road, (vii) an output from a separate forward facing imaging system, (viii) an external lighting condition and (ix) a mathematical adjustment of the height of the captured image such that the geometry of the target zone is adjusted.

47. The vision system of claim 43, wherein said processor is operable to provide a warning indication to a driver of the equipped vehicle.

48. The vision system of claim 43, wherein said processor is operable to provide a warning indication to a driver of the equipped vehicle in response to at least two conditions detected by said vision system, and wherein said processor provides a first warning in response to a detection that the equipped vehicle is on course to leave a lane of a road traveled by the equipped vehicle and travel onto a shoulder of the road traveled by the equipped vehicle and a second warning in response to a detection that the equipped vehicle is on course to leave the shoulder of the road traveled by the equipped vehicle.

49. The vision system of claim 43, wherein said vision system is operable to adjust a steering direction of the equipped vehicle in response to a detection by said vision system of the equipped vehicle drifting or leaving its occupied lane.

50. The vision system of claim 43, wherein said target zone is adjusted responsive to a daytime or nighttime driving condition for the equipped vehicle.

51. A vision system for a vehicle, said vision system comprising:

an imaging sensor disposed at a vehicle equipped with said vision system, wherein said imaging sensor comprises a plurality of photo-sensing pixels, and wherein said imaging sensor has a field of view exteriorly of the equipped vehicle, and wherein said imaging sensor is operable to capture an image data set representative of a scene occurring exteriorly of the equipped vehicle and in the exterior field of view of said imaging sensor;

a processor for processing said captured image data set via an edge detection algorithm;

wherein said edge detection algorithm is applied to a reduced image data set of said captured image data set to detect edges of objects present exteriorly of the equipped vehicle and within a target zone within the exterior field of view of said imaging sensor;

wherein said reduced data set is processed more than other image data of said captured image data set in order to detect objects present exteriorly of the equipped vehicle and within said target zone;

wherein said target zone encompasses a geographic area of the exterior scene where an object of interest to the driver of the equipped vehicle is realistically expected to be present; and

wherein said imaging sensor comprises a forward facing imaging sensor and is operable to capture oncoming traffic information, leading traffic information and lane marking information.

52. The vision system of claim 51, wherein said other image data is representative of other geographic areas of the exterior scene where an object of interest is unrealistically expected to be present or cannot be present.

53. The vision system of claim 51, wherein said processor is operable to determine whether detected edges constitute a portion of a vehicle.

54. The vision system of claim 51, wherein a distance to a portion of at least one detected object is approximated.

55. The vision system of claim 51, wherein a distance to a portion of at least one detected object is approximated in response to a pixel location of the portion in said captured image.

56. The vision system of claim 51, wherein said processor is operable to provide a warning indication to a driver of the equipped vehicle in response to a condition detected by said vision system.

57. A vision system for a vehicle, said vision system comprising:

an imaging sensor disposed at a vehicle equipped with said vision system, wherein said imaging sensor comprises a plurality of photo-sensing pixels, and wherein said imaging sensor has a field of view exteriorly of the equipped vehicle, and wherein said imaging sensor is operable to capture an image data set representative of a scene occurring exteriorly of the equipped vehicle and in the exterior field of view of said imaging sensor;

wherein said imaging sensor comprises a CMOS sensor;

a processor for processing said captured image data set via an edge detection algorithm;

wherein said edge detection algorithm is applied to a reduced image data set of said captured image data set to detect edges of objects present exteriorly of the equipped vehicle and within a target zone within the exterior field of view of said imaging sensor;

wherein said reduced data set is processed more than other image data of said captured image data set in order to detect objects present exteriorly of the equipped vehicle and within said target zone;

wherein said target zone encompasses a geographic area of the exterior scene where an object of interest to the driver of the equipped vehicle is realistically expected to be present; and

wherein said imaging sensor comprises a forward facing imaging sensor and wherein said processor is operable to identify lane or road markings in images captured by said imaging sensor.

58. The vision system of claim 57, wherein said processor is operable to identify lane or road markings in images captured by said imaging sensor to determine road curvature of the road section that the equipped vehicle is traveling along.

59. The vision system of claim 58, wherein said processor is operable to adjust said target zone in response to an identification of a curve in the road section.

60. The vision system of claim 57, wherein said target zone is selected in accordance with an adjustment input, and wherein said adjustment input comprises at least one of (i) an output from an ambient light sensor, (ii) a headlamp control, (iii) a manual control, (iv) a daytime driving condition for the equipped vehicle, (v) a night-time driving condition for the equipped vehicle, (vi) the equipped vehicle traveling through or along a curved section of road, (vii) an output from a separate forward facing imaging system, (viii) an external lighting condition and (ix) a mathematical adjustment of the height of the captured image such that the geometry of the target zone is adjusted.

61. The vision system of claim 57, wherein said other image data is representative of other geographic areas of the exterior scene where an object of interest is unrealistically expected to be present or cannot be present.

62. The vision system of claim 57, wherein said processor is operable to determine whether detected edges constitute a portion of a vehicle.

63. The vision system of claim 57, wherein a distance to a portion of at least one detected object is approximated in response to a pixel location of the portion in said captured image.

64. The vision system of claim 57, wherein said processor is operable to provide a warning indication to a driver of the equipped vehicle in response to a condition detected by said vision system.