Object detection system for vehicle

• US 20040016870A1
• Filed: 04/30/2003
• Published: 01/29/2004
• Est. Priority Date: 05/03/2002
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Gibson, Joel S., Chinn, Steven G., McMahon, Martha A., Pawlicki, John A.
• Granted Patent: US 7,038,577 B2
• Time in Patent Office: Days
• US Class Current: 250/208.1
• CPC Class Codes: B60K 31/00 Vehicle fittings, acting on a single sub-unit only, for automatically controlling vehicle speed, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator fittings acting on two or more sub-units B60W30/14; speedometers G01P; systems or devices for controlling speed in general G05D13/00 ; in traffic anti-collision system for road vehicles G08G1/16
• CPC Class Codes: B60K 31/0008 including means for detecting potential obstacles in vehicle path
• CPC Class Codes: B60Q 1/525 automatically indicating risk of collision between vehicles in traffic or with pedestrians, e.g. after risk assessment using the vehicle sensor data
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2011/004 outside the vehicle
• CPC Class Codes: B60T 2201/08 Lane monitoring; Lane Keeping Systems
• CPC Class Codes: B60T 2201/082 using alarm actuation
• CPC Class Codes: B60T 2201/089 using optical detection
• CPC Class Codes: B60T 2210/34 Blind spots
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2554/801 Lateral distance
• CPC Class Codes: B60W 2554/802 Longitudinal distance
• CPC Class Codes: B60W 2555/20 : Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2556/50 : of positioning data, e.g. GPS [Global Positioning System] data
• CPC Class Codes: B60W 2710/20 : Steering systems
• CPC Class Codes: B60W 2720/10 : Longitudinal speed
• CPC Class Codes: B60W 30/09 : Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 : Lane keeping
• CPC Class Codes: B60W 30/143 : Speed control B60W30/16 takes precedence
• CPC Class Codes: B60W 30/16 : Control of distance between vehicles, e.g. keeping a distance to preceding vehicle
• CPC Class Codes: B60W 30/18 : Propelling the vehicle
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: B62D 15/025 : Active steering aids, e.g. helping the driver by actively influencing the steering system after environment evaluation B62D1/28 takes precedence
• CPC Class Codes: G01S 11/12 : using electromagnetic waves other than radio waves
• CPC Class Codes: G01S 13/867 : Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 : of land vehicles
• CPC Class Codes: G06T 2207/30244 : Camera pose
• CPC Class Codes: G06T 2207/30256 : Lane; Road marking
• CPC Class Codes: G06T 2207/30261 : Obstacle
• CPC Class Codes: G06T 7/13 : Edge detection
• CPC Class Codes: G06T 7/174 : involving the use of two or more images
• CPC Class Codes: G06T 7/60 : Analysis of geometric attributes
• CPC Class Codes: G06T 7/73 : using feature-based methods
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 : of traffic signs
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 2201/10 : Recognition assisted with metadata
• CPC Class Codes: G08G 1/16 : Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums

Abstract

An imaging system for a vehicle includes an imaging sensor and a control. The imaging sensor is operable to capture an image of a scene occurring exteriorly of the vehicle. The control receives the captured image, which comprises an image data set representative of the exterior scene. The control may apply an edge detection algorithm to a reduced image data set of the image data set. The reduced image data set is representative of a target zone of the captured image. The control may be operable to process the reduced image data set more than other image data, which are representative of areas of the captured image outside of the target zone, to detect objects present within the target zone. The imaging system may be associated with a side object detection system, a lane change assist system, a lane departure warning system and/or the like.

Claims

1. An imaging system for a vehicle comprising:

an imaging array sensor comprising a plurality of photo-sensing pixels, said imaging array sensor being positioned at the vehicle and having a field of view exteriorly of the vehicle, said imaging array sensor being operable to capture an image of a scene occurring exteriorly of the vehicle; and

a control for processing said captured image, said captured image comprising an image data set representative of the exterior scene, said control including a microprocessor comprising an edge detection algorithm, said control applying said edge detection algorithm to a reduced image data set of said image data set, said reduced image data set being representative of a target zone of said captured image, said control being operable to process said reduced image data set more than other image data, which are representative of areas of said captured image outside of said target zone, to detect edges of objects present exteriorly of the vehicle and within said target zone.

2. The imaging system of claim 1, wherein said control is operable to substantially discount said other image data.

3. The imaging system of claim 1, wherein said imaging system comprises a lane change assist system with said imaging array being directed sidewardly and rearwardly relative to the subject vehicle.

4. The imaging system of claim 3, wherein said control is in communication with a forward facing imaging system, said forward facing imaging system communicating at least one of oncoming traffic information, leading traffic information and lane marking information to said control.

5. The imaging system of claim 4, wherein said control is operable to adjust said reduced image data set and said target zone in response to said forward facing imaging system.

6. The imaging system of claim 1, wherein said imaging system comprises a lane departure warning system with said imaging array being directed forwardly relative to the subject vehicle.

7. The imaging system of claim 6, wherein said lane departure warning system is operable to provide a warning indication to a driver of the subject vehicle in response to at least two conditions detected by said imaging system.

8. The imaging system of claim 7, wherein said lane departure warning system provides a first warning in response to a detection that the subject vehicle is on course to leave the road surface and travel onto the shoulder of the road and a second warning in response to a detection that the subject vehicle is on course to leave the shoulder and road surface.

9. The imaging system of claim 6, wherein said imaging system is in communication with a passive steering system operable to adjust a steering direction of the vehicle in response to a detection by said imaging system of the vehicle drifting or leaving its occupied lane.

10. The imaging system of claim 6, wherein said imaging system is in communication with an adaptive speed control operable to adjust a cruise control setting of the vehicle in response to road or traffic conditions detected by said imaging system.

11. The imaging system of claim 1, wherein said imaging system is selectably operable to provide image data to a remote device which is remote from the vehicle.

12. The imaging system of claim 11, wherein said imaging system provides said image data to indicate a location and/or condition of the vehicle.

13. The imaging system of claim 11, wherein said remote device comprises at least one of a personal display system, a security display system and an emergency display system.

14. The imaging system of claim 1, wherein said control is operable to process horizontal detected edges and substantially limit processing of vertical detected edges.

15. The imaging system of claim 1, wherein said control is operable to process horizontal detected edges and substantially limit processing of vertical detected edges and reduce the amount of data to be processed downstream.

16. The imaging system of claim 1, wherein said control is operable to process detected edges which have a concentration of edges in a second reduced image data set which is representative of a particular reduced area or zone of said target zone of said captured image.

17. The imaging system of claim 1, wherein said control is operable to adjust said reduced image data set and said target zone in response to various threshold criterion.

18. The imaging system of claim 1, wherein said control is operable to adjust said reduced image data set and said target zone between a daytime zone and a nighttime zone in response to a daytime or nighttime driving condition for the vehicle.

19. The imaging system of claim 18, wherein said control is operable to adjust a height input of said imaging array sensor such that said daytime zone is representative of an area generally along the road surface and said nighttime zone is representative of an area generally at a height of headlamps of vehicles.

20. The imaging system of claim 1, wherein said control is operable to adjust said reduced image data set and said target zone in response to a distance to a detected edge.

21. The imaging system of claim 1, wherein said control is operable to approximate a distance to a portion of some of said detected edges in response to a pixel location of the portion in said captured image.

22. The imaging system of claim 21, wherein said pixel location is determined relative to a target pixel which is directed generally at the horizon and along the direction of travel of the vehicle.

23. The imaging system of claim 21, wherein said control is operable to approximate said distance in response to a pixel size, a pixel resolution and a field of view of said imaging array sensor.

24. The imaging system of claim 21, wherein said control accesses an information array which provides a calculated distance representative of a distance to an object or portion of an object detected by each pixel associated with said reduced image data set.

25. The imaging system of claim 1, wherein said imaging array sensor is directed partially downwardly such that an upper portion of the captured image is directed generally at or along the horizon.

26. The imaging system of claim 1, wherein said control is operable to substantially limit processing of detected edges having an intensity which is less than a threshold intensity.

27. The imaging system of claim 1, wherein said control is operable to determine whether said detected edges constitute a portion of a vehicle in an adjacent lane.

28. The imaging system of claim 27, wherein said control is operable to determine that said detected edges may constitute a portion of a vehicle in response to a determination that said detected edges are associated with an ellipse or partial ellipse, said ellipse or partial ellipse being indicative of a vehicle tire.

29. The imaging system of claim 1, wherein said control is operable to track at least one of said detected edges between subsequent frames captured by said imaging array sensor to identify some of said detected edges.

30. The imaging system of claim 1, wherein said control is operable to identify lane or road markings in said captured images to determine road curvature of the road section that the vehicle is traveling along.

31. The imaging system of claim 30, wherein said control is operable to adjust said reduced image data set and said target zone in response to an identification of a curve in the road section.

32. The imaging system of claim 1, wherein said control is operable to primarily process detected edges or substantially limit processing of detected edges in response to at least one of a size, location, intensity and distance of said detected edges relative to the vehicle.

33. The imaging system of claim 1, wherein said control is operable to substantially limit processing of edges with a measured change in brightness or color across the edge which is less than a threshold change amount.

34. A lane change assist system for a vehicle comprising:

an imaging sensor positioned at the vehicle and having a field of view directed generally sidewardly from the vehicle, said imaging sensor being operable to capture an image of a scene occurring toward the side of the vehicle; and

a control operable to process said image captured by said imaging array sensor to detect objects present sidewardly of the vehicle, said captured image comprising an image data set representative of the exterior scene, said control being operable to process a reduced image data set of said image data set more than other image data of said image data set, said reduced image data set being representative of a target zone of said captured image.

35. The lane change assist system of claim 34, wherein said control is operable to adjust said reduced image data set and said target zone in response to an adjustment input.

36. The lane change assist system of claim 35, wherein said adjustment input comprises an output from one of an ambient light sensor, a headlamp control and a manual control or input.

37. The lane change assist system of claim 36, wherein said control is operable to adjust said reduced image data set and said target zone between a daytime zone and a nighttime zone in response to said output.

38. The lane change assist system of claim 37, wherein said control is operable to adjust a height input of said imaging array sensor such that said daytime zone is generally along the road surface and said nighttime zone is generally at a height of headlamps of vehicles.

39. The lane change assist system of claim 35, wherein said control is operable to adjust said reduced image data set and said target zone in response to detection of the subject vehicle traveling through a curved section of road.

40. The lane change assist system of claim 35, wherein said adjustment input comprises an output from a forward facing imaging system.

41. The lane change assist system of claim 34, wherein said control is operable to adjust said reduced image data set and said target zone in response to a distance to a detected object.

42. The lane change assist system of claim 34, wherein said control comprises an edge detection algorithm and is operable to detect edges of objects sidewardly of the vehicle, said control being operable to determine whether said detected edges are indicative of a significant object or other vehicle.

43. The lane change assist system of claim 34, wherein said imaging sensor is directed partially downwardly such that an upper portion of the captured image is generally at or along the horizon.

44. The lane change assist system of claim 34, wherein said control is operable to determine that said detected objects constitute a portion of a vehicle via a determination that said detected objects are associated with an ellipse or partial ellipse.

45. The lane change assist system of claim 34, wherein said control is operable to primarily process image data or substantially limit processing of image data in response to at least one of a size, location, intensity and distance of detected objects of said image data relative to the vehicle.

46. The lane change assist system of claim 34, wherein said control is operable to approximate a distance to a detected object in response to a location of a pixel capturing a portion of the detected object in said captured image.

47. The lane change assist system of claim 46, wherein said pixel location is determined relative to a target pixel which is directed generally at the horizon and along the direction of travel of the vehicle.

48. The lane change assist system of claim 46, wherein said control is operable to approximate said distance using spherical trigonometry in response to said pixel location, a pixel size, a pixel resolution and a field of view of said imaging sensor.

49. The lane change assist system of claim 46, wherein said control accesses an information array which provides a calculated distance for substantially each pixel associated with said reduced image data set and said target zone to approximate said distance to the portion of the detected object.

50. A passive steering system of a vehicle comprising:

a steering system operable to adjust the steering direction of the vehicle; and

a control responsive to an image capture device capable of detecting lane markers along the road surface, said control being operable to control said steering system to adjust the steering direction of the vehicle in response to an output of said image capture device indicative of the vehicle drifting out of its occupied lane.

51. The passive steering system of claim 50, wherein said steering system is manually controllable irrespective of said control.

52. The passive steering system of claim 50, wherein said control is operable to not control said steering system in response to activation of a turn signal of the vehicle.

53. The passive steering system of claim 50, wherein said image capture device is part of or in communication with a lane departure warning system of the vehicle.

54. An adaptive speed control system of a vehicle comprising:

a cruise control system operable to generally maintain a selected speed of the vehicle; and

a control responsive to an image capture device capable of detecting lane markers or objects or other vehicles along the road surface, said control being operable to adjust said cruise control system to a lower speed than said selected speed in response to an output of said image capture device indicative of at least one of an approaching object or vehicle and a change in road conditions.

55. The adaptive speed control system of claim 54, wherein said control is operable to adjust said cruise control system in response to said output being indicative of a curved section of road in front of the vehicle.

56. The adaptive speed control system of claim 55, wherein said control is operable to adjust said cruise control system to resume the selected speed in response to the vehicle being past the curved section of road.

57. The adaptive speed control system of claim 54, wherein said image capture device is part of or in communication with a lane departure warning system of the vehicle.

58. A monitoring system for monitoring an area surrounding a vehicle comprising:

an imaging system of the vehicle, said imaging system comprising at least one of an object detection system, a lane departure warning system, a lane change assist system, a side object detection system, a forward facing imaging system, a rearward facing imaging system, a back up aid and a panoramic view imaging system, said imaging system comprising an imaging sensor operable to capture an image of a scene occurring exteriorly of the vehicle; and

a control responsive to said imaging sensor and selectably operable to communicate image data of said captured image to a remote receiving device remote from the vehicle.

59. The monitoring system of claim 58, wherein said remote receiving device comprises at least one of a personal display system, a security display system and an emergency display system.

60. The monitoring system of claim 58, wherein said control is selectably operable to communicate said image data to said remote receiving device in response to an input from a transmitter associated with said remote receiving device.

61. The monitoring system of claim 58, wherein said control provides said image data to indicate a location and/or condition of the vehicle.