Driver assistance system for vehicle

• US 9,193,303 B2
• Filed: 04/20/2015
• Issued: 11/24/2015
• Est. Priority Date: 12/23/2004
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Higgins-Luthman, Michael J.
• Primary Examiner(s): Tran, Khoi
• Assistant Examiner(s): Figueroa, Jaime
• Application Number:
  US14/690,745
• Publication Number:
  US 20150224933A1
• Time in Patent Office: 218 Days
• Field of Search: 701/28, 701/36, 701/301, 250/208.1, 340/435, 340/436, 382/104, 382/199
• US Class Current: 1/1
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

A driver assistance system for a vehicle includes an imaging device disposed at least partially within an exterior rearview mirror assembly of the vehicle. With the exterior rearview mirror assembly attached at a side of the vehicle, the imaging device has a field of view forward of the mirror assembly, rearward of the mirror assembly and sideward of the mirror assembly. A control processes image data captured by the imaging device and is operable to determine that the imaging device is misaligned when the imaging device is disposed at least partially within the exterior rearview mirror assembly at the side of the equipped vehicle. The control, responsive to a determination of misalignment of the imaging device, is operable to at least partially compensate for misalignment of the imaging device by at least one of (i) adjusting processing of captured image data and (ii) adjusting the captured image data.

Claims

1. A driver assistance system for a vehicle, said driver assistance system comprising:

an imaging device comprising an array of photosensing pixels, wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said driver assistance system;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle and has a field of view rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and has a field of view sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein said control is operable to determine that said imaging device is misaligned when said imaging device is disposed at least partially within the exterior rearview mirror assembly at the side of the equipped vehicle; and

wherein said control, responsive to a determination of misalignment of said imaging device, is operable to at least partially compensate for misalignment of said imaging device by at least one of (i) adjusting processing of captured image data and (ii) adjusting said captured image data.

2. The driver assistance system of claim 1, wherein said control utilizes edge detection in processing captured image data and wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device.

3. The driver assistance system of claim 2, wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition, and wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) a bicycle and (ii) a bicycle rider.

4. The driver assistance system of claim 2, wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored.

5. The driver assistance system of claim 2, wherein said control determines that an object of interest determined to be present in the field of view of said imaging device is a bicycle.

6. The driver assistance system of claim 2, wherein said utilization of edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device, and wherein said control is operable with a daytime sensitivity and a nighttime sensitivity, and wherein said control operates with said daytime sensitivity or said nighttime sensitivity in response to an ambient light level at the equipped vehicle.

7. The driver assistance system of claim 2, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present in the field of view of said imaging device.

8. The driver assistance system of claim 7, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present in response to at least one of (a) the driver of the equipped vehicle actuating a turn signal toward the side of the equipped vehicle at which the object of interest is determined to be present, and (b) the driver of the equipped vehicle steering the equipped vehicle toward the side of the equipped vehicle at which the object of interest is determined to be present.

9. The driver assistance system of claim 1, wherein said utilization of said edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device.

10. The driver assistance system of claim 1, wherein processing of captured image data by said control is responsive, at least in part, to an ambient light level at the equipped vehicle.

11. The driver assistance system of claim 1, wherein said control processes captured image data in accordance with a daytime algorithm or a nighttime algorithm.

12. The driver assistance system of claim 11, wherein said control chooses said daytime algorithm or said nighttime algorithm in response to an ambient light level at the equipped vehicle.

13. The driver assistance system of claim 1, wherein processing of captured image data by said control is responsive, at least in part, to an ambient light level at the equipped vehicle, and wherein, in accordance with the ambient light level at the equipped vehicle, said control operates with a daytime sensitivity or a nighttime sensitivity.

14. The driver assistance system of claim 1, wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device and wherein said portion of the equipped vehicle comprises a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle.

15. The driver assistance system of claim 1, wherein processing by said control of image data captured by said imaging device comprises use of an algorithmically executed filter.

16. The driver assistance system of claim 1, wherein a video display screen is located in the interior cabin of the equipped vehicle and is viewable by the driver of the equipped vehicle, and wherein images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device.

17. The driver assistance system of claim 1, wherein said imaging device comprises a CMOS array of photosensing pixels.

18. The driver assistance system of claim 1, wherein said control is operable to determine that said imaging device is aligned within a desired tolerance when said imaging device is disposed at least partially within the exterior rearview mirror assembly at the side of the equipped vehicle and wherein said control, responsive to a determination that said imaging device is not aligned within a desired tolerance, is operable to (i) adjust processing of captured image data and (ii) adjust said captured image data.

19. The driver assistance system of claim 1, wherein said control, responsive to a determination of misalignment of said imaging device, is operable to determine a degree of misalignment of said imaging device and wherein said control, responsive to determination of the degree of misalignment of said imaging device, is operable to at least partially compensate for the determined degree of misalignment of said imaging device.

20. The driver assistance system of claim 1, wherein said control, responsive to a determination of a degree of misalignment of said imaging device, is operable to at least partially compensate for the determined degree of misalignment of said imaging device by at least one of (i) adjusting said processing of captured image data and (ii) adjusting said captured image data set.

21. The driver assistance system of claim 1, wherein the equipped vehicle comprises a left side and a right side, and wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at the left side of the equipped vehicle, and wherein when said exterior rearview mirror assembly is attached at the left side of the equipped vehicle, another exterior rearview mirror assembly is attached at the right side of the equipped vehicle and wherein another imaging device is disposed at least partially within the another exterior rearview mirror assembly that is attached at the right side of the equipped vehicle, and wherein a rear backup camera is located at a rear portion of the equipped generally centrally between where said left side-attached imaging device and the right side-attached another imaging device are disposed at the vehicle, and wherein the field of view of said left side-attached imaging device overlaps with the field of view of the rear backup camera and wherein the field of view of the right side-attached another imaging device overlaps with the field of view of the rear backup camera, and wherein image data captured by said left side-attached imaging device, by the right side-attached another imaging device and by the rear backup camera is processed by an image processor and, at least in part responsive to said processing by said image processor, a composite image is displayed by a display disposed in the interior cabin of the equipped vehicle positioned within the driver'"'"'s physiological field of view without obstructing the driver'"'"'s view through the windshield of the equipped vehicle, said composite image approximating a view from a single virtual image capture device.

22. The driver assistance system of claim 21, wherein said display is disposed in the interior cabin of the equipped vehicle within a lower portion of the driver'"'"'s field of view and wherein said display comprises a flat panel display.

23. The driver assistance system of claim 22, wherein said flat panel display comprises a back-lit liquid crystal display.

24. The driver assistance system of claim 23, wherein said composite image provides a panoramic view that approximates a view from a single virtual image capture device.

25. The driver assistance system of claim 24, wherein said composite image provides a substantially seamless panoramic view without redundant images of objects.

26. A driver assistance system for a vehicle, said driver assistance system comprising:

an imaging device comprising an array of photosensing pixels, wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said driver assistance system;

wherein said imaging device comprises a CMOS array of photosensing pixels;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle and has a field of view rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and has a field of view sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein said control is operable to determine that said imaging device is misaligned when said imaging device is disposed at least partially within the exterior rearview mirror assembly at the side of the equipped vehicle;

wherein said control, responsive to a determination of misalignment of said imaging device, is operable to at least partially compensate for misalignment of said imaging device by at least one of (i) adjusting processing of captured image data and (ii) adjusting said captured image data;

wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device and wherein said portion of the equipped vehicle comprises a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle;

wherein the equipped vehicle comprises a left side and a right side, and wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at the left side of the equipped vehicle, and wherein when said exterior rearview mirror assembly is attached at the left side of the equipped vehicle, another exterior rearview mirror assembly is attached at the right side of the equipped vehicle and wherein another imaging device is disposed at least partially within the another exterior rearview mirror assembly that is attached at the right side of the equipped vehicle, and wherein a rear backup camera is located at a rear portion of the equipped generally centrally between where said left side-attached imaging device and the right side-attached another imaging device are disposed at the vehicle, and wherein the field of view of said left side-attached imaging device overlaps with the field of view of the rear backup camera and wherein the field of view of the right side-attached another imaging device overlaps with the field of view of the rear backup camera;

wherein image data captured by said left side-attached imaging device, by the right side-attached another imaging device and by the rear backup camera is processed by an image processor and, at least in part responsive to said processing by said image processor, a composite image is displayed by a display disposed in the interior cabin of the equipped vehicle positioned within the driver'"'"'s physiological field of view without obstructing the driver'"'"'s view through the windshield of the equipped vehicle;

wherein said display comprises a flat panel display;

wherein said flat panel display comprises a back-lit liquid crystal display; and

wherein said composite image provides a panoramic view that approximates a view from a single virtual image capture device.

27. The driver assistance system of claim 26, wherein said composite image provides a substantially seamless panoramic view without redundant images of objects.

28. The driver assistance system of claim 27, wherein said control, responsive to a determination of misalignment of said imaging device, is operable to determine a degree of misalignment of said imaging device and wherein said control, responsive to determination of the degree of misalignment of said imaging device, is operable to at least partially compensate for the determined degree of misalignment of said imaging device.

29. The driver assistance system of claim 26, wherein processing of captured image data by said control is responsive, at least in part, to an ambient light level at the equipped vehicle.

30. The driver assistance system of claim 29, wherein, in accordance with the ambient light level at the equipped vehicle, said control operates with a daytime sensitivity or a nighttime sensitivity.

31. The driver assistance system of claim 26, wherein said control utilizes edge detection in processing captured image data and wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device.

32. The driver assistance system of claim 31, wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored.

33. The driver assistance system of claim 32, wherein, responsive at least in part to processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition.

34. The driver assistance system of claim 33, wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) a bicycle and (ii) a bicycle rider.

35. A driver assistance system for a vehicle, said driver assistance system comprising:

an imaging device comprising an array of photosensing pixels, wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said driver assistance system;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle and has a field of view rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and has a field of view sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein said control is operable to determine that said imaging device is misaligned when said imaging device is disposed at least partially within the exterior rearview mirror assembly at the side of the equipped vehicle;

wherein said control, responsive to a determination of misalignment of said imaging device, is operable to at least partially compensate for misalignment of said imaging device by at least one of (i) adjusting processing of captured image data and (ii) adjusting said captured image data;

wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device and wherein said portion of the equipped vehicle comprises a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle;

wherein the equipped vehicle comprises a left side and a right side, and wherein said imaging device is disposed at least partially within an exterior rearview mirror assembly that attaches at the left side of the equipped vehicle, and wherein when said exterior rearview mirror assembly is attached at the left side of the equipped vehicle, another exterior rearview mirror assembly is attached at the right side of the equipped vehicle and wherein another imaging device is disposed at least partially within the another exterior rearview mirror assembly that is attached at the right side of the equipped vehicle, and wherein a rear backup camera is located at a rear portion of the equipped generally centrally between where said left side-attached imaging device and the right side-attached another imaging device are disposed at the vehicle, and wherein the field of view of said left side-attached imaging device overlaps with the field of view of the rear backup camera and wherein the field of view of the right side-attached another imaging device overlaps with the field of view of the rear backup camera;

wherein image data captured by said left side-attached imaging device, by the right side-attached another imaging device and by the rear backup camera is processed by an image processor and, at least in part responsive to said processing by said image processor, a composite image is displayed by a display disposed in the interior cabin of the equipped vehicle positioned within the driver'"'"'s physiological field of view without obstructing the driver'"'"'s view through the windshield of the equipped vehicle; and

wherein said composite image provides a substantially seamless panoramic view that approximates a view from a single virtual image capture device.

36. The driver assistance system of claim 35, wherein processing of captured image data by said control is responsive, at least in part, to an ambient light level at the equipped vehicle.

37. The driver assistance system of claim 36, wherein, in accordance with the ambient light level at the equipped vehicle, said control operates with a daytime sensitivity or a nighttime sensitivity.

38. The driver assistance system of claim 36, wherein said composite image provides a substantially seamless panoramic view without redundant images of objects.

39. The driver assistance system of claim 38, wherein said display comprises a flat panel display and wherein said flat panel display comprises a back-lit liquid crystal display.

40. The driver assistance system of claim 35, wherein said control utilizes edge detection in processing captured image data and wherein, responsive at least in part to processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device.