IMAGING SYSTEM FOR VEHICLE

• US 20120293658A1
• Filed: 08/03/2012
• Published: 11/22/2012
• Est. Priority Date: 12/23/2004
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Higgins-Luthman, Michael J.
• Granted Patent: US 8,386,114 B2
• Time in Patent Office: Days
• US Class Current: 348/148
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/12 Mirror assemblies combined with other articles, e.g. clocks
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2001/1253 with cameras, video cameras or video screens
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/402 Image calibration
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/806 for aiding parking
• CPC Class Codes: B60R 2300/8066 for monitoring rearward traffic
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/80 Camera processing pipelines; Components thereof
• CPC Class Codes: H04N 25/76 Addressed sensors, e.g. MOS or CMOS sensors
• CPC Class Codes: H04N 7/18 Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources

Abstract

A vehicular imaging system includes an imaging device that is part of an exterior rearview mirror assembly and having a field of view rearward and/or sideward of the vehicle. A control processes captured image data utilizing edge detection. Responsive at least in part to the processing, the control determines objects of interest present in the field of view of the imaging device. Objects of interest determined to be present in the field of view include (i) a vehicle that is rearward and/or sideward of the vehicle, (ii) a headlight of a vehicle that is rearward and/or sideward of the vehicle, (iii) a bicycle that is rearward and/or sideward of the vehicle, and/or (iv) a bicycle rider that is rearward and/or sideward of the vehicle. Responsive at least in part to the processing of captured image data, a driver of the vehicle is alerted to a hazardous condition.

Claims

1. An imaging system for detecting objects exterior of a vehicle, said imaging system comprising:

an imaging device comprising a CMOS array of photo-sensing pixels, wherein said imaging device is part of an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said imaging system, and wherein said imaging device captures image data;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view at least one of rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein said control utilizes edge detection in processing captured image data;

wherein, responsive at least in part to said processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device;

wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) a vehicle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, (ii) a headlight of a vehicle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, (iii) a bicycle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, and (iv) a bicycle rider that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle;

wherein, responsive at least in part to said processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and

wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored in order to at least one of (a) reduce processing requirements and (b) reduce false signals.

2. The imaging system of claim 1, wherein said control adjusts said processing of captured image data to accommodate misalignment of said imaging device.

3. The imaging system of claim 1, wherein said control determines shadows present in the field of view of said imaging device and discerns shadows from objects present in the field of view of said imaging device.

4. The imaging system of claim 1, wherein said control at least one of (a) determines that an object present in the field of view is a headlight of a vehicle rearward of the equipped vehicle and (b) determines that an object present in the field of view is a headlight of a vehicle that is sideward of the equipped vehicle.

5. The imaging system of claim 4, wherein said control distinguishes a headlight from other light sources present in the field of view of said imaging device.

6. The imaging system of claim 1, wherein said control determines that an object of interest determined to be present in the field of view of said imaging device is a bicycle.

7. The imaging system of claim 1, wherein said control determines that an object of interest determined to be present in the field of view of said imaging device is a bicycle at least in part responsive to a determined speed of the bicycle.

8. The imaging system of claim 1, wherein said use of said edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device.

9. The imaging system of claim 8, wherein said determined generally vertical edges are used at least in part in identification of objects of interest by said control.

10. The imaging system of claim 1, wherein said control determines that an object of interest determined to be present in the field of view of said imaging device is a wheel of a vehicle.

11. The imaging system of claim 1, wherein said control determines the presence of a wheel of a vehicle by processing image data and determining generally vertical edges of objects present in the field of view of said imaging device.

12. The imaging system of claim 1, wherein said control is operable in a daytime mode and a nighttime mode.

13. The imaging system of claim 12, wherein said control switches between said daytime mode and said nighttime mode in response to an ambient light level at the equipped vehicle.

14. The imaging system of claim 1, wherein said control reduces captured image data to a reduced data set of said image data, said control processing said reduced data set to extract information from said reduced data set.

15. The imaging system of claim 14, wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device and that is not inclusive of a portion of the equipped vehicle.

16. The imaging system of claim 15, wherein said control adjusts said target zone responsive to a steering angle of the wheels of the equipped vehicle.

17. The imaging system of claim 1, wherein said control algorithmically processes captured image data to a reduced data set, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is not inclusive of a portion of the equipped vehicle.

18. The imaging system of claim 1, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present in the field of view of said imaging device.

19. The imaging system of claim 1, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present sideward of the equipped vehicle in response to at least one of (a) the driver of the equipped vehicle actuating a turn signal toward the side of the equipped vehicle at which the object of interest is determined to be present, and (b) the driver of the equipped vehicle steering the equipped vehicle toward the side of the equipped vehicle at which the object of interest is determined to be present.

20. The imaging system of claim 1, wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device.

21. The imaging system of claim 20, wherein said control determines that said portion of the equipped vehicle is not an object of interest.

22. The imaging system of claim 20, wherein said portion of the equipped vehicle comprises at least a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle.

23. The imaging system of claim 1, wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle.

24. The imaging system of claim 1, wherein, in determining objects of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that at least substantially ignores detected edges that are not indicative of objects of interest.

25. The imaging system of claim 1, wherein a video display screen is located in the interior cabin of the equipped vehicle and is viewable by the driver of the equipped vehicle, and wherein images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device.

26. The imaging system of claim 25, wherein said video display screen is disposed at an interior rearview mirror assembly of the equipped vehicle.

27. An imaging system for detecting objects exterior of a vehicle, said imaging system comprising:

an imaging device comprising a CMOS array of photo-sensing pixels, wherein said imaging device is part of an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said imaging system, and wherein said imaging device captures image data;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view at least one of rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein said control utilizes edge detection in processing captured image data;

wherein, responsive at least in part to said processing of captured image data by said control, said control determines an object of interest present in the field of view of said imaging device;

wherein an object of interest determined to be present in the field of view of said imaging device comprises at least one of (i) a vehicle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, (ii) a headlight of a vehicle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle;

wherein, responsive at least in part to said processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and

wherein a video display screen is located in the interior cabin of the equipped vehicle and is viewable by the driver of the equipped vehicle, and wherein images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device.

28. The imaging system of claim 27, wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored in order to at least one of (a) reduce processing requirements and (b) reduce false signals.

29. The imaging system of claim 27, wherein at least one of (a) said control adjusts said processing of captured image data to accommodate misalignment of said imaging device, and (b) said control determines shadows present in the field of view of said imaging device and discerns shadows from objects present in the field of view of said imaging device.

30. The imaging system of claim 27, wherein said control distinguishes a headlight from other light sources present in the field of view of said imaging device.

31. The imaging system of claim 27, wherein at least one of (a) said control determines that an object of interest determined to be present in the field of view of said imaging device is a bicycle, and (b) said control determines that an object of interest determined to be present in the field of view of said imaging device is a bicycle at least in part responsive to a determined speed of the bicycle.

32. The imaging system of claim 27, wherein at least one of (a) said use of said edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device and wherein said determined generally vertical edges are used at least in part in identification of objects of interest by said control, (b) said control determines that an object of interest determined to be present in the field of view of said imaging device is a wheel of a vehicle, and (c) said control determines the presence of a wheel of a vehicle by processing image data and determining generally vertical edges of objects present in the field of view of said imaging device.

33. The imaging system of claim 27, wherein said control is operable in a daytime mode and a nighttime mode, and wherein said control switches between said daytime mode and said nighttime mode in response to an ambient light level at the equipped vehicle.

34. The imaging system of claim 27, wherein said control reduces captured image data to a reduced data set of said image data, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device and that is not inclusive of a portion of the equipped vehicle, and wherein said control adjusts said target zone responsive to a steering angle of the wheels of the equipped vehicle.

35. The imaging system of claim 27, wherein at least one of (a) said control algorithmically processes captured image data to a reduced data set, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is not inclusive of a portion of the equipped vehicle, (b) in determining objects of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that at least substantially ignores detected edges that are not indicative of objects of interest, and (c) the driver of the equipped vehicle is alerted that an object of interest is determined to be present in the field of view of said imaging device.

36. The imaging system of claim 27, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present sideward of the equipped vehicle in response to at least one of (a) the driver of the equipped vehicle actuating a turn signal toward the side of the equipped vehicle at which the object of interest is determined to be present, and (b) the driver of the equipped vehicle steering the equipped vehicle toward the side of the equipped vehicle at which the object of interest is determined to be present.

37. The imaging system of claim 27, wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device, and wherein at least one of (a) said control determines that said portion of the equipped vehicle is not an object of interest, and (b) said portion of the equipped vehicle comprises at least a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle.

38. The imaging system of claim 27, wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle.

39. An imaging system for detecting objects exterior of a vehicle, said imaging system comprising:

an imaging device comprising a CMOS array of photo-sensing pixels, wherein said imaging device is part of an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said imaging system, and wherein said imaging device captures image data;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view at least one of rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle and sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein said control utilizes edge detection in processing captured image data;

wherein, responsive at least in part to said processing of captured image data by said control, said control determines an object of interest present in the field of view of said imaging device;

wherein an object of interest determined to be present in the field of view of said imaging device comprises at least one of (i) a bicycle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, and (ii) a rider of a bicycle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle;

wherein, responsive at least in part to said processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and

wherein a video display screen is located in the interior cabin of the equipped vehicle and is viewable by the driver of the equipped vehicle, and wherein images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device.

40. The imaging system of claim 39, wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored in order to at least one of (a) reduce processing requirements and (b) reduce false signals.

41. The imaging system of claim 39, wherein at least one of (a) said control adjusts said processing of captured image data to accommodate misalignment of said imaging device, and (b) said control determines shadows present in the field of view of said imaging device and discerns shadows from objects present in the field of view of said imaging device.

42. The imaging system of claim 39, wherein said control distinguishes a headlight from other light sources present in the field of view of said imaging device.

43. The imaging system of claim 39, wherein said control determines that the object of interest determined to be present in the field of view of said imaging device is a bicycle or rider of a bicycle at least in part responsive to a determined speed of the bicycle.

44. The imaging system of claim 39, wherein at least one of (a) said use of said edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device and wherein said determined generally vertical edges are used at least in part in identification of objects of interest by said control, (b) said control determines that an object of interest determined to be present in the field of view of said imaging device is a wheel of a vehicle, and (c) said control determines the presence of a wheel of a vehicle by processing image data and determining generally vertical edges of objects present in the field of view of said imaging device.

45. The imaging system of claim 39, wherein said control is operable in a daytime mode and a nighttime mode, and wherein said control switches between said daytime mode and said nighttime mode in response to an ambient light level at the equipped vehicle.

46. The imaging system of claim 39, wherein said control reduces captured image data to a reduced data set of said image data, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device and that is not inclusive of a portion of the equipped vehicle, and wherein said control adjusts said target zone responsive to a steering angle of the wheels of the equipped vehicle.

47. The imaging system of claim 39, wherein at least one of (a) said control algorithmically processes captured image data to a reduced data set, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is not inclusive of a portion of the equipped vehicle, (b) in determining objects of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that at least substantially ignores detected edges that are not indicative of objects of interest, and (c) the driver of the equipped vehicle is alerted that an object of interest is determined to be present in the field of view of said imaging device.

48. The imaging system of claim 39, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present sideward of the equipped vehicle in response to at least one of (a) the driver of the equipped vehicle actuating a turn signal toward the side of the equipped vehicle at which the object of interest is determined to be present, and (b) the driver of the equipped vehicle steering the equipped vehicle toward the side of the equipped vehicle at which the object of interest is determined to be present.

49. The imaging system of claim 39, wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device, and wherein at least one of (a) said control determines that said portion of the equipped vehicle is not an object of interest, and (b) said portion of the equipped vehicle comprises at least a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle.

50. The imaging system of claim 39, wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle.

51. An imaging system for detecting objects exterior of a vehicle, said imaging system comprising:

an imaging device comprising a CMOS array of photo-sensing pixels, wherein said imaging device is part of an exterior rearview mirror assembly that attaches at a side of a vehicle equipped with said imaging system, and wherein said imaging device captures image data;

wherein, with said exterior rearview mirror assembly attached at the side of the equipped vehicle, said imaging device has a field of view rearward of said exterior rearview mirror assembly in a direction towards the rear of the equipped vehicle, forward of said exterior rearview mirror assembly in a direction towards the front of the equipped vehicle and sideward of said exterior rearview mirror assembly in a direction away from the side of the equipped vehicle at which said exterior rearview mirror assembly is attached;

a control for processing image data captured by said imaging device;

wherein, responsive at least in part to said processing of captured image data by said control, said control determines objects of interest present in the field of view of said imaging device;

wherein objects of interest determined to be present in the field of view of said imaging device comprise at least one of (i) a vehicle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, (ii) a headlight of a vehicle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, (iii) a bicycle that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle, and (iv) a bicycle rider that is at least one of rearward of the equipped vehicle and sideward of the equipped vehicle;

wherein, responsive at least in part to said processing of captured image data by said control, a driver of the equipped vehicle is alerted to a hazardous condition; and

wherein a video display screen is located in the interior cabin of the equipped vehicle and is viewable by the driver of the equipped vehicle, and wherein images are displayed by said video display screen that are derived, at least in part, from image data captured by said imaging device.

52. The imaging system of claim 51, wherein said control utilizes edge detection in processing captured image data.

53. The imaging system of claim 52, wherein, in determining objects of interest present in the field of view of said imaging device, edges that are not indicative of objects of interest are substantially ignored in order to at least one of (a) reduce processing requirements and (b) reduce false signals.

54. The imaging system of claim 52, wherein at least one of (a) said use of said edge detection by said control comprises determination of generally vertical edges of objects present in the field of view of said imaging device and wherein said determined generally vertical edges are used at least in part in identification of objects of interest by said control, (b) said control determines that an object of interest determined to be present in the field of view of said imaging device is a wheel of a vehicle, and (c) said control determines the presence of a wheel of a vehicle by processing image data and determining generally vertical edges of objects present in the field of view of said imaging device.

55. The imaging system of claim 51, wherein at least one of (a) said control adjusts said processing of captured image data to accommodate misalignment of said imaging device, and (b) said control determines shadows present in the field of view of said imaging device and discerns shadows from objects present in the field of view of said imaging device.

56. The imaging system of claim 51, wherein said control distinguishes a headlight from other light sources present in the field of view of said imaging device.

57. The imaging system of claim 51, wherein said control is operable in a daytime mode and a nighttime mode, and wherein said control switches between said daytime mode and said nighttime mode in response to an ambient light level at the equipped vehicle.

58. The imaging system of claim 51, wherein said control reduces captured image data to a reduced data set of said image data, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is encompassed by the field of view of said imaging device and that is not inclusive of a portion of the equipped vehicle, and wherein said control adjusts said target zone responsive to a steering angle of the wheels of the equipped vehicle.

59. The imaging system of claim 51, wherein at least one of (a) said control algorithmically processes captured image data to a reduced data set, said control processing said reduced data set to extract information from said reduced data set, and wherein said reduced data set is representative of a target zone that is not inclusive of a portion of the equipped vehicle, (b) in determining objects of interest present in the field of view of said imaging device, an algorithmically executed filter is utilized that at least substantially ignores detected edges that are not indicative of objects of interest, and (c) the driver of the equipped vehicle is alerted that an object of interest is determined to be present in the field of view of said imaging device.

60. The imaging system of claim 51, wherein the driver of the equipped vehicle is alerted that an object of interest is determined to be present sideward of the equipped vehicle in response to at least one of (a) the driver of the equipped vehicle actuating a turn signal toward the side of the equipped vehicle at which the object of interest is determined to be present, and (b) the driver of the equipped vehicle steering the equipped vehicle toward the side of the equipped vehicle at which the object of interest is determined to be present.

61. The imaging system of claim 51, wherein a portion of the side of the equipped vehicle is present in the field of view of said imaging device, and wherein at least one of (a) said control determines that said portion of the equipped vehicle is not an object of interest, and (b) said portion of the equipped vehicle comprises at least a portion of the side of the equipped vehicle at which said exterior rearview mirror assembly is mounted when attached at the side of the vehicle.