Interior rearview mirror system including a forward facing video device

• US 20020056805A1
• Filed: 11/05/2001
• Published: 05/16/2002
• Est. Priority Date: 09/22/1997
• Status: Active Grant

• Current Assignee: Donnelly Corporation (Magna International Incorporated)
• Original Assignee: Donnelly Corporation (Magna International Incorporated)
• Inventors: Schofield, Kenneth, Lynam, Niall R., Larson, Mark L., Bos, Brent J.
• Granted Patent: US 6,806,452 B2
• Time in Patent Office: Days
• US Class Current: 250/208.1
• CPC Class Codes: B60N 2/002 Seats provided with an occupancy detection means mounted therein or thereon detection means responsive to presence or absence of children B60N2/266
• CPC Class Codes: B60Q 1/1423 Automatic dimming circuits, i.e. switching between high beam and low beam due to change of ambient light or light level in road traffic
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60R 2001/1223 with sensors or transducers
• CPC Class Codes: B60R 2300/8006 for monitoring and displaying scenes of vehicle interior, e.g. for monitoring passengers or cargo
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/8073 for vehicle security, e.g. parked vehicle surveillance, burglar detection
• CPC Class Codes: B60S 1/0822 characterized by the arrangement or type of detection means
• CPC Class Codes: B60S 1/0844 including a camera
• CPC Class Codes: B60S 1/0885 the sensor being integrated in a rear-view mirror module
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 20/56 exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: H04N 13/239 using two 2D image sensors having a relative position equal to or related to the interocular distance H04N13/243 takes precedence
• CPC Class Codes: H04N 2013/0081 Depth or disparity estimation from stereoscopic image signals

Abstract

A vehicular rain sensor system for detecting precipitation on an exterior surface of a window including a illumination sensor that is decoupled from the window. The illumination sensor is preferably an imaging array sensor which communicates a signal to a control which further determines whether rain is present on the window. The control preferably includes an edge detection function for detecting edges of precipitation droplets on the window and activating the windshield wipers of the vehicle when the number of edges detected exceeds a predetermined threshold value. A smoothing algorithm or filter is provided to account for surface irregularities on the window, thereby substantially precluding such irregularities from being erroneously detected as rain droplets by the edge detection function. The rain sensor system may further include a polarizing filter and an illumination source, such that the rain sensor system may not only prevent false signals of rain when only fog is present on an interior surface of the window, but also allows the rain sensor system to actually detect fog particles on an interior surface of the window, thereby allowing the control to further be connected to a ventilation blower within the vehicle for the purpose of activating the blower to eliminate the fog.

Claims

1. A vehicular rain sensor which senses precipitation at a vehicle window, said vehicular sensor comprising:

an imaging array sensor directed at the vehicle window from inside the vehicle, said imaging array sensor detecting precipitation at the window; and

a control which is responsive to said imaging array sensor detecting precipitation at the window, said control including a filtering process reducing effects of window irregularities.

2. The vehicular sensor in claim 1, wherein the precipitation indicated is water at an exterior surface of the window.

3. The vehicular sensor in claim 1, wherein the precipitation indicated is fogging at an interior cabin surface of the window.

4. The vehicular sensor in claim 1, wherein said imaging array sensor is a pixelated imaging array comprising a plurality of pixels, said filtering process being operable to adjust a value of at least some of said plurality of pixels in response to a calculated value, for an adjusted pixel, said calculated value being derived from a value of one or more neighboring pixels that are within a range of the adjusted pixel.

5. The vehicular sensor in claim 4, wherein the one or more neighboring pixels are a sub-array of pixels, the adjusted pixel being a center pixel of said sub-array.

6. The vehicular sensor in claim 5, wherein said sub-array is a three pixel by three pixel sub-array.

7. The vehicular sensor in claim 4, wherein the adjusted pixel is adjusted to an average of a sub-array of pixels surrounding the adjusted pixel.

8. The vehicular sensor in claim 4, wherein the adjusted pixel is adjusted to an average of pixels associated with a sub-array of low variance surrounding the adjusted pixel.

9. The vehicular sensor in claim 4, wherein said filtering process is one of a Median filter, a Sigma filter and a Nagao-Matsuyama filter.

10. The vehicular sensor in claim 1, wherein said filtering process reduces effects of window irregularities in response to a size of images detected by said imaging array sensor.

11. The vehicular sensor in claim 10, wherein said filtering process reduces effects of window irregularities which are of a size which is less than a size associated with water and fogging precipitation.

12. The vehicular sensor in claim 1, wherein said filtering process is a smoothing process.

13. The vehicular sensor in claim 1, wherein said control comprises a computer programmed with an edge detection algorithm.

14. The vehicular sensor in claim 13, wherein said edge detection algorithm detects edges of particles of precipitation at the window, said filtering process processing the edges detected by said edge detection algorithm, said control responding when a threshold amount of filtered edges is processed by said filtering process.

15. The vehicular sensor in claim 14, wherein said filtering process reduces effects of window irregularities in response to a size of said detected edges.

16. The vehicular sensor in claim 15, wherein said filtering process reduces effects of window irregularities which are of a size which is less than a size associated with edges of water and fogging precipitation.

17. The vehicular sensor in claim 14, wherein said filtering process is an edge preserving smoothing process.

18. The vehicular sensor in claim 14, wherein said edge detection algorithm analyzes a quantity of edges present at the window.

19. The vehicular sensor in claim 18, wherein said edge detection algorithm further analyzes a size of the edges present at the window.

20. The vehicular sensor in claim 14, wherein the vehicle has a windshield wiper, said control controlling the windshield wiper when said threshold amount is detected.

21. The vehicular sensor in claim 20, wherein said control varies a rate at which the windshield wiper is wiped according to a level of edges detected by said edge detection algorithm.

22. The vehicular sensor in claim 13, wherein said imaging array is a pixelated imaging array comprising a plurality of pixels, each of said plurality of pixels communicating a value associated with an amount of light being received by that pixel, said filtering process being operable to adjust the values of an adjusted pixel by a calculated value derived from light being received by at least one neighboring pixel within a range of the adjusted pixel.

23. The vehicular sensor in claim 22, wherein said filtering process is one of a Median filter, a Sigma filter and a Nagao-Matsuyama filter.

24. The vehicular sensor in claim 1 further including a polarizing filter between the window and said imaging array sensor.

25. The vehicular sensor in claim 1 including an optic between said imaging array sensor and the window, said optic establishing a small depth of field, thereby focusing on an area at or immediately adjacent to the vehicle window.

26. The vehicular sensor in claim 25, wherein said imaging array sensor and said optic are oriented relative to the window according to the Scheimpflug condition.

27. The vehicular sensor in claim 1 including an illumination device for illuminating the window.

28. The vehicular sensor in claim 27, wherein said filtering process is activated when said illumination device is activated.

29. The vehicular sensor in claim 28, wherein said filtering process is deactivated when said illumination device is deactivated.

30. The vehicular sensor in claim 27, wherein said illumination device provides illumination having an illumination wavelength approximately that of an infrared wavelength.

31. The vehicular sensor in claim 27, wherein said illumination device provides illumination having an illumination wavelength approximately within a range between 820 and 880 nanometers.

32. The vehicular sensor in claim 27 further including an illumination device control for at least occasionally activating said illumination device.

33. The vehicular sensor in claim 32, wherein said illumination device is pulsed on and off by said illumination device control.

34. The vehicular sensor in claim 33, wherein during low light conditions, said imaging array sensor is exposed to the vehicle window when said illumination device is pulsed on and not exposed to the window when said illumination device is pulsed off.

35. The vehicular sensor in claim 32, wherein said illumination device control activates said illumination device during low light conditions.

36. The vehicular sensor in claim 35, wherein said illumination device control senses ambient light and activates said illumination device in low ambient light conditions.

37. The vehicular sensor in claim 35, wherein said imaging array sensor includes an array of imaging pixels, said illumination device control determines a level of ambient light by monitoring light received by said pixels on said imaging array, said illumination device control switching to an active mode where said illumination device control activates said illumination device when light received by said pixels is below a predetermined threshold value.

38. The vehicular sensor in claim 37, wherein said filtering process is selectively operable such that said filtering process is activated when said illumination device control is in said active mode.

39. The vehicular sensor in claim 38, wherein said filtering process is operable to iteratively adjust values associated with an adjusted pixel in response to a calculated value associated with at least one neighboring pixel within a range of the adjusted pixel.

40. The vehicular sensor in claim 39, wherein said control comprises a computer programmed with an edge detection algorithm which detects edges of particles of precipitation at the window, said filtering process processing the edges detected by said edge detection algorithm, said control responding to control a windshield wiper on the vehicle when a threshold level of filtered edges at the window is processed.

41. The vehicular sensor in claim 40, wherein said threshold level varies according to a level of ambient light present on the window.

42. The vehicular sensor in claim 41, wherein said filtering process is one of a Median filter, a Sigma filter and a Nagao-Matsuyama filter.

43. The vehicular sensor in claim 36, wherein said illumination device control activates said illumination device when the ambient light conditions are below approximately 250 lux.

44. The vehicular sensor in claim 36, wherein said illumination device control activates said illumination device when the ambient light conditions are below approximately 150 lux.

45. The vehicular sensor in claim 36, wherein said illumination device control activates said illumination device when the ambient light conditions are below approximately 100 lux.

46. The vehicular sensor in claim 36, wherein said illumination device control activates said illumination device in response to when a vehicle'"'"'s headlamps are activated.

47. The vehicular sensor in claim 27 including a polarizing filter along an optic path between said illumination device and said imaging array sensor.

48. The vehicular sensor in claim 47, wherein said polarizing filter is oriented to filter out light signals radiating from fog particles at an interior surface of the window.

49. The vehicular sensor in claim 1, wherein said imaging array sensor is a CMOS sensor.

50. The vehicular sensor in claim 1, wherein the window is a vehicle windshield and the vehicle includes a windshield wiper and a rear window wiper, wherein said control causes the rear window wiper to cycle for every N cycles of the windshield wiper, wherein N is greater than one.

51. The vehicular sensor in claim 50, wherein the value of N varies as a function of the speed of the windshield wiper.

52. The vehicular sensor in claim 1, wherein said control comprises a micro-computer having one of an embedded control application, a custom digital logic circuit and a digital signal processor circuit.

53. The vehicular sensor in claim 52, wherein said control is adaptable to be positioned substantially adjacent a rear view mirror housing.

54. A vehicular rain sensor which senses precipitation at a vehicle window, comprising:

at least one illumination source and at least one illumination sensor defining at least one optical path from said at least one illumination source to the window and from the window to said at least one illumination sensor; and

a control which processes an electrical characteristic of said at least one illumination sensor to detect precipitation at the window, said control including a light sensing function and said control activating said at least one illumination source in low light conditions and processing said electrical characteristic using a filtering process when said at least one illumination source is activated.

55. The rain sensor in claim 54, wherein said illumination sensor is a pixelated imaging array comprising a plurality of pixels, said filtering process being operable to adjust values of at least some of said plurality of pixels in response to a calculated value, for an adjusted pixel, said calculated value being derived from a value of one or more neighboring pixels that are within a range of the adjusted pixel.

56. The rain sensor in claim 55, wherein the one or more neighboring pixels are a sub-array of pixels, the adjusted pixel being a center pixel of said sub-array of pixels.

57. The rain sensor in claim 56, wherein said sub-array being a three pixel by three pixel sub-array.

58. The rain sensor in claim 56, wherein said sub-array is a five pixel by five pixel sub-array.

59. The rain sensor in claim 55, wherein the adjusted pixel is adjusted to an average of a sub-array of pixels surrounding the adjusted pixel.

60. The rain sensor in claim 55, wherein the adjusted pixel is adjusted to an average of pixels associated with a sub-array of low variance surrounding the adjusted pixel.

61. The rain sensor in claim 55, wherein said filtering process is one of a Median filter, a Sigma filter and a Nagao-Matsuyama filter.

62. The rain sensor in claim 54, wherein said filtering process is a smoothing process.

63. The rain sensor in claim 54 further including a polarizing filter at least occasionally in said at least one optical path in a manner that polarized light from the window is more attenuated than is nonpolarized light from the window.

64. The rain sensor in claim 63, wherein said polarizing filter is at least occasionally not in said at least one optical path and wherein said control responds to an electrical characteristic of said at least one illumination sensor in order to detect precipitation at an interior surface of the window by comparing a signal from said at least one optical path to a signal from said at least one illumination sensor when said polarizing filter is not in said at least one optical path to a signal from said at least one illumination sensor when said polarizing filter is in said at least one optical path.

65. The rain sensor in claim 63, wherein said polarizing filter is at least occasionally in said at least one optical path between said at least one illumination source and the window.

66. The rain sensor in claim 63, wherein said polarizing filter is occasionally in said at least one optical path between the window and said at least one illumination sensor.

67. The rain sensor in claim 54, wherein said at least one illumination source and said at least one illumination sensor are oriented at a range of approximately 80 to 100 degrees with respect to each other along said at least one optical path.

68. The rain sensor in claim 54, wherein said at least one illumination source and said at least one illumination sensor are oriented at approximately 90 degrees with respect to each other along said at least one optical path.

69. The rain sensor in claim 54, wherein said at least one illumination sensor is an integrating sensor.

70. The rain sensor in claim 54, wherein said at least one illumination sensor is an imaging array sensor.

71. The rain sensor in claim 70, wherein said control is a computer which is programmed with an edge detection algorithm to detect edges of precipitation particles.

72. The rain sensor in claim 71, wherein said filtering process reduces effects of window irregularities in response to a size of said detected edges.

73. The rain sensor in claim 72, wherein said filtering process reduces effects of window irregularities which are of a size which is less than a size associated with edges of water and fog precipitation.

74. The rain sensor in claim 73, wherein said filtering process is an edge preserving smoothing process.

75. The rain sensor in claim 71, wherein said at least one illumination sensor is at least one pixelated imaging array sensor comprising a plurality of pixels, said filtering process being operable to iteratively adjust values of some of said plurality of pixels in response to a calculated value, for an adjusted pixel, said calculated value being derived from a value of at least one neighboring pixel within a range of the adjusted pixel.

76. The rain sensor in claim 75, wherein the at least one neighboring pixel is a sub-array of pixels surrounding the adjusted pixel.

77. The rain sensor in claim 75, wherein said calculated value is an average of values associated with the at least one neighboring pixel.

78. The rain sensor in claim 75, wherein said filtering process is one of a Median filter, a Sigma filter and a Nagao-Matsuyama filter.

79. The rain sensor in claim 54, wherein said filtering process reduces effects of window irregularities in response to a size of images sensed by said at least one illumination sensor.

80. The rain sensor in claim 79, wherein said filtering process reduces effects of window irregularities which are of a size which is less than a size associated with water and fogging precipitation.

81. The rain sensor in claim 54, wherein said at least one illumination sensor is a CMOS sensor.

82. The rain sensor in claim 54, wherein the vehicle includes a windshield wiper and a rear window wiper, wherein said control causes the rear window wiper to cycle for every N cycles of the windshield wiper, wherein N is greater than 1.

83. The rain sensor in claim 82, wherein the value of N varies as a function of the speed of the windshield wiper.

84. The rain sensor in claim 54 including an illumination control which measures a level of ambient light that is present on the window and switches said rain sensor from an active mode, where said at least one illumination source is on and provides light to the window, to a passive mode, where said at least one illumination source is off and ambient light provides illumination to said at least one illumination sensor, said illumination control being in said passive mode when said ambient light level is greater than a predetermined threshold value.

85. The rain sensor in claim 84, wherein said filtering process is operable when said rain sensor is in said active mode.

86. The rain sensor in claim 54, wherein said control does not use said filtering process when said at least one illumination source is deactivated.

87. A vehicular sensor adaptable to receive an image through a vehicle window, said sensor comprising:

an imaging sensor directed at the vehicle window from inside the vehicle, said imaging sensor capturing an image; and

a control which processes an output of said imaging sensor to produce image data of the image captured by said imaging sensor, said control including a filtering process in order to reduce effects of vehicle window irregularities on the image data.

88. The vehicular sensor in claim 87, wherein said imaging sensor is a pixelated imaging array sensor comprising a plurality of pixels.

89. The vehicular sensor in claim 88, wherein said filtering process adjusts a value of some of said plurality of pixels in response to a calculated value, for an adjusted pixel, said calculated value being derived from a value of at least one neighboring pixel within a range of the adjacent pixel.

90. The vehicular sensor in claim 89, wherein said calculated value is derived from an average of at least two neighboring pixels within a range of the adjusted pixel.

91. The vehicular sensor in claim 89, wherein said filtering process is one of a Median filter, a Sigma filter and a Nagao-Matsuyama filter.

92. The vehicular sensor in claim 89 further including an illumination source which is at least occasionally operable by said control for providing illumination to the vehicle window.

93. The vehicular sensor in claim 92, wherein said filtering process is operable when said illumination source is activated.

94. The vehicular sensor in claim 93, wherein said filtering process is deactivated when said illumination source is deactivated.

95. The vehicular sensor in claim 92, wherein said control includes first and second filtering processes, said first filtering process being operable when said illumination source is activated and said second filtering process being operable when said illumination source is not activated.

96. The vehicular sensor in claim 87, wherein said image data is precipitation data.

97. The vehicular sensor in claim 87, wherein said image data is a signature of oncoming headlights and leading tail lights.

98. The vehicular sensor in claim 87, wherein said image data is associated with a scene rearward of the vehicle.

99. The vehicular sensor in claim 98, wherein the window is a transparent panel of a rearward portion of the vehicle.

100. The vehicular sensor in claim 87, wherein said filtering process is a smoothing process.

101. The vehicular sensor in claim 87, wherein said filtering process reduces effects of window irregularities in response to a size of images sensed by said imaging sensor.

102. The vehicular sensor in claim 101, wherein said filtering process reduces effects of window irregularities which are of a size which is less than a size associated with water and fog precipitation.