Image acquisition and processing methods for automatic vehicular exterior lighting control

• US 20040143380A1
• Filed: 08/20/2003
• Published: 07/22/2004
• Est. Priority Date: 08/21/2002
• Status: Active Grant

• Current Assignee: Gentex Corporation
• Original Assignee: Gentex Corporation
• Inventors: Roberts, John K., Pierce, Mark W., Walstra, Eric J., Bush, Gregory S., Stam, Joseph S., Bechtel, Jon H., Berends, Keith H., Rycenga, Brock R., Mart, Gregory A.
• Granted Patent: US 7,565,006 B2
• Time in Patent Office: Days
• US Class Current: 701/36
• CPC Class Codes: B60Q 1/085 due to special conditions, e.g. adverse weather, type of road, badly illuminated road signs or potential dangers B60Q1/10, B60Q1/12, B60Q1/1423 take precedence
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 1/18 being additional front lights bracket mounted headlights B60Q1/0483
• CPC Class Codes: B60Q 2300/052 Switching delay, i.e. the beam is not switched or changed instantaneously upon occurrence of a condition change
• CPC Class Codes: B60Q 2300/054 Variable non-standard intensity, i.e. emission of various beam intensities different from standard intensities, e.g. continuous or stepped transitions of intensity
• CPC Class Codes: B60Q 2300/056 Special anti-blinding beams, e.g. a standard beam is chopped or moved in order not to blind
• CPC Class Codes: B60Q 2300/112 Vehicle speed
• CPC Class Codes: B60Q 2300/122 Steering angle
• CPC Class Codes: B60Q 2300/132 Pitch
• CPC Class Codes: B60Q 2300/134 Yaw
• CPC Class Codes: B60Q 2300/21 Manual control
• CPC Class Codes: B60Q 2300/312 Adverse weather
• CPC Class Codes: B60Q 2300/314 Ambient light
• CPC Class Codes: B60Q 2300/322 Road curvature
• CPC Class Codes: B60Q 2300/331 characterised by the driving side, e.g. on the left or right hand side
• CPC Class Codes: B60Q 2300/332 on city roads
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: H05B 47/165 : following a pre-assigned programmed sequence; Logic control [LC]
• CPC Class Codes: H05B 47/17 : Operational modes, e.g. switching from manual to automatic mode or prohibiting specific operations
• CPC Class Codes: Y02B 20/40 : Control techniques providing energy savings, e.g. smart controller or presence detection

Abstract

The present invention relates to various apparatus, algorithms and methods for acquiring and processing images of a scene. Details of various aspects of the associated images are identified and may be utilized to generate various vehicular equipment control signals.

Claims

1. An automatic vehicular exterior light control, comprising:

a controller configured to generate at least one exterior light control signal as a function of a classification network, said controller is further configured to execute an algorithm comprising at least one transition state selected from the group comprising;

an on state to off state transition state and an off state to on state transition state, wherein the classification network is trained using light sources classified using expert knowledge.

2. An automatic vehicular exterior light control as in claim 1 wherein said classification network is selected from the group comprising:

a neural network and a probability function.

3. An automatic vehicular exterior light control as in claim 1 wherein said expert knowledge is selected from the group comprising:

empirical data, experimental data, statistical data and manually classified data.

4. An automatic vehicular exterior light control, comprising:

a controller configured to generate at least one exterior light control signal as a function of a neural network analysis.

5. An automatic vehicular exterior light control as in claim 4 wherein said neural network analysis comprises:

a plurality of inputs and a plurality of weights, at least one of which is associated with each input.

6. An automatic vehicular exterior light control as in claim 5 further comprising at least one output, wherein said at least one output is based upon at least one of the group comprising:

the sum of the inputs, the products of the inputs, the sum of the inputs with associated weighting factors and the products of the inputs with associated weighting factors.

7. An automatic vehicular exterior light control as in claim 4 wherein said neural network analysis further comprises:

at least one hidden layer node; and

at least one weighting factor, wherein each hidden layer node is associated with at least one weighting factor.

8. An automatic vehicular exterior light control as in claim 7 wherein the value of each hidden layer node is based upon the product of at least one or more input and at least one weighting factor associated with each input.

9. An automatic vehicular exterior light control as in claim 8 wherein said exterior light control signal is based upon the product of at least one hidden layer node and the associated weights.

10. An automatic vehicular exterior light control as in claim 4 wherein said input variables are selected from the group of light source characteristics comprising:

peak brightness, total brightness, centroid location, gradient, width, height, color, x-direction motion, y-direction motion, brightness change, age, average x-direction motion, average y-direction motion, motion jitter, a change in brightness that correlates to a change in brightness of an exterior light of a controlled vehicle and average brightness change.

11. An automatic vehicular exterior light control as in claim 4 wherein said input variables are selected from the group of controlled vehicle associated operating parameters comprising:

vehicle speed, ambient light level, vehicle turn rate, lane tracking, vehicle pitch, vehicle yaw, geographic location and road type.

12. An automatic vehicular exterior light control as in claim 11 wherein said vehicle turn rate is determined via at least one of the items selected from the group comprising:

steering wheel angle, a compass, wheel speed, GPS and image analysis results.

13. An automatic vehicular exterior light control as in claim 4 wherein said neural network further comprising at least one output selected from the group comprising:

a Boolean true-false value and a substantially continuous value indicative of a probability.

14. An automatic vehicular exterior light control as in claim 4 wherein said controller is further configured to determine whether at least one light source is either a headlight of an oncoming vehicle, a taillight of a leading vehicle or a non-vehicular light source as a function of said neural network analysis.

15. An automatic vehicular exterior light control as in claim 14 wherein said determination is further a function of the brightness of the light source.

16. An automatic vehicular exterior light control as in claim 14 wherein said determination is further a function of any AC flicker that may be present in the light source.

17. An automatic vehicular exterior light control as in claim 4 wherein said neural network is trained utilizing empirical data.

18. An automatic vehicular exterior light control as in claim 17 wherein said empirical data is obtained by analyzing at least one image comprising known light sources.

19. An automatic vehicular exterior light control as in claim 4 comprising twenty three input variables.

20. An automatic vehicular exterior light control, comprising:

a controller configured to generate at least one exterior light control signal as a function of at least one probability function, wherein said at least one probability function comprises a plurality of variables and a substantially continuous output value indicative of a probability.

21. An automatic vehicular exterior light control as in claim 20 wherein said variables are selected from the group of light source characteristics comprising:

peak brightness, total brightness, centroid location, gradient, width, height, color, x-direction motion, y-direction motion, brightness change, age, average x-direction motion, average y-direction motion, motion jitter, a change in brightness that correlates to a change in brightness of an exterior light of a controlled vehicle and average brightness change.

22. An automatic vehicular exterior light control as in claim 20 wherein said variables are selected from the group of controlled vehicle associated operating parameters comprising:

vehicle speed, ambient light level, vehicle turn rate, lane tracking, vehicle pitch, vehicle yaw, geographic location and road type.

23. An automatic vehicular exterior light control as in claim 22 wherein said vehicle turn rate is determined via at least one of the items selected from the group comprising:

steering wheel angle, a compass, wheel speed, GPS and image analysis results.

24. An automatic vehicular exterior light control as in claim 20 wherein said controller is further configured to determine whether at least one light source is either a headlight of an oncoming vehicle, a taillight of a leading vehicle or a non-vehicular light source as a function of said probability function.

25. An automatic vehicular exterior light control as in claim 24 wherein said determination is further a function of the brightness of the light source.

26. An automatic vehicular exterior light control as in claim 24 wherein said determination is further a function of any AC flicker that may be present in the light source.

27. An automatic vehicular exterior light control as in claim 20 wherein said probability function is selected from the group comprising:

a first order equation, a second order equation, a third order equation and a fourth order equation.

28. An automatic vehicular exterior light control, comprising:

a controller configured to generate at least one exterior light control signal as a function of at least one probability function, wherein said at least one probability function comprises a plurality of variables, a plurality of weighting factors and an output.

29. An automatic vehicular exterior light control as in claim 28 wherein said variables are selected from the group of light source characteristics comprising:

peak brightness, total brightness, centroid location, gradient, width, height, color, x-direction motion, y-direction motion, brightness change, age, average x-direction motion, average y-direction motion, motion jitter, a change in brightness that correlates to a change in brightness of an exterior light of a controlled vehicle and average brightness change.

30. An automatic vehicular exterior light control as in claim 28 wherein said variables are selected from the group of controlled vehicle associated operating parameters comprising:

vehicle speed, ambient light level, vehicle turn rate, lane tracking, vehicle pitch, vehicle yaw, geographic location and road type.

31. An automatic vehicular exterior light control as in claim 30 wherein said vehicle turn rate is determined via at least one of the items selected from the group comprising:

steering wheel angle, a compass, wheel speed, GPS and image analysis results.

32. An automatic vehicular exterior light control as in claim 31 wherein said controller is further configured to determine whether at least one light source is either a headlight of an oncoming vehicle, a taillight of a leading vehicle or a non-vehicular light source as a function of said probability function.

33. An automatic vehicular exterior light control as in claim 32 wherein said determination is further a function of the brightness of the light source.

34. An automatic vehicular exterior light control as in claim 32 wherein said determination is further a function of any AC flicker that may be present in the light source.

35. An automatic vehicular exterior light control as in claim 28 wherein said at least one output is selected from the group comprising:

a Boolean true-false value and a substantially continuous value indicative of a probability.

36. An automatic vehicular exterior light control as in claim 28 wherein said weighting factors are determined experimentally by examining at least one image containing at least one known light source.

37. An automatic vehicular exterior light control as in claim 28 wherein said weighting factors are determined by examining statistical data.

38. An automatic vehicular exterior light control as in claim 37 wherein said statistical data is derived from a plurality of images containing known light sources.

39. An automatic vehicular exterior light control as in claim 28 wherein said probability function is selected from the group comprising:

a first order equation, a second order equation, a third order equation and a fourth order equation.

40. An automatic vehicular exterior light control, comprising:

a controller configured to generate an exterior light control signal, said controller is further configured to execute an algorithm comprising at least one transition state selected from the group comprising;

an on state to off state transition state and an off state to on state transition state.

41. An automatic vehicular exterior light control as in claim 40 wherein said off state transition state is entered when at least one of the conditions is satisfied selected from the group comprising:

scene free of headlamps and tail lamps with brightness above a threshold, less than threshold number of AC lights in image, less than threshold number of lights in the image, threshold number of continuous clear cycles reached, controlled vehicle speed above threshold, controlled vehicle steering wheel angle magnitude below threshold value, HOLD timer elapsed, INACTIVITY timer elapsed, TAILLAMP OVERTAKE timer, FOG condition clear, RAIN condition clear, street lamp density below threshold and traffic density delay.

42. An automatic vehicular exterior light control as in claim 40 wherein said on state transition state is entered when at least one light source is detected.

43. An automatic vehicular exterior light control as in claim 40 wherein at least one of said transition states comprises a series of levels and movement between levels is a function of at least one of the variables selected from the group comprising:

light source brightness, light source position, confidence of classification, light source type, controlled vehicle speed, and controlled vehicle turn rate.

44. An automatic vehicular exterior light control comprising a method of classifying detected light sources, said method comprising the steps of:

classifying at least one detected light source with a classification network, wherein an output of said classification network is indicative of the likelihood that said detected light source is a headlamp of an oncoming vehicle or a tail lamp of a leading vehicle.

45. The method of claim 44 further comprising the step of:

determining the control state of at least one exterior light of the controlled vehicle based upon said output of said classification network.

46. The method of claim 44 wherein said classification network is selected from the group comprising:

a neural network and a probability function.

47. An automatic vehicular exterior light control comprising a method of classifying detected light sources, said method comprising the steps of:

classifying at least one detected light source with a classification network, wherein said classification network determines the type of light source detected based upon at least one characteristic of at least one previously classified light source verified to be accurately classified.

48. The method of claim 47 further comprising the step of:

determining the control state of at least one exterior light of the controlled vehicle based upon an output of the classification network.

49. The method of claim 47 wherein said classification network is selected from the group comprising:

a neural network and a probability function.

50. An automatic vehicular exterior light control comprising a method of classifying detected light sources, said method comprising the steps of:

classifying at least one detected light source with a trainable classification network, wherein said classification network is trained using at least one light source classified using expert knowledge.

51. The method of claim 50 wherein said expert knowledge is selected from the group comprising:

empirical data, experimental data, statistical data and manually classified data.

52. The method of claim 50 wherein said classification network is selected from the group comprising:

a neural network and a probability function.

53. The method of claim 50 further comprising the step of:

determining the control state of at least one exterior light of the control vehicle based upon an output of said classification network.

54. An automatic vehicular exterior light control, comprising a method of analyzing pixel values from an imager, said method comprising the steps of:

acquiring multiple images at multiple sensitivities,

for each examined pixel location, selecting a pixel value from one of the images for processing based upon the value of the pixel.

55. A method as in claim 54 wherein at least one of said multiple images is stored in a memory location.

56. A method as in claim 55 wherein said memory location is within an image sensor interface/memory buffer.

57. A method as in claim 54 wherein at least two of said multiple images are synthesized into a high dynamic range image.

58. An automatic vehicular exterior light control, comprising:

a controller configured to switch vehicle lights between being substantially ON and substantially OFF, wherein a time delay switching from ON to OFF is variable, and wherein said time delay is a function of the likelihood of accurate detection of a vehicle light source.

59. An automatic vehicular exterior light control of claim 58, wherein said time delay is further a function of one or more of a group comprising:

brightness of a detected light source, position of a light source, type of detected light source, color of detected light source, controlled vehicle speed, and controlled vehicle turn rate.

60. An automatic vehicular exterior light control, comprising:

a controller configured to transition vehicle lights between two or more illumination patterns, wherein the rate of transition between illumination patterns is variable, and wherein the rate of transition is a function of the confidence of detection of a vehicle light source.

61. An automatic vehicular exterior light control as in claim 60 wherein at least one vehicle light is substantially continuously variable between said illumination patterns.

62. An automatic vehicular exterior light control of claim 60, wherein said rate of transition is further a function of one or more of a group comprising:

brightness of a detected light source, position of a light source, type of detected light source, color of detected light source, controlled vehicle speed, and controlled vehicle turn rate.

63. An automatic vehicular exterior light control, comprising:

a controller configured to detect a clear condition when no other light sources of other vehicles are detected within a range, said controller is further configured to automatically activate at least one vehicle exterior light upon detection of said clear condition, wherein the time between detecting said clear condition and automatic activating the at least one vehicle exterior light is variable, and wherein said time delay is a function of vehicle speed.

64. An automatic vehicular exterior light control, comprising:

a controller configured to detect a clear condition when no'"'"'other lights of other vehicles are detected within a range, wherein automatic activation of head lamps is inhibited by one or more events of the group comprising;

threshold number of streetlights exceeded, threshold streetlight density exceeded, steering wheel angle magnitude threshold exceeded.