Intelligent headlight control using camera sensors

• US 8,493,446 B2
• Filed: 06/22/2009
• Issued: 07/23/2013
• Est. Priority Date: 04/17/2009
• Status: Expired due to Fees

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Li, Ying, Pankanti, Sharathchandra
• Primary Examiner(s): Kelley, Christopher S
• Assistant Examiner(s): Kim, Hee-Yong
• Application Number:
  US12/488,924
• Publication Number:
  US 20100265330A1
• Time in Patent Office: 1,492 Days
• Field of Search: 348/148
• US Class Current: 348/148
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/322 Road curvature
• CPC Class Codes: B60Q 2300/332 on city roads
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: G06V 10/44 Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

A system and method for intelligently controlling headlights receive a multiplicity of images that represent frames of a video sequence of an external environment of a vehicle. At least one bright spot, or blob, is found that stands out from a dark background of the external environment within each frame of the multiplicity of images. A multiplicity of features is extracted from a found blob. A type is recognized of a found blob that is selected from a multiplicity of types of blobs. A determination is then made whether to turn on a high beam light or a low beam light based at least on the recognized type of the found blob and a set of rules. Finally, an action based on such decision is performed.

Claims

1. An automated visual processing system comprising:

a memory;

an information processor and controller, communicatively coupled with the memory;

a video imaging system, communicatively coupled with the information processor and controller and the memory, for receiving a plurality of images representing frames of a video sequence of an external environment of a vehicle;

a blob detector, communicatively coupled with the video imaging system, the information processor and controller, and the memory, for identifying a blob within each frame of the plurality of images;

a blob feature extractor, communicatively coupled with the blob detector, for extracting a plurality of features from a blob found by the blob detector;

a blob classifier, communicatively coupled with the blob feature extractor and the blob detector, for recognition of a type of a blob, selected from a plurality of types of blobs, that is found by the blob detector, wherein the blob classifier is a machine learning classifier that utilizes a support vector machine model to recognize the plurality of types of blobs, wherein the support vector machine model has been trained with respect to the plurality of types of blobs; and

an event controller, communicatively coupled with the blob classifier and the blob detector, for determining a binary decision based at least on the recognized type of blob that is found by the blob classifier and a set of rules comprising at least one rule for detecting a given environment type based on calculated ambient brightness and average edge energy of frames of the plurality of images, and performing an action based on such decision.

2. The intelligent headlight control system of claim 1, wherein the binary decision is a decision to one of turn on a high headlight beam and a low headlight beam.

3. The intelligent headlight control system of claim 1, wherein the plurality of features extracted from a blob identified by the blob detector comprises at least one of the following types of features:

blob position, blob brightness, blob shape, blob spatial relation, blob color, and blob motion.

4. The intelligent headlight control system of claim 1, wherein the plurality of types of blobs recognized by the blob classifier comprises at least one of the following types of blobs:

headlight, taillight, streetlight, and other.

5. The intelligent headlight control system of claim 1, wherein the light beam controller changes a current state of a light beam of the vehicle based on the blob types recognized by the blob detector, and a given set of rules for changing the current state of the light beam when the environment type is one of a curved road and an urban area.

6. The intelligent headlight control system of claim 1, further comprising a video camera mounted around a windshield region of a vehicle and aimed toward the external environment of the vehicle, the video camera being communicatively coupled with the video imaging system for capturing the plurality of images representing frames of the video sequence of the external environment of the vehicle.

7. The intelligent headlight control system of claim 1, wherein the environment type is detected as an urban scene based on a detected number of blobs within each frame, the ambient brightness and the average edge energy of the frame.

8. The intelligent headlight control system of claim 1, wherein the set of rules comprise rules associated with an outcome of tracking light objects of specific light types being detected in the previous frame, and a decision based on a voting strategy applied to a given timing window, where different voting strategies are applied to turning on a high headlight beam and turning on a low headlight beam, respectively.

9. A method, with an information processing system, for an intelligent headlight control system, the method comprising:

receiving a plurality of images representing frames of a video sequence of an external environment of a vehicle;

identifying at least one blob that stands out from a dark background of the external environment within each frame of the plurality of images;

extracting a plurality of features from a found blob;

recognizing a type of a found blob, selected from a plurality of types of blobs;

determining whether to turn on a high beam light or a low beam light based at least on the recognized type of the found blob using a machine learning classifier and a set of rules comprising at least one rule for detecting a given environment type based on calculated ambient brightness and average edge energy of frames of the plurality of images, wherein the machine learning classifier utilizes a support vector machine model to recognize the plurality of types of blobs, and wherein the support vector machine model has been trained with respect to the plurality of types of blobs; and

performing an action based on such decision.

10. The method of claim 9, wherein the plurality of features extracted from a found blob comprises at least one of the following types of features:

blob position, blob brightness, blob shape, blob spatial relation, blob color, and blob motion.

11. The method of claim 9, wherein the plurality of types of blobs comprises at least one of the following types of blobs:

headlight, taillight, streetlight and other.

12. The method of claim 9, further comprising:

generating a beam history log;

storing a historical information of state of high beam light on or low beam light on for one or more frames of the plurality of images; and

tracking the historical information.

13. The method of claim 9, further comprising:

detecting the environment type as an urban scene based on

a detected number of blobs within each frame; and

an ambient brightness and an average edge energy of each frame.

14. The method of claim 9, further comprising:

deriving a set of rules associated with an outcome of tracking light objects of specific light types being detected in the previous frame, and a decision based on a voting strategy applied to a certain timing window, where different voting strategies are applied to turning on a high headlight beam and turning on a low headlight beam, respectively.

15. A non-transitory computer program product tangibly embodying computer readable instructions which, when implemented cause a computer to carry out the steps of a method for intelligently controlling a headlight system, the method for:

receiving a plurality of images representing frames of a video sequence of an external environment of a vehicle;

identifying at least one blob that stands out from a dark background of the external environment within each frame of the plurality of images;

extracting a plurality of features from a found blob;

recognizing a type of a found blob, selected from a plurality of types of blobs;

determining whether to turn on a high beam light or a low beam light based at least on the

recognized type of the found blob using a machine learning classifier and a set of rules comprising at least one rule for detecting a given environment type based on calculated ambient brightness and average edge energy of frames of the plurality of images, wherein the machine learning classifier utilizes a support vector machine model to recognize the plurality of types of blobs, and wherein the support vector machine model has been trained with respect to the plurality of types of blobs; and

performing an action based on such decision.

16. The non-transitory computer program product of claim 15, wherein the plurality of features extracted from a found blob comprises at least one of the following types of features:

blob position, blob brightness, blob shape, blob spatial relation, blob color, and blob motion.

17. The non-transitory computer program product of claim 15, wherein the plurality of types of blobs comprises at least one of the following types of blobs:

headlight, taillight, streetlight and other.

18. The non-transitory computer program product of claim 15, further comprising instructions for:

generating a beam history log;

storing a historical information of state of high beam light on or low beam light on for one or more frames of the plurality of images; and

tracking the historical information.

19. The computer program product of claim 15, further comprising instructions for:

detecting the environment type as an urban scene based on

a detected number of blobs within each frame; and

an ambient brightness and an average edge energy of each frame.

20. The computer program product of claim 15, wherein the method is further for:

deriving a set of rules associated with an outcome of tracking light objects of specific light types being detected in the previous frame, and a decision based on a voting strategy applied to a certain timing window, where different voting strategies are applied to turning on a high headlight beam and turning on a low headlight beam, respectively.