Headlight, Taillight And Streetlight Detection

• US 20070221822A1
• Filed: 03/22/2007
• Published: 09/27/2007
• Est. Priority Date: 03/24/2006
• Status: Active Grant

• Current Assignee: MobilEye Vision Technologies Ltd. (Intel Corporation)
• Original Assignee: Mobileye Technologies Limited (Intel Corporation)
• Inventors: Stein, Gideon P., Wolfovitz, Uri, Hadassi, Ofer, Ben Haim, Noam
• Granted Patent: US 7,566,851 B2
• Time in Patent Office: Days
• US Class Current: 250/205
• CPC Class Codes: B60Q 1/143 combined with another condition, e.g. using vehicle recognition from camera images or activation of wipers
• CPC Class Codes: B60Q 2300/3321 Detection of streetlights
• CPC Class Codes: B60Q 2300/41 preceding vehicle
• CPC Class Codes: B60Q 2300/42 oncoming vehicle
• CPC Class Codes: B60Q 2300/45 Special conditions, e.g. pedestrians, road signs or potential dangers
• CPC Class Codes: G06V 20/582 of traffic signs
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: G08G 1/167 Driving aids for lane monitoring, lane changing, e.g. blind spot detection

Abstract

A method in a computerized system including an image sensor mounted in a moving vehicle. The image sensor captures image frames consecutively in real time. In one of the image flames, a spot is detected of measurable brightness; the spot is matched in subsequent image frames. The image frames are available for sharing between the computerized system and another vehicle control system. The spot and the corresponding spot are images of the same object. The object is typically one or more of headlights from an oncoming vehicle, taillights of a leading vehicle, streetlights, street signs and/or traffic signs. Data is acquired from the spot and from the corresponding spot. By processing the data, the object (or spot) is classified. producing an object classification. The vehicle control system controls preferably headlights of the moving vehicle based on the object classification. The other vehicle control system using the image frames is one or more of: lane departure warning system, collision warning system and/or ego-motion estimation system.

Claims

1. In a computerized vehicle control system including an image sensor mounted on a moving vehicle, wherein the image sensor captures consecutively in real time a plurality of image frames, a method comprising the steps of:

(a) in at least one of the image frames, detecting a spot of measurable brightness;

(b) matching in at least one subsequent image frame of the image frames, a corresponding spot, wherein said spot and said corresponding spot are images of an object;

(c) acquiring data respectively from said spot and from said corresponding spot; and

(d) processing said data, thereby classifying the object based on said data, and producing an object classification, wherein substantially all the image frames are available to the computerized vehicle control system and at least one other vehicle control system.

2. The method, according to claim 1, wherein said object is selected from the group consisting of:

headlights from an oncoming vehicle, taillights of a leading vehicle, streetlights, street sips and traffic signs.

3. The method, according to claim 1, wherein the vehicle control system controls headlights of the moving vehicle based on said object classification and said at least one other vehicle control system is selected from the group consisting of lane departure warning system, collision warning system and ego-motion estimation system.

4. The method, according to claim 1, wherein said object classification is provided to and used by said at least one other vehicle control system.

5. The method, according to claim 1, wherein said data relates to at least one property of said spot and said corresponding spot, wherein said at least one property is selected from the group consisting of position in said one image frame, shape, brightness, motion, color and spatial alignment.

6. The method, according to claim 1, further comprising the step of:

(e) tracking motion of said spot by comparing respective image frame positions of said spot and said corresponding spot.

7. The method, according to claim 6, said classifying said object not as a street sign when said motion is outward and upward.

8. The method, according to claim 6, further comprising the steps of:

(f) deactivating high beams upon said classifying said object as being a portion of a vehicle selected from the group consisting of;

a passing vehicle, a preceding vehicle and an oncoming vehicle;

(g) reactivating high beams based on said tracking motion.

9. The method, according to claim 1, wherein said data is related to a shape of said spot, wherein said shape is indicative of said spot splitting into a plurality of spots in said at least one subsequent image frame.

10. The method, according to claim 1, further comprising the steps of:

(e) in said at least one image frame, detecting a second spot of measurable brightness; and

(f) tracking motion of said spot and a second motion of said second spot between said at least one image frame and said at least one subsequent image frame.

11. The method, according to claim 10, pairing said spot and said second spot based on comparing said motion to said second motion

12. The method, according to claim 10, said classifying said object as a taillight when said motion and said second motion is inward.

13. The method, according to claim 10, further comprising the step of, prior to said tracking motion:

(g) compensating for yaw motion of the vehicle.

14. The computerized system used to perform the method steps of claim 1

15. In a computerized system including an image sensor mounted on a moving vehicle, wherein the image sensor captures in real time an image frame, a method comprising the steps of.

(a) detecting in the image frame a plurality of spots of measurable brightness, wherein said spots are respective images of a plurality of objects;

(b) acquiring data from said spots; and

(c) processing said data, thereby classifying said objects based on said data;

wherein said classifying is performed by previously training with a plurality of known images.

16. The method, according to claim 15, wherein the image sensor includes a filter said filter having a spatial profile including at least one portion preferentially transmitting red light, further comprising the step of:

(d) correlating said spatial profile with at least one of said spots.

17. The method, according to claim 15, wherein said known images include images from objects selected from the group consisting of:

taillights, headights of an oncoming vehicle, streetlights, and traffic signs

18. The method, according to claim 15, further comprising the step of:

(d) upon classifying said objects as taillights of a leading vehicle, determining distance to said leading vehicle; and

(e) adaptively controlling headlights of the moving vehicle based on said distance.

19. The method, according to claim 15, further comprising the step of:

(d) upon classifying said objects as at least three streetlights along a road, determining curvature of said road; and

(e) adaptively controlling headlights of the moving vehicle based on said curvature.

20. The method, according to claim 15, wherein said detecting includes determining a quadrant of said spots within said image frame, further comprising the step of:

(d) controlling solely one headlight of the moving vehicle based on said quadrant.

21. The method, according to claim 15, wherein said training and said classifying are performed using radial basis functions.

22. The method, according to claim 15, wherein said classifying uses pairs of said spots to identify vehicle taillights.

23. The method, according to claim 15, wherein said classifying includes spatial alignment of at least a portion of said spots to detect streetlights.

24. The method, according to claim 15, wherein texture characteristics of said spots are used for said classifying.

25. The method, according to claim 15, wherein edge characteristics of said spots are used for said classifying.

26. The method, according to claim 15, wherein for each of said spots, said classifying uses said data from an area centered around each said spot including N by N picture elements of the image frame.

27. The computerized system used to perform the method steps of claim 15.

28. In a computerized system including an image sensor mounted on a moving vehicle, wherein the image sensor captures in real time an image frame, a method comprising the steps of:

(a) detecting in the image frame a plurality of spots of measurable brightness, wherein said spots are respective images of a plurality of objects;

(b) matching in at least one subsequent image frame of the image frames, a plurality of corresponding spots, wherein said corresponding spots are respective images of said objects;

(c) acquiring data from said spots and from said corresponding spots; and

(d) processing said data, thereby classifying said objects based on said data;

wherein said classifying includes previously training with a plurality of known images;

(e) tracking motion of said spots by comparing respective image frame portions of said spots and said corresponding spots;

(f) deactivating high beams upon said classifying said objects as being a portion of a vehicle selected from the group consisting of;

a passing vehicle, a preceding vehicle and an oncoming vehicle; and

(g) reactivating high beams based on said tracking motion

29. The computerized system used to perform the method steps of claim 28.