Driver's aid using image processing

• US 20030112132A1
• Filed: 12/14/2001
• Published: 06/19/2003
• Est. Priority Date: 12/14/2001
• Status: Active Grant

• Current Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Original Assignee: Koninklijke Philips Electronics N.V. (Koninklijke Philips N.V.)
• Inventors: Trajkovic, Miroslav, Gutta, Srinivas, Colmenarez, Antonio Jose
• Granted Patent: US 6,727,807 B2
• Time in Patent Office: Days
• US Class Current: 340/435
• CPC Class Codes: B60K 2360/785 on or in relation to the windshield or windows
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/60 Instruments characterised by their location or relative disposition in or on vehicles arrangements of lighting devices on dashboards B60Q3/10
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/30 characterised by the type of image processing
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/308 by overlaying the real scene, e.g. through a head-up display on the windscreen

Abstract

A system and method for alerting a driver of an automobile to a traffic condition. The system comprises at least one camera having a field of view and facing in the forward direction of the automobile. The camera captures images of the field of view in front of the automobile. A control unit receives images of the field of view from the camera and identifies objects therein of a predetermined type. The control unit analyzes the object images of at least one predetermined type to determine whether one or more of the identified objects present a condition that requires the driver'"'"'s response. A display receives control signals from the control unit regarding the one or more of the objects that present a condition that requires the driver'"'"'s response. The display displays an image of the object to the driver that is positioned and scaled so that it overlays the actual object as seen by the driver, the displayed image of the object enhancing a feature of the actual object to alert the driver.

Claims

1. A system for alerting a driver of an automobile to a traffic condition, the system comprising:

a) at least one camera having a field of view and facing in the forward direction of the automobile and capturing images of the field of view in front of the automobile, b) a control unit that receives images of the field of view from the at least one camera and identifies objects therein of at least one predetermined type and analyzes the object images to determine whether any of the identified objects present a condition that requires the driver'"'"'s response, and c) a display that receives control signals from the control unit regarding one or more identified objects determined to present a condition that requires the driver'"'"'s response and displays an image of the one or more objects to the driver that is positioned and scaled so that it overlays the actual object as seen by the driver, the displayed image of the object enhancing a feature of the actual object to alert the driver.

2. The system as in claim 1, wherein the objects of at least one predetermined type comprise at least one selected from the group of:

pedestrian, other automobiles, traffic signs, traffic controls, and road obstructions.

3. The system as in claim 1, wherein the control unit further identifies features in the image of at least one predetermined type.

4. The system as in claim 3, wherein the features of at least one predetermined type comprise borders of the roadway.

5. The system as in claim 3, wherein the control unit identifies borders of the roadway in the image and determines whether the movement of the automobile is inconsistent with the borders of the roadway.

6. The system as in claim 1, wherein the control unit identifies pedestrians in the image and determines whether the pedestrian and the automobile are on a collision course.

7. The system as in claim 1, wherein the control unit identifies traffic signs in the image and determines whether the movement of the automobile is inconsistent with the traffic signs.

8. The system as in claim 1, wherein the control unit identifies traffic signals in the image and determines whether the movement of the automobile is inconsistent with the traffic signals.

9. The system as in claim 1, wherein the control unit identifies other automobiles in the image and determines whether any of the other automobiles and the automobile are on a collision course.

10. The system as in claim 1, wherein the display is a head-up display (HUD) that projects an image of the one or more identified objects that requires the driver'"'"'s response on a windshield of the automobile.

11. The system as in claim 10, wherein each image of the one or more identified objects projected by the HUD on the windshield is positioned and scaled to overlay the corresponding actual objects as seen by the driver through the windshield.

12. The system as in claim 10, wherein the border of the projected image of the object is enhanced to gain the driver'"'"'s attention.

13. The system as in claim 1, wherein the system comprises two cameras each facing in the forward direction of the automobile from different positions and each having a different respective field of view, the two cameras each capturing images of their respective field of view in front of the automobile.

14. The system as in claim 13, wherein the control unit receives the images of the field of views from the respective cameras, identifies object images of at least one predetermined type in the images from each of the respective cameras and analyzes the object images in the images received from each of the respective cameras to determine whether one or more of the identified objects present a condition that requires the driver'"'"'s response.

15. The system as in claim 14, wherein, when the control unit determines that an identified object presents a condition that requires the driver'"'"'s response, the image of the identified object in images received from each of the respective cameras is used by the control unit to determine the distance between a point on the automobile and the object.

16. The system as in claim 15, wherein the control unit determines the distance between a point at automobile and the object using stereo techniques.

17. The system as in claim 16, wherein the point at the automobile is located approximately at the position of the driver'"'"'s head when driving the automobile.

18. The system as in claim 17, wherein the control unit generates control signals for the display unit that designates the position and scale of the image of the object using the distance between the driver'"'"'s head and the object.

19. A method for alerting a driver of an automobile to a traffic condition, the method comprising the steps of:

a) capturing images of the field of view in front of the automobile, b) identifying objects in the image of at least one predetermined type and analyzing the objects to determine whether one or more of the identified objects present a condition that requires the driver'"'"'s response, and c) displaying to the driver an image of at least one of the objects identified in the image that require the driver'"'"'s response, the displayed image being positioned and scaled so that it overlays the actual object as seen by the driver, the displayed image of the object enhancing a feature of the actual object to alert the driver.

20. The method as in claim 19, wherein the displayed image is projected onto a windshield of the automobile.

21. The method as in claim 19, wherein the step of identifying objects in the image of at least one predetermined type and analyzing the objects to determine whether one or more of the identified objects present a condition that requires the driver'"'"'s response includes identifying at least one of pedestrians and other automobiles in the image and determining whether any identified pedestrians and other automobiles are on a collision course with the automobile.

22. The method as in claim 19, wherein the step of identifying objects in the image of at least one predetermined type and analyzing the objects to determine whether one or more of the identified objects present a condition that requires the driver'"'"'s response includes identifying at least one of traffic signs and traffic signals in the image and determining whether the movement of the automobile is inconsistent with any identified traffic signs and traffic signals.