Detection System for Assisting A Driver When Driving A Vehicle

• US 20100328055A1
• Filed: 06/30/2009
• Published: 12/30/2010
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: Hong Kong Polytechnic University
• Original Assignee: Hong Kong Polytechnic University
• Inventors: Fong, Bernard Cheuk Mun, Siu, Wan Chi
• Granted Patent: US 8,174,375 B2
• Time in Patent Office: Days
• US Class Current: 340/425.5
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8046 for replacing a rear-view mirror system
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A detection system (5) for assisting a driver when driving a vehicle, the system (5) comprising: a plurality of image capturing devices (98) mounted to the vehicle to capture images of the external environment of the vehicle; an image processing module (95) to identify potential hazards in real-time from the captured images and to superimpose information of the external environment relating to the identified potential hazards; and at least one display device (92) to display images processed by the image processing module in real-time; wherein the superimposed information is provided in an area of the display device (92) to ensure visual clarity of the images of the external environment.

Claims

1. A detection system for assisting a driver when driving a vehicle, the system comprising:

a plurality of image capturing devices mounted to the vehicle to capture images of the external environment of the vehicle;

an image processing module to identify potential hazards in real-time from the captured images and to superimpose information of the external environment relating to the identified potential hazards; and

at least one display device to display images processed by the image processing module in real-time;

wherein the superimposed information is provided in an area of the display device to ensure visual clarity of the images of the external environment.

2. The system according to claim 1, wherein the image processing module comprises video extraction logics to extract and identify potential hazards, the video extraction logics detecting the variations between successive frames captured by the plurality of image capturing devices.

3. The system according to claim 1, further comprising a camera for tracking eye movement and head movement in order to detect the behaviour of the driver in anticipation of intention, and the detected behaviour is used by the image processing module to determine objects of concern that are highlighted to the driver on the at least one display device.

4. The system according to claim 1, further comprising a switch operatively connected to a steering wheel of the vehicle to detect the abnormal drift of the vehicle, and the detected abnormal drift is used by the image processing module to determine objects of concern that are highlighted to the driver on the at least one display device.

5. The system according to claim 1, wherein the displayed images are augmented by adjusting contrast levels of the captured images in response to a background color from the external environment.

6. The system according to claim 1, further comprising an alert module to generate a speed dependent audible alert when immediate danger is detected to warn the driver not to change lane when there is a risk of colliding with an object.

7. The system according to claim 2, wherein if the video extraction logics detects variations between the frames, a tracking sequence of an identified potential hazard is activated in subsequent frames as a precautionary measure without issuing an indication to the driver while the vehicle is continuously driven in the same lane.

8. The system according to claim 7, further comprising a spatial segmentation module and a temporal segmentation module to estimate spatial and temporal separation based on relative traveling speed defected between two sequential frames.

9. The system according to claim 8, wherein spatial segmentation is performed in each frame to improve accuracy of the video extraction logics or when large variations in the background between successive frames are detected.

10. The system according to claim 8, wherein spatial and temporal segmentation and fusion are adaptively performed by detecting the variation between successive frames to reduce computational complexity and improve data processing speed.

11. The system according to claim 7, further comprising a variation computation module to execute an adaptive frame skipping algorithm to decompose the successive frames to preclude error augmentation and propagating during the tracking sequence.

12. The system according to claim 2, wherein the video extraction logics generates a hierarchy of temporal variation representations with luminance consistency that describes the illumination variations in order to estimate and compensate for global motion.

13. The system according to claim 2, wherein if user interaction is imposed, spatial segmentation is fused with a mask before fusing with temporal segmentation.