Detection system for assisting a driver when driving a vehicle using a plurality of image capturing devices

• US 8,405,491 B2
• Filed: 04/19/2012
• Issued: 03/26/2013
• Est. Priority Date: 06/30/2009
• Status: Active Grant

• Current Assignee: Hong Kong Polytechnic University
• Original Assignee: Hong Kong Polytechnic University
• Inventors: Fong, Bernard Cheuk Mun, Siu, Wan Chi
• Primary Examiner(s): HOFSASS, JEFFERY A
• Application Number:
  US13/451,211
• Publication Number:
  US 20120281093A1
• Time in Patent Office: 341 Days
• Field of Search: 340/425.5, 340/435, 340/436, 348/148, 348/149, 701/301
• US Class Current: 340/425.5
• CPC Class Codes: B60Q 9/005 using a video camera
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/306 using a re-scaling of images
• CPC Class Codes: B60R 2300/70 characterised by an event-triggered choice to display a specific image among a selection of captured images
• CPC Class Codes: B60R 2300/802 for monitoring and displaying vehicle exterior blind spot views
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8046 for replacing a rear-view mirror system
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A detection system (5) for assisting a driver when driving a vehicle, the system (5) comprising: a plurality of image capturing devices (98) mounted to the vehicle to capture images of the external environment of the vehicle; an image processing module (95) to identify potential hazards in real-time from the captured images and to superimpose information of the external environment relating to the identified potential hazards; and at least one display device (92) to display images processed by the image processing module in real-time; wherein the superimposed information is provided in an area of the display device (92) to ensure visual clarity of the images of the external environment.

Claims

1. A detection system for assisting a driver when driving a vehicle, the system comprising:

a plurality of image capturing devices mounted to the vehicle to capture images of the external environment of the vehicle;

an image processing module to identify potential hazards in real-time from the captured images and to superimpose information of the external environment relating to the identified potential hazards; and

at least one display device to display images processed by the image processing module in real-time;

wherein the superimposed information is provided in an area of the display device to ensure visual clarity of the images of the external environment;

wherein the image processing module comprises video extraction logics to extract and identify potential hazards, the video extraction logics detecting the variations between successive frames captured by the plurality of image capturing device; and

wherein if the video extraction logics detects variations between the frames, a tracking sequence of an identified potential hazard is activated in subsequent frames as a precautionary measure without issuing an indication or alarm to the driver while the vehicle is continuously driven in the same lane.

2. The system according to claim 1, wherein the image processing module comprises video extraction logics to extract and identify potential hazards, the video extraction logics detecting the variations between successive frames captured by the plurality of image capturing device.

3. The system according to claim 1, further comprising a camera for tracking eye movement and head movement in order to detect the behaviour of the driver in anticipation of intention, and the detected behaviour is used by the image processing module to determine objects of concern that are highlighted to the driver on the at least one display device.

4. The system according to claim 1, further comprising a switch operatively connected to a steering wheel of the vehicle to detect the abnormal drift of the vehicle, and the detected abnormal drift is used by the image processing module to determine objects of concern that are highlighted to the driver on the at least one display device.

5. The system according to claim 1, wherein the displayed images are augmented by adjusting contrast levels of the captured images in response to a background color from the external environment.

6. The system according to claim 1, further comprising an alert module to generate a speed dependent audible alert when immediate danger is detected to warn the driver not to change lane when there is a risk of colliding with an object.

7. The system according to claim 2, wherein the video extraction logics generates a hierarchy of temporal variation representations with luminance consistency that describes the illumination variations in order to estimate and compensate for global motion.

8. The system according to claim 2, wherein if user interaction is imposed, spatial segmentation is fused with a mask before fusing with temporal segmentation.