Real-time nighttime vehicle detection and recognition system based on computer vision

• US 7,949,190 B2
• Filed: 08/07/2006
• Issued: 05/24/2011
• Est. Priority Date: 05/12/2006
• Status: Expired due to Fees

• Current Assignee: CSSP Incorporated
• Original Assignee: National Chiao Tung University (Government of The Republic of China)
• Inventors: Chen, Yen-Lin, Chen, Chao-Jung, Chen, Yuan-Hsin, Wu, Bing-Fei
• Primary Examiner(s): Bella; Matthew C
• Assistant Examiner(s): Rosario; Dennis
• Application Number:
  US11/500,141
• Publication Number:
  US 20070263901A1
• Time in Patent Office: 1,751 Days
• Field of Search: 382/103, 382/104, 382/153, 382/174, 382/206
• US Class Current: 382/206
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/584 of vehicle lights or traffic lights

Abstract

The present invention relates to a real-time nighttime vehicle detection and identification system comprises an illuminant object image segmentation device 1, an illuminant object classifying device 2, a vehicle lighting object identification device 3, a vehicle position determining device 4 and a vehicle tracking device 5. Under various circumstances of road lighting during nighttime, the system can efficiently and accurately demarcate and identify the lamps of oncoming and preceding vehicles and accurately provides the driver with auxiliary information needed to analyze the traffic conditions in front of the vehicle during the conditions met on the road at that time.

Claims

1. A system for detecting and identifying vehicles in real-time at night-time, the system including:

an illuminant object image segmentation device for segmenting illuminant objects from images in front of a host vehicle;

an illuminant object classification device for classifying the illuminant objects segmented from the images by a connected object demarcation unit, so as to extract spatial feature of correlation between the illuminant objects as illuminant object groups;

a vehicle-light object identification device for extracting spatial feature information of each vehicle from the illuminant object groups by a pattern analyzing unit;

a vehicle position determining device for gaining relative position information between each target vehicle that appears ahead on the road and the host vehicle from the spatial feature information by a distance estimation unit; and

a vehicle tracking device for detecting a vehicle'"'"'s moving direction in accordance with a vehicle-light group targeted from the following continued image frames based on the relative position information after gaining the illuminant object groups, so as to determine a movement information of each vehicle that enters an area under surveillance;

wherein the pattern analyzing unit identifies whether an illuminant object group has characteristics of vehicle lighting features in accordance with a demarcated illuminant object group based on characteristics of horizontal and vertical projection profiles of illuminant objects with horizontal alignment of illuminant objects, and then identifies the vehicle lighting features in vehicle head lamps or vehicle tail lamps.

2. The system according to claim 1, wherein the relative position information includes a distance of the target vehicle from the lane and its relative positions.

3. The system according to claim 1, wherein the movement information includes relative direction of motion and relative velocity with regard to each of the vehicles.

4. The system according to claim 1, which further includes an automatic control mechanism, using the relative position information and the movement information, to adopt a correct operation corresponding to a cruise velocity of the vehicle and a drive route.

5. The system according to claim 1, wherein the connected object demarcation unit demarcates each of the illuminant objects and analyzes and compares the illuminant objects by utilizing a characteristics of projection analysis, size, ratio and distance of the illuminant objects to obtain a shortest distance of horizontal and vertical sub-area of connected-components of the illuminant objects and an overlapping ratio of a vertical projection of the sub-area of the connected-components.

6. The system according to claim 1, wherein the distance estimation unit estimates a perspective depth of field of target vehicle on the imaginary and real coordinate axes system at a particular timeline on the basis of a perspective image module, and derives a relative spatial position between the target vehicle and the host vehicle on the lane using the corresponding relation between a coordinate position of pixel and the perspective depth of field of the target vehicle in the image.

7. The system according to claim 6, wherein the relative space position includes distance of perspective depth of field, position of left-hand edge and position of right-hand edge.

8. The system according to claim 1, further comprising a switch controller for switching the head lamps of the host vehicle to a high beam or a low beam.

9. A method for detecting and identifying vehicles in real-time at night-time, the method including:

step of illuminant object image segmentation, segmenting illuminant objects from images in front of a host vehicle;

step of illuminant object classification, classifying the illuminant objects segmented from the images by a connected object demarcation unit so as to generalize spatial feature correlation between the illuminant objects as illuminant object groups;

step of vehicle-light object identification, extracting spatial feature information of each vehicle from the illuminant object groups by a pattern analyzing unit;

step of vehicle position determination, gaining relative position information between each target vehicle that appears ahead on the road and the host vehicle from the spatial feature information by a distance estimation unit; and

step of vehicle tracking, for detecting a vehicle'"'"'s moving direction in accordance with a vehicle-light group targeted from each continued image frames based on the relative position information after gaining the illuminant object groups, so as to determine a movement information of each vehicle that enters an area under surveillance wherein the pattern analyzing unit identifies whether an illuminant object group has characteristics of vehicle lighting features in accordance with a demarcated illuminant object group based on characteristics of horizontal and vertical projection profiles of illuminant objects with horizontal alignment of illuminant objects, and then identifies the vehicle lighting features in vehicle head lamps or vehicle tail lamps.

10. The method according to claim 9, wherein the relative position information includes a distance of the target vehicle from the lane and its corresponding positions.

11. The method according to claim 9, wherein the movement information includes relative direction of motion and relative velocity with regard to each of vehicles.

12. The method according to any one of claim 9, which further includes an automatic control mechanism, using the relative position information and the movement information, to adopt a correct operation corresponding to a cruise velocity of the vehicle and a drive route.

13. The method according to claim 9, wherein the connected object demarcation unit demarcates each of the illuminant objects and analyzes and compares the illuminant objects by utilizing the characteristics of projection analysis, size, ratio and distance of the illuminant objects to obtain a shortest distance of horizontal and vertical sub-area of connected-components of the illuminant objects and an overlapping ratio of a vertical projection of the sub-area of the connected-components.

14. The method according to claim 9, wherein further comprises a switch controller of high beam and low beam of the vehicle'"'"'s head light.

15. The method according to claim 9, wherein the distance estimation unit estimates a perspective depth of field of target vehicle on the imaginary and real coordinate axes system at a particular timeline on the basis of a perspective image module, and derives a relative spatial position between the target vehicle and the host vehicle on the lane using the corresponding relation between a coordinate position of pixel and the perspective depth of field of the target vehicle in the image.

16. The method according to claim 9, wherein the relative space position includes distance of perspective depth of field, position of left-hand edge and position of right-hand edge.