Method and apparatus for detecting a pedestrian by a vehicle during night driving

• US 10,391,937 B2
• Filed: 12/06/2014
• Issued: 08/27/2019
• Est. Priority Date: 08/25/2014
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Oh, Young Chul, Heo, Myung Seon, Lee, Wan Jae, You, Byung Yong
• Primary Examiner(s): Rao, Anand S
• Assistant Examiner(s): Edwards, Tyler B
• Application Number:
  US14/562,686
• Publication Number:
  US 20160052452A1
• Time in Patent Office: 1,725 Days
• Field of Search: None
• CPC Class Codes: B60K 2360/179 Distances to obstacles or vehicles
• CPC Class Codes: B60K 2360/21 using cameras
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/22 Display screens
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60R 1/23 with a predetermined field of view
• CPC Class Codes: B60R 1/30 providing vision in the non-visible spectrum, e.g. night or infrared vision
• CPC Class Codes: B60R 11/04 Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
• CPC Class Codes: B60R 2300/102 using 360 degree surveillance camera system
• CPC Class Codes: B60R 2300/105 using multiple cameras
• CPC Class Codes: B60R 2300/106 using night vision cameras
• CPC Class Codes: B60R 2300/205 using a head-up display
• CPC Class Codes: B60R 2300/303 using joined images, e.g. multiple camera images
• CPC Class Codes: B60R 2300/307 virtually distinguishing relevant parts of a scene from the background of the scene
• CPC Class Codes: B60R 2300/8033 for pedestrian protection
• CPC Class Codes: B60R 2300/8053 for bad weather conditions or night vision
• CPC Class Codes: B60Y 2400/92 Driver displays
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: H04N 23/11 : for generating image signals from visible and infrared light wavelengths
• CPC Class Codes: H04N 7/181 : for receiving images from a plurality of remote sources
• CPC Class Codes: H04N 7/188 : Capturing isolated or intermittent images triggered by the occurrence of a predetermined event, e.g. an object reaching a predetermined position signal generation from motion picture films H04N5/253

Abstract

A method and an apparatus for detecting a pedestrian by a vehicle during night driving are provided, in which the apparatus includes: a first camera configured to take a first image including color information of a vicinity of the vehicle during night driving; a second camera configured to take a second image including thermal distribution information of the vicinity of the vehicle; a pedestrian detector configured to detect a non-pedestrian area by using the color information from the first image and detect a pedestrian area by excluding the non-pedestrian area from the second image; and a display configured to match and display the pedestrian area on the second image.

Claims

1. An apparatus for detecting a pedestrian by a vehicle during night driving, comprising:

a first camera being a charge coupled device (CCD) camera configured to take a first image that is a color image including color information of a vicinity of the vehicle during night driving;

a second camera being an infrared camera configured to take a second image that is a far-infrared image including thermal distribution information of the vicinity of the vehicle;

an image matcher configured to match the first image and the second image;

a pedestrian detector configured to detect a non-pedestrian area by using an area in which a color value is more than a reference value from the first image including the color information of the vicinity of the vehicle during night driving, and detect a pedestrian area in the second image excluding the detected non-pedestrian area and including the thermal distribution information of the vicinity of the vehicle by using a feature detection and learning algorithm; and

a display configured to display the first image including the pedestrian area of the second image,

wherein the image matcher calculates a real coordinate of an object from a coordinate of the first image by using inside and outside parameters of the first camera and a real distance between a virtual starting point and the object,

wherein the image matcher calculates a corresponding coordinate of the second image corresponding to the coordinate of the first image by using the real coordinate of the object, inside and outside parameters of the second camera, and the real distance, and

wherein the virtual starting point is a central point between points representing locations of the first and second cameras, in which a vertical line from the points of the first camera and the second camera and planes of the first image and the second image meet.

2. The apparatus of claim 1, wherein the pedestrian detector comprises the image matcher configured to match the first image and the second image, a non-pedestrian area detector configured to detect the area in which the color value is more than the reference value as the non-pedestrian area from the first image based on the color information of the first image, an attention area extractor configured to extract an attention area by excluding the non-pedestrian area from the second image, and a pedestrian area extractor configured to extract the pedestrian area from the attention area.

3. A method for detecting a pedestrian by a vehicle during night driving, comprising the steps of:

taking a first image that is a color image and a second image that is a far-infrared image of a vicinity of the vehicle through a first camera and a second camera, respectively, during night driving, the first camera being a charge coupled device (CCD) camera and the second camera being a far-infrared camera;

matching the first image and the second image by an image matcher;

detecting a non-pedestrian area by using an area in which a color value is more than a reference value of the first image including color information of the vicinity of the vehicle during night driving;

excluding the non-pedestrian area from the second image including thermal distribution information of the vicinity of the vehicle;

detecting a pedestrian area in the second image excluding the detected non-pedestrian area and including the thermal distribution information of the vicinity of the vehicle by using a feature detection and learning algorithm;

displaying the first image including the pedestrian area of the second image,

wherein the step of matching the first image and the second image further comprises steps of;

calculating a real coordinate of an object from a coordinate of the first image by using inside and outside parameters of the first camera and a real distance between a virtual starting point and the object; and

calculating a corresponding coordinate of the second image corresponding to the coordinate of the first image by using a real coordinate of the object, inside and outside parameters of the second camera, and the real distance,

wherein the virtual starting point is a central point between points representing locations of the first and second cameras, in which a vertical line from starting points of the first camera and the second camera and planes of the first image and the second image meet.