Sensor integration based pedestrian detection and pedestrian collision prevention apparatus and method

• US 10,821,946 B2
• Filed: 12/06/2017
• Issued: 11/03/2020
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Motors Corporation (Kia Corp.)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Kim, Yoon Soo, Park, Seung Wook, Lee, Wan Jae, Lee, Min Byeong, Jung, In Yong, Lee, Kang Hoon
• Primary Examiner(s): Lonsberry, Hunter B
• Assistant Examiner(s): Martinez Borrero, Luis A
• Application Number:
  US15/833,886
• Publication Number:
  US 20180236986A1
• Time in Patent Office: 1,063 Days
• CPC Class Codes: B60Q 1/44 for indicating braking action or preparation for braking, e.g. by detection of the foot approaching the brake pedal
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60T 2201/024 Collision mitigation systems
• CPC Class Codes: B60T 2210/32 Vehicle surroundings
• CPC Class Codes: B60T 2250/04 Vehicle reference speed; Vehicle body speed
• CPC Class Codes: B60T 2270/30 ESP control system
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60T 8/171 Detecting parameters used in the regulation; Measuring values used in the regulation
• CPC Class Codes: B60T 8/172 Determining control parameters used in the regulation, e.g. by calculations involving measured or detected parameters B60T8/17551 takes precedence
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01S 13/345 using triangular modulation
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G06F 16/2365 : Ensuring data consistency and integrity
• CPC Class Codes: G06V 10/143 : Sensing or illuminating at different wavelengths
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An apparatus for operating a pedestrian detection and collision mitigation system (PDCMS) function of a vehicle includes: a front detection sensor detecting a presence of a pedestrian on a driving lane of the vehicle and a distance and a relative speed between the pedestrian and the vehicle; a vehicle sensor detecting a speed of the vehicle; an electronic control unit operating a PDCMS function based on information detected by the front detection sensor and the vehicle sensor; and a warning unit operated to inform a driver of a collision of the pedestrian with the vehicle by a control of the electronic control unit.

Claims

1. An apparatus for activating a pedestrian detection and collision mitigation system (PDCMS) function of a vehicle, the apparatus comprising:

a front detection sensor detecting a presence of a pedestrian on a driving lane of the vehicle and a distance and a relative speed between the pedestrian and the vehicle;

a vehicle sensor detecting a speed of the vehicle;

an electronic control unit activating a PDCMS function when a collision between the pedestrian and the vehicle is expected based on information detected by the front detection sensor and the vehicle sensor; and

a warning unit providing a driver with a warning of the collision,

wherein the front detection sensor includes;

a radar sensor detecting the pedestrian on the driving lane using a radio wave reflected from the pedestrian to generate radar detection information and determining a moving speed of the pedestrian based on the relative speed between the pedestrian and the speed of the vehicle;

a far-infrared camera sensor imaging heat radiated from the pedestrian to generate far-infrared recognition information; and

an information integration processor integrating the radar detection information and the far-infrared recognition information, and

wherein the PDCMS function includes an operation of the warning unit and an operation of a brake regardless of whether the driver operates the brake, and

wherein the electronic control unit is configured to determine that the PDCMS function is operated if the speed of the vehicle is within a first speed range and the moving speed of the pedestrian on the driving lane is within a second speed range.

2. The apparatus of claim 1, wherein the information integration processor includes:

a consistency determination processor calculating a correlation between the radar detection information and the far-infrared recognition information to form a binary matrix;

a similarity calculation processor generating a measurement value using the binary matrix formed by the consistency determination processor and calculating similarity in a state variable area between the measurement value and track tracking information of a previous time;

a state variable updating processor updating each tracking information by performing Kalman filtering using the measurement value and the state variable; and

a tracking information management processor performing merging, generation, and deletion of the track.

3. The apparatus of claim 2, wherein the similarity calculation processor represents longitudinal/lateral relative positions of the radar detection information and the far-infrared recognition information using a polar coordinate system to generate the measurement value.

4. The apparatus of claim 2, wherein the state variable updating processor applies an extended Kalman filter to perform the Kalman filtering.

5. The apparatus of claim 1, wherein the front detection sensor further includes a radar detection information post-processing processor removing a result other than the pedestrian among the radar detection information before the integration of the result with the far-infrared recognition information.

6. The apparatus of claim 1, wherein the radar detection information is generated 64 times or less.

7. The apparatus of claim 1, wherein the far-infrared recognition information is generated 8 times or less.

8. The apparatus of claim 1, wherein the electronic control unit performs the operation of the brake so that the speed of the vehicle is reduced to at least a predetermined speed or more from a time point when the operation of the brake is operated to a time point when the collision occurs.

9. The apparatus of claim 1, wherein the electronic control unit permits the driver to operate the brake for a maximum possible deceleration even after the operation of the brake starts.

10. The apparatus of claim 1, wherein the electronic control unit controls the warning unit to inform the driver that the PDCMS function is in an available state.

11. The apparatus of claim 1, wherein the warning unit includes a display visually providing the warning of the collision or a speaker audibly providing the warning of the collision.

12. The apparatus of claim 1, wherein the PDCMS function further includes an operation of a rear brake lamp.

13. The apparatus of claim 1, wherein the PDCMS function further includes an operation of an electrical stability control (ESC).

14. A method for activating a pedestrian detection and collision mitigation system (PDCMS) function of a vehicle, comprising:

detecting, by a front detection sensor, a pedestrian on a driving lane of the vehicle using a radio wave reflected from the pedestrian to generate radar detection information;

imaging, by the front detection sensor, heat radiated from the pedestrian to generate far-infrared recognition information;

integrating, by the front detection sensor, the radar detection information with the far-infrared recognition information to detect pedestrian information including a presence of the pedestrian on the driving lane and a distance and a relative speed between the pedestrian and the vehicle;

detecting, by a vehicle sensor, vehicle information including a vehicle speed;

determining a moving speed of the pedestrian based on the relative speed between the pedestrian and the vehicle speed; and

activating, by an electronic control unit, a PDCMS function when a collision between the pedestrian and the vehicle is expected based on the pedestrian information and the vehicle information,

wherein the PDCMS function includes an operation of a warning unit operating to provide a driver with a warning of the collision, and an operation of a brake regardless of whether the driver operates the brake, and

wherein the method further includes determining that the PDCMS function is operated if the vehicle speed is within a first speed range and the moving speed of the pedestrian on the driving lane is within a second speed range.