Sensor integration based pedestrian detection and pedestrian collision prevention apparatus and method

• US 10,814,840 B2
• Filed: 12/06/2017
• Issued: 10/27/2020
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Motors Corporation (Kia Corp.)
• Original Assignee: Hyundai Motor Company (Hyundai Motor Group)
• Inventors: Kim, Yoon Soo, Min, Suk Ki, Sung, Dong Hyun, Jung, In Yong, Park, Seung Wook, Lee, Sang Min, Lee, Kang Hoon, Lee, Wan Jae, Lee, Tae Young, Lee, Min Byeong, Kim, Eung Seo, Kwon, Yong Seok
• Primary Examiner(s): Lonsberry, Hunter B
• Assistant Examiner(s): Martinez Borrero, Luis A
• Application Number:
  US15/833,799
• Publication Number:
  US 20180236985A1
• Time in Patent Office: 1,056 Days
• CPC Class Codes: B60Q 1/44 for indicating braking action or preparation for braking, e.g. by detection of the foot approaching the brake pedal
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60T 2201/024 Collision mitigation systems
• CPC Class Codes: B60T 2210/32 Vehicle surroundings
• CPC Class Codes: B60T 2220/03 Driver counter-steering; Avoidance of conflicts with ESP control
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60T 8/171 Detecting parameters used in the regulation; Measuring values used in the regulation
• CPC Class Codes: B60T 8/172 Determining control parameters used in the regulation, e.g. by calculations involving measured or detected parameters B60T8/17551 takes precedence
• CPC Class Codes: B60T 8/58 responsive to speed and another condition or to plural speed conditions
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2510/182 Brake pressure, e.g. of fluid or between pad and disc
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2520/105 Longitudinal acceleration
• CPC Class Codes: B60W 2540/18 Steering angle
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2710/182 : Brake pressure, e.g. of fluid or between pad and disc
• CPC Class Codes: B60W 2720/10 : Longitudinal speed
• CPC Class Codes: B60W 30/09 : Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0953 : the prediction being responsive to vehicle dynamic parameters
• CPC Class Codes: B60W 30/0956 : the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: G06F 16/2365 : Ensuring data consistency and integrity
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

An apparatus includes a front detection sensor detecting presence of a pedestrian on a driving lane of the vehicle, gaze information of the pedestrian, and a distance and a relative speed between the pedestrian and the vehicle; a vehicle sensor detecting at least one of a speed, an acceleration, a steering angle, a steering angular velocity, or a pressure of a master cylinder of the vehicle; an electronic control unit activating a function of a pedestrian detection and collision mitigation system based on information detected by the front detection sensor and the vehicle sensor; and a warning unit operated to inform a driver of a collision of the pedestrian with the vehicle by controlling the electronic control unit.

Claims

1. An apparatus for activating a pedestrian detection and collision mitigation system (PDCMS) of a vehicle, comprising:

a front detection sensor detecting presence of a pedestrian on a driving lane of the vehicle, gaze information of the pedestrian, and a distance and a relative speed between the pedestrian and the vehicle;

a vehicle sensor detecting at least one of a speed, an acceleration, a steering angle, a steering angular velocity, or a pressure of a master cylinder of the vehicle;

an electronic control unit activating a function of the PDCMS when a collision of the pedestrian with the vehicle is expected based on information detected by the front detection sensor and the vehicle sensor; and

a warning unit providing a driver with a warning of the collision of the pedestrian with the vehicle,

wherein the front detection sensor includes;

a radar sensor detecting the pedestrian on the driving lane of the vehicle using a radio wave reflected from the pedestrian to generate radar detection information, and determining a moving speed of the pedestrian based on the relative speed between the pedestrian and the speed of the vehicle;

a far-infrared sensor imaging heat radiated from the pedestrian to generate far-infrared recognition information; and

a processor integrating the radar detection information and the far-infrared recognition information,

wherein the function of the PDCMS includes an operation of the warning unit and an operation of a brake of the vehicle regardless of whether the driver operates the brake,

wherein the operation of the warning unit and the operation of the brake are performed in order of the operation of the warning unit, a partial braking of the vehicle, and a full braking of the vehicle, and

wherein the electronic control unit is configured to determine that the function of the PDCMS is operated, if a speed of the vehicle is within a first speed range and the moving speed of the pedestrian on a driving lane is within a second speed range.

2. The apparatus of claim 1, wherein the processor is further configured to:

calculate a correlation between the radar detection information and the far-infrared recognition information to form a binary matrix;

generate a measurement value using the binary matrix and calculate similarity in a state variable area between the measurement value and track tracking information of a previous time;

update each tracking information by performing Kalman filtering using the measurement value and the state variable; and

perform merging, generation, and deletion of the track.

3. The apparatus of claim 2, wherein the processor is further configured to represent longitudinal/lateral relative positions of the radar detection information and the far-infrared recognition information using a polar coordinate system to generate the measurement value.

4. The apparatus of claim 2, wherein the processor is further configured to apply an extended Kalman filter to perform the Kalman filtering.

5. The apparatus of claim 1, wherein the processor is further configured to remove a result other than the pedestrian among the radar detection information before the integration of the result with the far-infrared recognition information.

6. The apparatus of claim 1, wherein the radar detection information is generated 64 times or less.

7. The apparatus of claim 1, wherein the far-infrared recognition information is generated 8 times or less.

8. The apparatus of claim 1, wherein when a horizontal length of an entire face of the pedestrian is defined as x, a horizontal length from a left face contour to a left eye of the pedestrian is defined as x₁, a horizontal length from a right face contour to a right eye of the pedestrian is defined as x₂, and a horizontal length between the left eye and the right eye of the pedestrian is defined as x₃, the gaze information of the pedestrian corresponds to a front or a diagonal if |(x₁−

x₂)/x|<

a, a rear if x₁=x₂=x₃=0, aside if |(x₁−

x₂)/x|≥

x₁=0 or x₂=0, and a is selected in a range between 0.65 and 0.95.

9. The apparatus of claim 8, wherein the electronic control unit performs the operation of the warning unit and the operation of the brake by delaying the operation of the warning unit and the operation of the brake by a predetermined time when the gaze information of the pedestrian is the front or the diagonal, compared to when the gaze information of the pedestrian is the rear or the side.

10. The apparatus of claim 1, wherein the vehicle sensor further includes at least one of a rain sensor, a temperature sensor, and an illumination sensor.

11. The apparatus of claim 1, wherein the electronic control unit performs the operation of the brake so that the speed of the vehicle is reduced to at least a predetermined speed or more from time when the operation of the brake is operated to a time point when the collision of the pedestrian with the vehicle occurs.

12. The apparatus of claim 1, wherein the electronic control unit permits the driver to operate the brake for a maximum possible deceleration even after the operation of the brake starts.

13. The apparatus of claim 1, wherein the electronic control unit controls the warning unit to inform the driver that the PDCMS function is in an available state.

14. The apparatus of claim 1, wherein the warning unit includes a display visually providing the driver with the warning of the collision of the pedestrian with the vehicle or a speaker audibly providing the driver with the warning of the collision of the pedestrian with the vehicle.

15. The apparatus of claim 1, wherein the PDCMS function further includes an operation of a rear brake lamp.

16. The apparatus of claim 1, wherein the PDCMS function further includes an operation of an electrical stability control (ESC).

17. A method for activating a pedestrian detection and collision mitigation system (PDCMS) of a vehicle, comprising:

detecting a pedestrian on a driving lane of the vehicle using a radio wave reflected from the pedestrian to generate radar detection information;

imaging heat radiated from the pedestrian to generate far-infrared recognition information;

integrating the radar detection information with the far-infrared recognition information to detect pedestrian information including presence of the pedestrian on the driving lane of the vehicle, gaze information of the pedestrian, and a distance and a relative speed between the pedestrian and the vehicle;

detecting vehicle information including at least any one of a speed, an acceleration, a steering angle, a steering angular velocity, or a pressure of a master cylinder of the vehicle;

determining a moving speed of the pedestrian based on the relative speed between the pedestrian and the speed of the vehicle; and

activating a PDCMS function when a collision of the pedestrian with the vehicle is expected based on the pedestrian information and the vehicle information,

wherein the PDCMS function includes an operation of a warning unit operated to provide a driver with a warning of the collision of the pedestrian with the vehicle and an operation of a brake of the vehicle regardless of whether the driver operates the brake,

wherein the operation of the warning unit and the operation of the brake are performed in order of the operation of the warning unit, a partial braking of the vehicle, and a full braking of the vehicle, and

wherein the method further includes determining that the function of PDCMS is operated, if a speed of the vehicle is within a first speed range and the moving speed of the pedestrian on a driving lane is within a second speed range.