Passive infra-red pedestrian detection and avoidance system

• US 10,467,903 B1
• Filed: 03/15/2019
• Issued: 11/05/2019
• Est. Priority Date: 05/11/2018
• Status: Active Grant

• Current Assignee: Arnold Chase
• Original Assignee: Arnold Chase
• Inventors: Chase, Arnold
• Primary Examiner(s): Lee, Tyler J
• Application Number:
  US16/354,833
• Publication Number:
  US 20190347938A1
• Time in Patent Office: 235 Days
• CPC Class Codes: B60K 2360/178 Warnings
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/48 Sensors
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/213 Virtual instruments
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/184 with wheel brakes
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2554/40 Dynamic objects, e.g. animals, windblown objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/12 : Lane keeping
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01C 21/3602 : Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0223 : involving speed control of the vehicle vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00
• CPC Class Codes: G05D 1/0242 : using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/147 : Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A passive infra-red pedestrian detection and avoidance system and method for augmenting the operation of a vehicle on a roadway, especially for identifying potential pedestrian/vehicular collision danger for the vehicle in operation and adjusting the position and operation of the vehicle accordingly, includes at least one passive infra-red sensor array mounted on the vehicle in operative communication with an image processor tied into the operational system of the vehicle. The system detects, using thermal imaging and processing, the presence of people that may be in or laterally crossing into the travel lane of the vehicle. The image processor analyzes the detection of a human thermal signature and determines if the detected human thermal signature is moving, in what direction and at what speed, to assess any potential threat to the pedestrian or biker, and further whether any responsive action needs to be triggered in the vehicle'"'"'s operation to avoid a collision.

Claims

1. A method for identifying and preventing pedestrian dangers between a moving vehicle and a pedestrian, and for adjusting operation of the vehicle accordingly, said method comprising:

detecting a thermal signature generated by a human on the side or within the traveled portion of a roadway using passive thermal emission detection;

determining if the human thermal signature is within or moving in a direction toward a predicted pathway of the vehicle;

assessing the need for responsive action in the vehicle'"'"'s operation; and

triggering responsive action in the vehicle'"'"'s operation if a need for such action is identified.

2. The method according to claim 1, wherein the detection of the thermal signature generated by the human on the side or traveled portion of the roadway comprises using a thermal emission sensor assembly disposed on the vehicle.

3. The method according to claim 2, wherein the thermal emission sensor assembly comprises at least one infra-red sensor array mounted to the vehicle.

4. The method according to claim 3, wherein the at least one infra-red sensor array comprises a forward-looking thermal sensor mounted on the front of the vehicle.

5. The method according to claim 3, wherein the at least one infra-red sensor array comprises:

a left infra-red sensor array mounted on the left side of the vehicle for thermal emission detecting the roadway and/or adjacent areas on the left side of the vehicle; and

a right infra-red sensor mounted on the right side of the vehicle for thermal emission detecting the roadway and/or adjacent areas on the right side of the vehicle.

6. The method according to claim 1, wherein the detection of the human thermal signature comprises the use of detected thermal emission data that is first passed through a predefined thermal acceptance window correlated to a thermal emission range of a human.

7. The method according to claim 6, wherein said pre-defined thermal acceptance window can be dynamically altered to compensate for the effect of ambient temperature conditions on the thermal emission range of a human.

8. The method according to claim 1, wherein the movement determining step comprises:

first determining if the human thermal signature is exhibiting movement upon detection of the human thermal signature;

then determining if the movement is generally perpendicular to the roadway; and

then determining if the movement is toward or away from the roadway.

9. The method according to claim 1, wherein the assessment of the need for responsive action includes determining the speed of movement of the human thermal signature.

10. The method according to claim 1, wherein the assessment of the need for responsive action includes determining the speed of movement of the vehicle.

11. The method according to claim 1, wherein the assessment of a need for responsive action includes determining if the human thermal signature is within the predicted pathway of the vehicle.

12. The method according to claim 1, wherein the responsive action in the vehicle'"'"'s operation comprises at least one of:

(i) providing adjustment input to the vehicle'"'"'s steering system;

(ii) activating the vehicle'"'"'s braking system;

(iii) sounding an audible alert associated with the vehicle; and

(iv) activating a visual alert associated with the vehicle.

13. The method according to claim 1, wherein the assessment of the need for responsive action for a stationary human thermal signature that is within the predicted pathway of the vehicle comprising using an image processor that looks for multiple adjacent qualified pixels from a thermal sensor array that represent a human thermal signature.

14. The method according to claim 13, wherein the assessment of the need for responsive action for a stationary human thermal signature that is within the predicted pathway of the vehicle comprises using the image processor that looks for a change in the number of adjacent qualified pixels from a thermal sensor array that represent a human thermal signature.

15. The method according to claim 14, wherein a rate of change in the number of adjacent qualified pixels that represent the human thermal signature is correlated with the closing speed of the vehicle towards the stationary human thermal signature.

16. A system for identifying pedestrian dangers between a moving vehicle and a pedestrian, and for adjusting operation of the vehicle accordingly, said system comprising:

a thermal sensor array assembly for detecting a human thermal signature generated by a human on the side of the roadway or within a predicted pathway of the vehicle on the roadway using passive thermal emission detection;

an image processor in operative communication with the thermal sensor array assembly for determining if the human thermal signature is within or moving in a direction toward the predicted pathway of the vehicle and assessing the need for responsive action in the vehicle'"'"'s operation; and

a central vehicle control system in operative communication with the image processor for triggering responsive action in the vehicle'"'"'s operation if a need for such action is identified.

17. The system according to claim 16, wherein the thermal sensor assembly comprises at least one infra-red sensor array mounted to the vehicle.

18. The method according to claim 17, wherein the at least one infra-red sensor array comprises a forward-looking thermal sensor mounted on the front of the vehicle.

19. The system according to claim 17, wherein the at least one infra-red sensor comprises:

a left infra-red sensor array mounted on the left side of the vehicle for thermal emission detecting the roadway and/or adjacent areas on the left side of the vehicle; and

a right infra-red sensor array mounted on the right side of the vehicle for thermal emission detecting the roadway and/or adjacent areas on the right side of the vehicle.

20. The system according to claim 16, wherein the detection of the human thermal signature by the thermal sensor array assembly comprises the use of detected thermal emission data passed through a pre-defined thermal acceptance window correlated to a thermal emission range of a human.

21. The system according to claim 20, wherein said pre-defined thermal acceptance window can be dynamically altered to compensate for the effect of ambient temperature conditions on the thermal emission range of a human.

22. The system according to claim 16, wherein the image processor, upon receiving a signal from the thermal sensor array assembly identifying a human thermal signature, first determines if the human thermal signature is moving;

then determines if the movement of the human thermal signature is generally perpendicular to the roadway; and

then determines if the movement of the human thermal signature is toward or away from the roadway.

23. The system according to claim 16, wherein the image processor, in assessing the need for responsive action by the vehicle, determines the speed of movement of the human thermal signature in connection with the thermal sensor assembly.

24. The system according to claim 16, wherein the system, in assessing the need for responsive action by the vehicle, determines the speed of movement of the vehicle.

25. The system according to claim 16, wherein the image processor, in assessing the need for responsive action by the vehicle, determines if the human thermal signature is within the predicted pathway of the vehicle in connection with the thermal sensor array assembly.

26. The system according to claim 16, wherein the responsive action in the vehicle'"'"'s operation comprises at least one of:

(i) providing adjustment input to the vehicle'"'"'s steering system;

(ii) activating the vehicle'"'"'s braking system;

(iii) sounding an audible alert associated with the vehicle; and

(iv) activating a visual alert associated with the vehicle.

27. A method for identifying and preventing pedestrian dangers between a moving vehicle and a pedestrian, and for adjusting operation of the vehicle accordingly, said method comprising:

detecting a thermal signature generated by a human on a side or within a traveled portion of a roadway using passive thermal sensing;

determining if the thermal signature generated by the human is indicative of the human being within or moving in a direction toward a predicted pathway of the vehicle;

assessing the need for responsive action in the operation of the vehicle; and

triggering responsive action in the operation of the vehicle if a need for such action is identified;

wherein the detection of the thermal signature generated by the human comprises filtering the detected thermal emission data through a predefined thermal acceptance window correlated to a thermal emission range of a human.