Passive infra-red guidance system

• US 10,755,576 B2
• Filed: 05/17/2019
• Issued: 08/25/2020
• Est. Priority Date: 05/11/2018
• Status: Active Grant

• Current Assignee: Arnold Chase
• Original Assignee: Arnold Chase
• Inventors: Chase, Arnold
• Primary Examiner(s): Moyer, Dale
• Application Number:
  US16/415,288
• Publication Number:
  US 20190346857A1
• Time in Patent Office: 466 Days
• CPC Class Codes: B60K 2360/178 Warnings
• CPC Class Codes: B60K 2360/334 Projection means
• CPC Class Codes: B60K 2360/48 Sensors
• CPC Class Codes: B60K 35/00 Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
• CPC Class Codes: B60K 35/213 Virtual instruments
• CPC Class Codes: B60K 35/23 Head-up displays [HUD] optical aspects of head-up displays G02B27/01
• CPC Class Codes: B60K 35/28 characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
• CPC Class Codes: B60W 10/04 including control of propulsion units
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/184 with wheel brakes
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2050/143 Alarm means
• CPC Class Codes: B60W 2050/146 Display means
• CPC Class Codes: B60W 2420/40 Photo, light or radio wave sensitive means, e.g. infrared sensors
• CPC Class Codes: B60W 2554/40 Dynamic objects, e.g. animals, windblown objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2554/804 Relative longitudinal speed
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 30/12 : Lane keeping
• CPC Class Codes: B60W 50/14 : Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G01C 21/3602 : Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0088 : characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0223 : involving speed control of the vehicle vehicle fittings for automatically controlling, i.e. preventing speed from exceeding an arbitrarily established velocity or maintaining speed at a particular velocity, as selected by the vehicle operator B60K31/00
• CPC Class Codes: G05D 1/0242 : using non-visible light signals, e.g. IR or UV signals
• CPC Class Codes: G05D 1/0246 : using a video camera in combination with image processing means
• CPC Class Codes: G06V 10/147 : Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/56 : exterior to a vehicle by using sensors mounted on the vehicle
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G06V 40/103 : Static body considered as a whole, e.g. static pedestrian or occupant recognition
• CPC Class Codes: G06V 40/23 : Recognition of whole body movements, e.g. for sport training
• CPC Class Codes: G08G 1/166 : for active traffic, e.g. moving vehicles, pedestrians, bikes
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A passive infra-red guidance system and method for augmenting operation of an autonomous vehicle on a roadway includes at least one forward-looking infra-red imaging sensor mounted on the vehicle in operative communication with an image processor tied into the vehicle'"'"'s operational system. The system determines the left and right edges of the roadway using thermal imaging, and then determines the centerline of the travel lane in which the vehicle is travelling based on the determined left and right edges of the roadway. The system then compares the determined centerline of the travel lane with the actual position of the vehicle and identifies any adjustment needed for the vehicle'"'"'s position based on the comparison. The left and right edge determination may comprise identifying a difference between a thermal signature representative of the roadway and a thermal signature representative of a non-roadway portion that is located proximate to the roadway portion.

Claims

1. A method for centrally locating a vehicle within a travel lane on a roadway, said method comprising:

determining the left edge of the roadway using thermal imaging;

determining the right edge of the roadway using thermal imaging;

determining a centerline of the travel lane based on the determined left and right edges of the roadway;

comparing the determined centerline of the travel lane with the actual position of the vehicle; and

identifying any adjustment for the vehicle'"'"'s position based on the comparison;

wherein the left and right edges of the roadway are located at material interfaces between different materials, wherein the determining of the left edge of the roadway and the determining of the right edge of the roadway are based on identifying a radiated temperature difference between a first thermal signature of a first material of the different materials and a second thermal signature of a second material of the different materials.

2. The method according to claim 1, wherein the left roadway edge determination and the right roadway edge determination comprise using a thermal imaging sensor assembly disposed on the vehicle.

3. The method according to claim 2, wherein the thermal imaging sensor assembly comprises at least one forward-looking infra-red thermal imaging sensor mounted to the vehicle.

4. The method according to claim 3, wherein the at least one infra-red thermal sensor comprises:

a left infra-red thermal sensor mounted on the left side of the vehicle for thermal imaging the roadway on the left side of the vehicle; and

a right infra-red thermal sensor mounted on the right side of the vehicle for thermal imaging the roadway on the right side of the vehicle.

5. The method according to claim 1, wherein the left roadway edge determination and the right roadway edge determination comprise identifying a thermal difference between a first thermal signature representative of a portion of the roadway being imaged and a second thermal signature representative of a non-roadway portion being imaged that is located proximate to the roadway portion.

6. The method according to claim 1, further comprising the step of adjusting the vehicle'"'"'s position based on the identified adjustment using a vehicle'"'"'s guidance system.

7. The method according to claim 6, wherein the adjustment of the vehicle'"'"'s position comprises at least one of (i) providing input to the vehicle'"'"'s steering system;

(ii) activating the vehicle'"'"'s braking system; and

(iii) activating the vehicle'"'"'s propulsion system.

8. The method according to claim 1, further comprising the step of displaying the identified adjustment on at least one of a heads-up display and a see-through OLED panel associated with the vehicle, either affixed to the interior of a windshield of the vehicle or sandwiched between layers of said windshield.

16. The method according to claim 1, wherein the step of determining the centerline of the travel lane based on the determined left and right edges of the roadway comprises calculating a width between curbs of the roadway.

17. The method according to claim 1, wherein the step of determining the centerline of the travel lane based on the determined left and right edges of the roadway is also based on information from a GPS or navigational system comprising at least one of:

a number of lanes of the roadway, a travel direction of each travel lane of a plurality of travel lanes of the roadway, whether the vehicle is proximate to an exit, off-ramp, or side street, and a width of a shoulder of the roadway.

9. A system for centrally locating a vehicle within a travel lane on a roadway, said system comprising:

a thermal imaging sensor assembly disposed on the vehicle and comprising at least one infra-red sensor for thermal imaging the roadway on the left and right sides of the vehicle, respectively;

an image processor in operative communication with the thermal imaging sensor assembly for determining a centerline of the travel lane in the roadway based on thermal imaging signals received from the thermal imaging sensor assembly associated with the left and right edges of the roadway; and

a central vehicle control system in operative communication with the image processor for adjusting the vehicle'"'"'s position within the travel lane using a vehicle'"'"'s guidance system based on a comparison of the vehicle'"'"'s actual position and the determined centerline of the travel lane

wherein the left and right edges of the roadway are located at material interfaces between different materials, wherein the image processor is configured to identify a radiated temperature difference between a first thermal signature of a first material of the different materials and a second thermal signature of a second material of the different materials in order to determine the centerline of the travel lane in the roadway based on thermal imaging with the left and right edges of the roadway.

10. The system according to claim 9, wherein the thermal imaging sensor assembly comprises at least one forward-looking infra-red thermal imaging sensor mounted to the vehicle.

11. The system according to claim 10, wherein the at least one infra-red thermal sensor comprises:

a left infra-red thermal sensor mounted on the left side of the vehicle for thermal imaging the roadway on the left side of the vehicle; and

a right infra-red thermal sensor mounted on the right side of the vehicle for thermal imaging the roadway on the right side of the vehicle.

12. The system according to claim 9, wherein image processor determines the left edge and the right edge of the roadway based on thermal imaging signals received from the thermal imaging sensor assembly.

13. The system according to claim 12, wherein the left roadway edge determination and the right roadway edge determination comprise identifying a thermal difference between a first thermal signature representative of a portion of the roadway being imaged and a second thermal signature representative of a non-roadway portion being imaged that is located proximate to the roadway portion.

14. The system according to claim 9, wherein the adjustment of the vehicle'"'"'s position comprises at least one of (i) providing input to the vehicle'"'"'s steering system;

(ii) activating the vehicle'"'"'s braking system; and

(iii) activating the vehicle'"'"'s propulsion system.

15. The system according to claim 9, further comprising a see-through OLED panel, either affixed to the interior of a windshield of the vehicle or sandwiched between layers of said windshield, for displaying any adjustment needed for the vehicle'"'"'s position based on the comparison of the vehicle'"'"'s position and the determined centerline of the travel lane.

18. A system for centrally locating a vehicle within a travel lane on a roadway, said system comprising:

a thermal imaging sensor assembly disposed on the vehicle and comprising at least one infra-red sensor for thermal imaging the roadway on the left and right sides of the vehicle, respectively;

an image processor in operative communication with the thermal imaging sensor assembly for determining a centerline of the travel lane in the roadway based on thermal imaging signals received from the thermal imaging sensor assembly associated with the left and right edges of the roadway; and

a central vehicle control system in operative communication with the image processor for adjusting the vehicle'"'"'s position within the travel lane using a vehicle'"'"'s guidance system based on a comparison of the vehicle'"'"'s actual position and the determined centerline of the travel lane;

wherein the image processor is configured to calculate a width between curbs of the roadway based on the determined left and right edges of the roadway in order to determine the centerline of the travel lane.