Vehicle vision system with thermal sensor

• US 10,023,118 B2
• Filed: 03/22/2016
• Issued: 07/17/2018
• Est. Priority Date: 03/23/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Salomonsson, Ove J.
• Primary Examiner(s): Brushaber, Frederick M
• Application Number:
  US15/076,915
• Publication Number:
  US 20160280133A1
• Time in Patent Office: 847 Days
• CPC Class Codes: B60Q 1/085 due to special conditions, e.g. adverse weather, type of road, badly illuminated road signs or potential dangers B60Q1/10, B60Q1/12, B60Q1/1423 take precedence
• CPC Class Codes: B60Q 1/245 Searchlights, e.g. adjustable from within the vehicle adjustable headlights B60Q1/06
• CPC Class Codes: B60Q 2300/45 Special conditions, e.g. pedestrians, road signs or potential dangers
• CPC Class Codes: B60Q 9/008 for anti-collision purposes
• CPC Class Codes: B60T 2201/022 Collision avoidance systems
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B62D 15/0265 Automatic obstacle avoidance by steering
• CPC Class Codes: G06V 10/147 Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: H04N 23/45 for generating image signals from two or more image sensors being of different type or operating in different modes, e.g. with a CMOS sensor for moving images in combination with a charge-coupled device [CCD] for still images
• CPC Class Codes: H04N 23/56 provided with illuminating means
• CPC Class Codes: H04N 5/33 Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A vision system for a vehicle includes a high resolution camera disposed at the vehicle windshield and having a field of view forward of the vehicle, and a low resolution sensor disposed at a forward portion of the vehicle and having a field of sensing forward of the vehicle. Responsive to processing of an output of the low resolution sensor, a control determines the presence of a hot spot forward of the vehicle. Responsive to determination of a hot spot, the control enhances processing of image data captured by the camera to determine if the hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle. Responsive to determination that the hot spot is indicative of presence of a pedestrian or animal, the control generates an alert to a driver of the vehicle and/or controls a vehicle system to mitigate impact with the pedestrian or animal.

Claims

1. A vision system for a vehicle, said vision system comprising:

a high resolution camera disposed at a windshield of a vehicle and having a field of view through the windshield and forward of the vehicle;

wherein said camera comprises an imaging array having a plurality of photosensing elements, said plurality of photosensing elements comprising at least 1,000,000 photosensing elements;

a low resolution sensor disposed at a forward portion of the vehicle and having a field of sensing forward of the vehicle, said low resolution sensor comprising a plurality of sensing elements, said plurality of sensing elements comprising less than 240,000 sensing elements;

a control;

wherein, responsive to processing of an output of said low resolution sensor, said control determines the presence of a hot spot forward of the vehicle, wherein the hot spot comprises a region within the field of sensing of said low resolution sensor where there exists a concentration of electromagnetic radiation above a threshold level;

wherein, responsive to determination of a hot spot, said control enhances image processing of image data captured by said camera to determine if the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle; and

wherein, responsive to determination, via enhanced image processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control at least one of (i) generates an alert to a driver of the vehicle and (ii) controls a system of the vehicle to mitigate impact with the pedestrian or animal.

2. The vision system of claim 1, wherein said low resolution sensor comprises a low resolution thermal sensor.

3. The vision system of claim 2, wherein said low resolution thermal sensor comprises a sensing array having a plurality of thermal sensing elements, said plurality of thermal sensing elements comprising less than 480 thermal sensing elements.

4. The vision system of claim 3, wherein said plurality of thermal sensing elements comprises less than 240 thermal sensing elements.

5. The vision system of claim 3, wherein said low resolution thermal sensor comprises a bolometer sensor.

6. The vision system of claim 2, wherein said control determines the presence of a hot spot forward of the vehicle by determining a region in the field of sensing of said low resolution thermal sensor that has a temperature above a threshold temperature.

7. The vision system of claim 1, wherein said control enhances image processing of captured image data by directing illumination towards the determined hot spot.

8. The vision system of claim 7, wherein said control directs illumination towards the determined hot spot by adjusting a headlight of the vehicle.

9. The vision system of claim 1, wherein, responsive to determination, via processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control generates an alert to the driver of the vehicle.

10. The vision system of claim 1, wherein, responsive to determination, via processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control controls a system of the vehicle to mitigate impact with the pedestrian or animal.

11. The vision system of claim 10, wherein, responsive to determination, via processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control controls at least one of a brake system of the vehicle and a steering system of the vehicle.

12. A vision system for a vehicle, said vision system comprising:

a high resolution camera disposed at a windshield of a vehicle and having a field of view through the windshield and forward of the vehicle;

wherein said camera comprises an imaging array having a plurality of photosensing elements, said plurality of photosensing elements comprising at least 1,000,000 photosensing elements;

a low resolution thermal sensor disposed at a forward portion of the vehicle and having a field of sensing forward of the vehicle, said low resolution thermal sensor comprising a sensing array having a plurality of sensing elements, said plurality of sensing elements comprising less than 100,000 sensing elements;

a control;

wherein, responsive to processing of an output of said low resolution thermal sensor, said control determines the presence of a hot spot forward of the vehicle, wherein the hot spot comprises a region within the field of sensing of said low resolution thermal sensor where there exists a concentration of heat above a threshold level;

wherein, responsive to determination of a hot spot, said control enhances image processing of image data captured by said camera to determine if the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle; and

wherein, responsive to determination, via enhanced image processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control at least one of (i) generates an alert to a driver of the vehicle and (ii) controls a system of the vehicle to mitigate impact with the determined pedestrian or animal.

13. The vision system of claim 12, wherein said plurality of sensing elements of said low resolution thermal sensor comprises less than 240 sensing elements.

14. The vision system of claim 12, wherein said low resolution thermal sensor comprises a bolometer sensor.

15. The vision system of claim 12, wherein said control enhances image processing of captured image data by directing illumination towards the determined hot spot.

16. The vision system of claim 15, wherein said control directs illumination towards the determined hot spot by adjusting a headlight of the vehicle.

17. The vision system of claim 12, wherein, responsive to determination, via processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control generates an alert to the driver of the vehicle.

18. The vision system of claim 12, wherein, responsive to determination, via processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control controls at least one of a brake system of the vehicle and a steering system of the vehicle to mitigate impact with the pedestrian or animal.

19. A vision system for a vehicle, said vision system comprising:

a high resolution camera disposed at a windshield of a vehicle and having a field of view through the windshield and forward of the vehicle;

wherein said camera comprises an imaging array having a plurality of photosensing elements, said plurality of photosensing elements comprising at least 1,000,000 photosensing elements;

a low resolution thermal sensor disposed at a forward portion of the vehicle and having a field of sensing forward of the vehicle, said low resolution thermal sensor comprising a sensing array having a plurality of sensing elements, said plurality of sensing elements comprising less than 240 sensing elements;

wherein said low resolution thermal sensor comprises a bolometer sensor;

a control;

wherein, responsive to processing of an output of said low resolution thermal sensor, said control determines the presence of a hot spot forward of the vehicle, wherein the hot spot comprises a region within the field of sensing of said low resolution thermal sensor where there exists a concentration of heat above a threshold level;

wherein, responsive to determination of a hot spot, said control enhances image processing of image data captured by said camera to determine if the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle;

wherein said control enhances image processing of captured image data by directing illumination towards the determined hot spot; and

wherein, responsive to determination, via enhanced image processing of image data captured by said camera, that the determined hot spot is indicative of presence of a pedestrian or animal ahead of the vehicle, said control at least one of (i) generates an alert to a driver of the vehicle and (ii) controls a system of the vehicle to mitigate impact with the pedestrian or animal.

20. The vision system of claim 19, wherein said control directs illumination towards the determined hot spot by adjusting a headlight of the vehicle.