VEHICLE ASSIST SYSTEM

• US 20200133295A1
• Filed: 10/31/2018
• Published: 04/30/2020
• Est. Priority Date: 10/31/2018
• Status: Active Grant

• Current Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Original Assignee: TRW Automotive USLLC (Zeppelin - Stiftung der Stadt Friedrichshafen)
• Inventors: Indrakanti, Ramakirankumar, Elkhatib, Hassan
• Granted Patent: US 10,698,415 B2
• Time in Patent Office: Days
• CPC Class Codes: B60T 7/12 for automatic initiation; for initiation not subject to will of driver or passenger limiting speed of vehicles other than rail vehicles B60K31/00
• CPC Class Codes: B60T 7/22 initiated by contact of vehicle, e.g. bumper, with an external object, e.g. another vehicle , or by means of contactless obstacle detectors mounted on the vehicle
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2555/20 Ambient conditions, e.g. wind or rain
• CPC Class Codes: B60W 2710/18 Braking system
• CPC Class Codes: B60W 2720/10 Longitudinal speed
• CPC Class Codes: B60W 2720/24 Direction of travel
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G06F 18/25 Fusion techniques
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/10048 Infrared image
• CPC Class Codes: G06T 2207/20221 Image fusion; Image merging
• CPC Class Codes: G06T 2207/30168 Image quality inspection
• CPC Class Codes: G06T 2207/30192 Weather; Meteorology
• CPC Class Codes: G06T 2207/30252 Vehicle exterior; Vicinity of vehicle
• CPC Class Codes: G06T 7/0002 Inspection of images, e.g. flaw detection
• CPC Class Codes: G06V 10/147 : Details of sensors, e.g. sensor lenses fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19
• CPC Class Codes: G06V 10/803 : of input or preprocessed data
• CPC Class Codes: G06V 20/58 : Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G06V 20/582 : of traffic signs
• CPC Class Codes: G06V 20/584 : of vehicle lights or traffic lights
• CPC Class Codes: G06V 20/588 : Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: H04N 23/63 : by using electronic viewfinders
• CPC Class Codes: H04N 23/90 : Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
• CPC Class Codes: H04N 5/265 : Mixing
• CPC Class Codes: H04N 5/33 : Transforming infrared radiation cameras or camera modules for generating image signals from infrared radiation H04N23/20; circuitry of SSIS for transforming infrared radiation into image signals H04N25/20

Abstract

A method for assisting operation of a vehicle traveling on a roadway includes acquiring visual images around the vehicle with at least one visual camera having a field of view and acquiring thermal images around the vehicle with at least one thermal camera having the field of view. The thermal images are superimposed over the visual images to produce composite images. An object is detected in the composite images. A vehicle assist system adjusts at least one of a direction of travel and speed of the vehicle in response to detecting the object.

Claims

1. A method for assisting operation of a vehicle traveling on a roadway, comprising:

acquiring a visual image around the vehicle with at least one visual camera having a field of view;

acquiring a thermal image around the vehicle with at least one thermal camera having the field of view;

superimposing the thermal image over the visual image to produce a composite image;

detecting an object in the composite image;

adjusting at least one of a direction of travel and speed of the vehicle in response to detecting the object;

sensing weather conditions and lighting conditions in the field of view;

assigning a first visibility score to the visual image based on the sensed weather conditions;

assigning a second visibility score to the visual image based on the sensed lighting conditions;

combining the first and second visibility scores into an aggregate visibility score; and

adjusting the transparency of thermal image based on the aggregate visibility score prior to superimposing the thermal image onto the visual image.

5. The method of claim 1, wherein the step of combining the first and second visibility scores comprises calculating a weighted average of the first and second visibility scores to determine the aggregate visibility score.

6. The method of claim 1 further comprising:

detecting lane lines of the roadway with the composite image; and

autonomously performing lane keep assistance in response to detecting the lane lines.

7. The method of claim 1 further comprising:

detecting an object on the roadway with the composite image; and

autonomously braking the vehicle in response to detecting the object.

8. The method of claim 1 further comprising:

detecting a traffic pattern indicator along the roadway with the composite image; and

autonomously braking the vehicle in response to detecting the traffic pattern indicator.

2-4. -4. (canceled)

9. A vehicle assist system for a host vehicle traveling on a roadway, comprising:

a visual camera for acquiring a visual image around the host vehicle in a field of view;

a thermal camera for acquiring a thermal image around the host vehicle in the field of view;

a controller connected to the visual camera and the thermal camera and configured to superimpose the thermal image onto the visual image to form a composite image for detecting an object within the field of view and adjusting at least one of a direction of travel and speed of the vehicle in response to detecting the object; and

a rain sensor connected to the controller for detecting precipitation within the field of view, the controller being configured to adjust a transparency of the thermal image in response to the amount of precipitation detected by the rain sensor within the field of view before superimposing the thermal image onto the visual image.

11. The vehicle assist system of claim 9 further comprising a light sensor connected to the controller for detecting light intensity in the field of view, the controller being configured to adjust a transparency of the thermal image in response to the light intensity detected by the light sensor within the field of view before superimposing the thermal image onto the visual image.

12. The vehicle assist system of claim 9, wherein the controller is configured to perform at least one of lane keep assistance and braking in response to detecting the object.

10. (canceled)