VEHICLE VISION SYSTEM WITH SITUATIONAL FUSION OF SENSOR DATA

• US 20160162743A1
• Filed: 12/03/2015
• Published: 06/09/2016
• Est. Priority Date: 12/05/2014
• Status: Abandoned Application

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Chundrlik, Jr., William J., Raudszus, Dominik
• Application Number:
  US14/957,708
• Publication Number:
  US 20160162743A1
• Time in Patent Office: Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 1/24 in front of the vehicle
• CPC Class Codes: B60R 2300/301 combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
• CPC Class Codes: G01S 13/867 Combination of radar systems with cameras
• CPC Class Codes: G01S 13/931 of land vehicles
• CPC Class Codes: G01S 17/86 Combinations of lidar systems with systems other than lidar, radar or sonar, e.g. with direction finders
• CPC Class Codes: G01S 2013/9316 combined with communication equipment with other vehicles or with base stations
• CPC Class Codes: G01S 2013/9323 Alternative operation using light waves
• CPC Class Codes: G01S 2013/9324 Alternative operation using ultrasonic waves
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06T 2207/30261 Obstacle
• CPC Class Codes: G06T 7/277 involving stochastic approaches, e.g. using Kalman filters
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads

Abstract

A vision system of a vehicle includes a camera and a non-imaging sensor. With the camera and the non-imaging sensor disposed at the vehicle, the field of view of the camera at least partially overlaps the field of sensing of the non-imaging sensor at an overlapping region. A processor is operable to process image data captured by the camera and sensor data captured by the non-imaging sensor to determine a driving situation of the vehicle. Responsive to determination of the driving situation, Kalman Filter parameters associated with the determined driving situation are determined and, using the determined Kalman Filter parameters, a Kalman Filter fusion may be determined. The determined Kalman Filter fusion may be applied to captured image data and captured sensor data to determine an object present in the overlapping region.

Claims

1. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle equipped with said vision system so as to have a field of view exterior of the equipped vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

a non-imaging sensor configured to be disposed at the equipped vehicle so as to have a field of sensing exterior of the equipped vehicle;

wherein, with said camera and said non-imaging sensor disposed at the equipped vehicle, the field of view of said camera at least partially overlaps the field of sensing of said non-imaging sensor at an overlapping region;

a processor operable to process image data captured by said camera and sensor data captured by said non-imaging sensor;

wherein, with said camera and said non-imaging sensor disposed at the equipped vehicle, said processor is operable to process captured image data and captured sensor data to determine a driving situation of the equipped vehicle; and

wherein, responsive to determination by said processor of the driving situation by processing of captured image data and captured sensor data, Kalman Filter parameters associated with the determined driving situation are determined, and, using the determined Kalman Filter parameters, a Kalman Filter fusion is determined, and wherein the determined Kalman Filter fusion is applied to captured image data and captured sensor data to determine an object present in the overlapping region.

2. The vision system of claim 1, wherein said processor is operable to process captured image data and captured sensor data to match objects determined, via processing of captured image data, to be present in the overlapping region and objects determined, via processing of captured sensor data, to be present in the overlapping region, and wherein, responsive to matching of determined objects, said processor determines if the matched objects are stationary or moving and Kalman Filter parameters associated with the determined matched objects are determined.

3. The vision system of claim 2, wherein the Kalman Filter parameters comprise a gain and covariance.

4. The vision system of claim 3, wherein, responsive to matching of objects, said processor determines if a moving object is indicative of an approaching head-on vehicle and, responsive to determination that the moving object is indicative of an approaching head-on vehicle, a gain and covariance associated with an approaching head-on vehicle are determined, and wherein, using the determined gain and covariance, the Kalman Filter fusion is determined.

5. The vision system of claim 3, wherein, responsive to matching of objects, said processor determines if a moving object is not indicative of an approaching head-on vehicle and, responsive to determination that the moving object is not indicative of an approaching head-on vehicle, a gain and covariance associated with other object motion are determined, and wherein, using the determined gain and covariance, the Kalman Filter fusion is determined.

6. The vision system of claim 1, wherein said processor is operable to process captured image data and captured sensor data to match objects determined, via processing of captured image data, to be present in the overlapping region and objects determined, via processing of captured sensor data, to be present in the overlapping region, and wherein, responsive to matching of determined objects, said processor determines a classification of the matched objects and Kalman Filter parameters associated with the determined matched objects are determined.

7. The vision system of claim 6, wherein the determined classification comprises one of (i) a vehicle cutting in front of the equipped vehicle and (ii) a vehicle stopped in front of the equipped vehicle.

8. The vision system of claim 1, wherein the Kalman Filter parameters comprise a gain and covariance.

9. The vision system of claim 1, wherein said non-imaging sensor comprises a radar sensor.

10. The vision system of claim 1, wherein said non-imaging sensor comprises one of a lidar sensor and an ultrasonic sensor.

11. The vision system of claim 1, wherein said processor is operable to communicate via a vehicle-to-vehicle communication system of the equipped vehicle.

12. The vision system of claim 1, wherein said camera has a field of view forward of the equipped vehicle and wherein said non-imaging sensor has a field of sensing forward of the equipped vehicle.

13. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle equipped with said vision system so as to have a field of view exterior and forward of the equipped vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

a non-imaging sensor configured to be disposed at the equipped vehicle so as to have a field of sensing exterior and forward of the equipped vehicle, wherein said non-imaging sensor comprises one of a radar sensor and a lidar sensor;

wherein, with said camera and said non-imaging sensor disposed at the equipped vehicle, the field of view of said camera at least partially overlaps the field of sensing of said non-imaging sensor at an overlapping region;

a processor operable to process image data captured by said camera and sensor data captured by said non-imaging sensor;

wherein, with said camera and said non-imaging sensor disposed at the equipped vehicle, said processor is operable to process captured image data and captured sensor data to determine a driving situation of the equipped vehicle;

wherein the determined driving situation comprises a vehicle cutting in front of the equipped vehicle; and

wherein, responsive to determination by said processor of the driving situation by processing of captured image data and captured sensor data, Kalman Filter parameters associated with the determined driving situation are determined.

14. The vision system of claim 13, wherein said processor is operable to process captured image data and captured sensor data to match objects determined, via processing of captured image data, to be present in the overlapping region and objects determined, via processing of captured sensor data, to be present in the overlapping region, and wherein, responsive to matching of determined objects, said processor determines if the matched objects are stationary or moving and Kalman Filter parameters associated with the determined matched objects are determined.

15. The vision system of claim 13, wherein the Kalman Filter parameters comprise a gain and covariance.

16. The vision system of claim 13, wherein said processor is operable to process captured image data and captured sensor data to match objects determined, via processing of captured image data, to be present in the overlapping region and objects determined, via processing of captured sensor data, to be present in the overlapping region, and wherein, responsive to matching of determined objects, said processor determines a classification of the matched objects and Kalman Filter parameters associated with the determined matched objects are determined.

17. The vision system of claim 13, wherein, using the determined Kalman Filter parameters, a Kalman Filter fusion is determined, and wherein the determined Kalman Filter fusion is applied to captured image data and captured sensor data to determine an object present in the overlapping region.

18. A vision system of a vehicle, said vision system comprising:

a camera configured to be disposed at a vehicle equipped with said vision system so as to have a field of view exterior and forward of the equipped vehicle;

wherein said camera comprises a pixelated imaging array having a plurality of photosensing elements;

a non-imaging sensor configured to be disposed at the equipped vehicle so as to have a field of sensing exterior and forward of the equipped vehicle, wherein said non-imaging sensor comprises one of a radar sensor and a lidar sensor;

wherein, with said camera and said non-imaging sensor disposed at the equipped vehicle, the field of view of said camera at least partially overlaps the field of sensing of said non-imaging sensor at an overlapping region;

a processor operable to process image data captured by said camera and sensor data captured by said non-imaging sensor;

wherein, with said camera and said non-imaging sensor disposed at the equipped vehicle, said processor is operable to process captured image data and captured sensor data to determine a driving situation of the equipped vehicle;

wherein, responsive to determination by said processor of the driving situation by processing of captured image data and captured sensor data, Kalman Filter parameters associated with the determined driving situation are determined; and

wherein said processor is operable to process captured image data and captured sensor data to match objects determined, via processing of captured image data, to be present in the overlapping region and objects determined, via processing of captured sensor data, to be present in the overlapping region.

19. The vision system of claim 18, wherein, responsive to matching of determined objects, said processor determines if the matched objects are stationary or moving and Kalman Filter parameters associated with the determined matched objects are determined.

20. The vision system of claim 18, wherein, using the determined Kalman Filter parameters, a Kalman Filter fusion is determined, and wherein the determined Kalman Filter fusion is applied to captured image data and captured sensor data to determine an object present in the overlapping region.