VEHICLE VISION SYSTEM WITH OBJECT AND LANE FUSION

• US 20170032196A1
• Filed: 07/28/2016
• Published: 02/02/2017
• Est. Priority Date: 07/31/2015
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Gupta, Nikhil
• Granted Patent: US 10,268,904 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G06F 18/253 of extracted features
• CPC Class Codes: G06V 10/806 of extracted features
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A vision system of a vehicle includes at least one camera configured to be disposed at a vehicle so as to have a field of view exterior and forward of the vehicle, and at least one other sensor configured to be disposed at a vehicle so as to have a sensing field exterior of the vehicle. A processor is operable to process data captured by the camera and the sensor. Responsive to processing of captured data, the processor determines an object present in the sensing field of the sensor. Responsive to processing of image data captured by the camera, the processor determines lane markings present in the field of view of the camera and forward of the vehicle. The processor receives object hypotheses, sensor properties and vehicle data as inputs and performs a data transformation to generate fused object hypotheses.

Claims

1. A vision system of a vehicle, said vision system comprising:

at least one camera configured to be disposed at a vehicle equipped with said vision system so as to have a field of view exterior and forward of the vehicle;

at least one other sensor configured to be disposed at the equipped vehicle so as to have a sensing field exterior of the vehicle;

a processor operable to process image data captured by said camera and sensor data captured by said sensor;

wherein, responsive to processing by said processor of captured image data and captured sensor data, an object present in the sensing field of said sensor is determined;

wherein, responsive to processing of image data captured by said camera, said processor determines lane markings present in the field of view of said camera and forward of the equipped vehicle; and

wherein said processor receives object hypotheses, sensor properties and vehicle data as inputs and performs a data transformation to generate fused object hypotheses.

2. The vision system of claim 1, wherein the data transformation includes core fusion.

3. The vision system of claim 2, wherein the core fusion comprises synched frame-hypotheses fusion and temporal hypotheses propagation and temporal smoothing and prediction.

4. The vision system of claim 3, wherein, after synched frame-hypotheses fusion, said vision system performs said temporal hypotheses propagation, where a hypotheses map is generated corresponding to the current instance of time in memory.

5. The vision system of claim 1, wherein said vision system receives hypotheses definition and view properties and conducts view transformations to generate transformed localization.

6. The vision system of claim 5, wherein said vision system provides synched frame-hypotheses fusion and performs a cost matrix calculation.

7. The vision system of claim 6, wherein optimal assignment and property association/merging are performed to generate the fused object hypotheses.

8. The vision system of claim 6, wherein said cost matrix calculation comprises a matrix where the input hypotheses list from said sensors forms the two axes for the matrix.

9. The vision system of claim 1, wherein said at least one other sensor comprises at least one other camera.

10. The vision system of claim 1, wherein said processor receives vehicle inputs via CAN bus communications.

11. A vision system of a vehicle, said vision system comprising:

at least one camera configured to be disposed at a vehicle equipped with said vision system so as to have a field of view exterior and forward of the vehicle;

at least one other sensor configured to be disposed at the equipped vehicle so as to have a sensing field exterior of the vehicle;

a processor operable to process image data captured by said camera and sensor data captured by said sensor;

wherein, responsive to processing by said processor of captured image data and captured sensor data, an object present in the sensing field of said sensor is determined;

wherein, responsive to processing of image data captured by said camera, said processor determines lane markings present in the field of view of said camera and forward of the equipped vehicle;

wherein said processor receives object hypotheses, sensor properties and vehicle data as inputs and performs a data transformation to generate fused object hypotheses;

wherein said processor receives vehicle inputs via CAN bus communications; and

wherein the data transformation includes core fusion that comprises synched frame-hypotheses fusion and temporal hypotheses propagation and temporal smoothing and prediction.

12. The vision system of claim 11, wherein, after synched frame-hypotheses fusion, said vision system performs said temporal hypotheses propagation, where a hypotheses map is generated corresponding to the current instance of time in memory.

13. The vision system of claim 11, wherein said vision system receives hypotheses definition and view properties and conducts view transformations to generate transformed localization.

14. The vision system of claim 11, wherein said at least one other sensor comprises at least one other camera.

15. The vision system of claim 11, wherein said vision system provides synched frame-hypotheses fusion and performs a cost matrix calculation.

16. A vision system of a vehicle, said vision system comprising:

at least one camera configured to be disposed at a vehicle equipped with said vision system so as to have a field of view exterior and forward of the vehicle;

at least one other sensor configured to be disposed at the equipped vehicle so as to have a sensing field exterior of the vehicle;

a processor operable to process image data captured by said camera and sensor data captured by said sensor;

wherein, responsive to processing by said processor of captured image data and captured sensor data, an object present in the sensing field of said sensor is determined;

wherein, responsive to processing of image data captured by said camera, said processor determines lane markings present in the field of view of said camera and forward of the equipped vehicle;

wherein said processor receives object hypotheses, sensor properties and vehicle data as inputs and performs a data transformation to generate fused object hypotheses;

wherein said processor receives vehicle inputs via CAN bus communications; and

wherein said vision system receives hypotheses definition and view properties and conducts view transformations to generate transformed localization.

17. The vision system of claim 16, wherein said vision system provides synched frame-hypotheses fusion and performs a cost matrix calculation.

18. The vision system of claim 17, wherein optimal assignment and property association/merging are performed to generate the fused object hypotheses.

19. The vision system of claim 17, wherein said cost matrix calculation comprises a matrix where the input hypotheses list from said sensors forms the two axes for the matrix.

20. The vision system of claim 16, wherein said at least one other sensor comprises at least one other camera.