Sensor fusion of camera and V2V data for vehicles

• US 9,922,565 B2
• Filed: 06/30/2016
• Issued: 03/20/2018
• Est. Priority Date: 07/20/2015
• Status: Active Grant

• Current Assignee: New Eagle LLC (Woodward Incorporated)
• Original Assignee: Dura Operating LLC (Dura Automotive Systems)
• Inventors: Thomas, Gordon M., Trigui, Nizar, Liubakka, Michael
• Primary Examiner(s): Ramesh, Krishnan
• Application Number:
  US15/197,918
• Publication Number:
  US 20170025017A1
• Time in Patent Office: 628 Days
• Field of Search: 701 41
• CPC Class Codes: B60W 10/18 including control of braking systems
• CPC Class Codes: B60W 10/20 including control of steering systems
• CPC Class Codes: B60W 2400/00 Indexing codes relating to detected, measured or calculated conditions or factors
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2510/18 Braking system
• CPC Class Codes: B60W 2510/20 Steering systems
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 2554/20 Static objects
• CPC Class Codes: B60W 2554/4029 Pedestrians
• CPC Class Codes: B60W 2554/80 Spatial relation or speed relative to objects
• CPC Class Codes: B60W 2556/65 Data transmitted between vehicles
• CPC Class Codes: B60W 30/08 Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/06 Road conditions
• CPC Class Codes: G06F 18/22 Matching criteria, e.g. proximity measures
• CPC Class Codes: G06F 18/254 of classification results, e.g. of results related to same input data
• CPC Class Codes: G06T 11/60 Editing figures and text; Combining figures or text
• CPC Class Codes: G06T 2207/30256 Lane; Road marking
• CPC Class Codes: G06V 10/809 of classification results, e.g. where the classifiers operate on the same input data
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/161 : Decentralised systems, e.g. inter-vehicle communication
• CPC Class Codes: G08G 1/163 : involving continuous checking
• CPC Class Codes: G08G 1/167 : Driving aids for lane monitoring, lane changing, e.g. blind spot detection
• CPC Class Codes: H04L 67/12 : specially adapted for proprietary or special-purpose networking environments, e.g. medical networks, sensor networks, networks in vehicles or remote metering networks

Abstract

A method for fusing sensor information detected by a host vehicle and at least one remote vehicle-to-vehicle (V2V) communication equipped vehicle includes collecting visual data from an optical sensor of a vision sub-system, and collecting V2V data from remote vehicles. The method further includes executing a control logic including a first control logic for generating a base lane model and a base confidence level, a second control logic that fuses together the V2V data, the base lane model, and the base confidence level, and a third control logic that generates from the fused lane model, the V2V data, the base lane model, and the base confidence level, a final lane model and final confidence level, and assigns a priority to the final lane model.

Claims

1. A method for fusing sensor information detected by a host vehicle and at least one remote vehicle-to-vehicle (V2V) communication equipped vehicle, the method comprising:

collecting visual data from an optical sensor of a vision sub-system, and generating a base lane model and a base confidence level from the visual data;

collecting V2V data from a receiver of a V2V sub-system;

fusing together the V2V data, the base lane model, and the base confidence level;

generating a final lane model with a final confidence level from the fused together V2V data, the base lane model and the base confidence level;

assigning a priority to the final lane model; and

determining a location of an object in the final lane model relative to the host vehicle and assigning a high priority to the object when the object is in a lane also occupied by the host vehicle, and sending a command to at least one advanced driver assistance system (ADAS), and wherein the at least one ADAS performs a function to avoid the object to which a high priority has been assigned.

2. The method of claim 1 wherein the final confidence level indicates that the final lane model is more accurate than the base lane model.

3. The method of claim 2 wherein the base lane model and the final lane model comprise lane positioning, lane markings, lane curvature, speed, and trajectory data for the host vehicle and for any objects within a predefined area around the host vehicle.

4. The method of claim 3 wherein the V2V data comprises lane positioning, speed, and trajectory data for any remote V2V equipped vehicles in communication with the host vehicle and within a predefined area around the host vehicle, and for any objects sensed by any remote V2V vehicles in communication with the host vehicle and within a predefined area around the host vehicle.

5. The method of claim 1, wherein the fusing together of the V2V data, the base lane model, and the base confidence level further comprises comparing the visual data to the V2V data and determining a relative accuracy and precision of the visual data and the V2V data.

6. A system for fusing sensor information detected by a host vehicle and at least one remote vehicle-to-vehicle (V2V) communication equipped vehicle, the system comprising:

a vision sub-system having an optical sensor;

a V2V sub-system having a receiver;

a controller in communication with the vision sub-system and the V2V sub-system, the controller having memory for storing control logic and a processor configured to execute the control logic, the control logic including a first control logic for collecting visual data from the vision sub-system, and from the visual data generating a base lane model and a base confidence level;

the processor including a second control logic for collecting V2V data from the V2V sub-system, and for fusing together the V2V data and the base lane model and base confidence level;

the processor including a third control logic for generating, from the fused V2V data, base lane model and base confidence level, a final lane model with a final confidence level;

the processor including a fourth logic for assigning a priority to the final lane model; and

determining a location of an object in the final lane model relative to the host vehicle and assigning a high priority to the object when the object is in a lane also occupied by the host vehicle, wherein information about the object that has been assigned a high priority is passed to the at least one ADAS, and the at least one ADAS performs a function to avoid the object.

7. The system of claim 6 wherein the final confidence level indicates that the final lane model is more accurate than the base lane model.

8. The system of claim 7 wherein the base and the final lane models comprise lane positioning, lane markings, lane curvature, speed, and trajectory data for the host vehicle and for any objects within a predefined area around the host vehicle.

9. The system of claim 8 wherein the V2V data comprises lane positioning, speed, and trajectory data for any remote V2V equipped vehicles in communication with the host vehicle and within a predefined area around the host vehicle, and for any objects sensed by any remote V2V vehicles in communication with the host vehicle and within a predefined area around the host vehicle.

10. The system of claim 6, wherein the controller fuses together further comprises comparing the visual data to the V2V data and determining an accuracy and a precision of the visual data and the V2V data.