Vehicle vision system with curvature estimation

• US 10,406,981 B2
• Filed: 03/20/2015
• Issued: 09/10/2019
• Est. Priority Date: 03/20/2014
• Status: Active Grant

• Current Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Original Assignee: Magna Electronics Incorporated (Magna International Incorporated)
• Inventors: Chundrlik, Jr., William J., Wimmershoff, Marc
• Primary Examiner(s): Bailey, Frederick D
• Assistant Examiner(s): Sechser, Jill D
• Application Number:
  US14/663,502
• Publication Number:
  US 20150266422A1
• Time in Patent Office: 1,635 Days
• CPC Class Codes: B60R 1/00 Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
• CPC Class Codes: B60R 2300/302 combining image information with GPS information or vehicle data, e.g. vehicle speed, gyro, steering angle data
• CPC Class Codes: B60R 2300/804 for lane monitoring
• CPC Class Codes: B60R 2300/8086 for vehicle path indication
• CPC Class Codes: G01S 19/39 the satellite radio beacon positioning system transmitting time-stamped messages, e.g. GPS [Global Positioning System], GLONASS [Global Orbiting Navigation Satellite System] or GALILEO
• CPC Class Codes: G01S 19/45 by combining measurements of signals from the satellite radio beacon positioning system with a supplementary measurement
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

A vision system of a vehicle includes a camera disposed at a vehicle and having a field of view forward of the vehicle and operable to capture image data. A vehicle-based GPS system is operable to determine a geographical location of the vehicle. A control includes an image processor operable to process image data captured by the camera. Responsive at least in part to processing of captured image data by the image processor to determine lane markings and responsive at least in part to the GPS system and responsive at least in part to map data mapping the road along which the vehicle is traveling, the control is operable to determine an enhanced estimation of the path of travel of the vehicle. The enhanced estimation may be based on reliability levels or weighting factors of the determined lane markings and the map data.

Claims

1. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle equipped with said vision system, wherein said camera has a field of view forward of the equipped vehicle and is operable to capture image data;

a vehicle-based GPS system operable to determine a geographical location of the equipped vehicle;

a control including an image processor;

wherein said image processor processes image data captured by said camera;

wherein, responsive at least in part to processing of captured image data, said control is operable to determine (i) left lane markings along a left side of a lane in which the equipped vehicle is traveling on a road and (ii) right lane markings along a right side of the lane in which the equipped vehicle is traveling on the road;

wherein, responsive at least in part to said GPS system determining the geographic location of the equipped vehicle, said vision system generates a map-based road curvature for the road ahead of the equipped vehicle based on map data;

wherein, responsive at least in part to processing of captured image data by said image processor, said control determines a particular lane in which the equipped vehicle is traveling from a plurality of lanes of the road and generates a camera-based road curvature for the road ahead of the equipped vehicle;

wherein, responsive to generation of the map-based road curvature and the camera-based road curvature, said control fuses the map-based road curvature and the camera-based road curvature;

wherein, responsive to fusing of the map-based road curvature and the camera-based road curvature, said control is operable to estimate an actual road curvature of the road ahead of the equipped vehicle; and

wherein, in fusing of map-based road curvature and camera-based road curvature, said control diminishes weight of the camera-based road curvature when reliability level is below a threshold level for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the determined right lane markings; and

wherein, in fusing of map-based road curvature and camera-based road curvature, said control diminishes weight of the map-based road curvature when reliability level is below a threshold level for the map data.

2. The vision system of claim 1, wherein, responsive to determined reliability levels, said control determines weighting factors for (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map-based road curvature.

3. The vision system of claim 2, wherein, responsive at least in part to determined weighting factors, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling.

4. The vision system of claim 3, wherein, responsive to a determination that the map data used in generating the map-based road curvature is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the determined right lane markings.

5. The vision system of claim 4, wherein, responsive to a determination that the map data used in generating the map-based road curvature is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the map data.

6. The vision system of claim 3, wherein, responsive to a determination that the left lane markings determination is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined right lane markings and (ii) the map-based road curvature.

7. The vision system of claim 6, wherein, responsive to a determination that the left lane markings determination is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the left lane markings determination.

8. The vision system of claim 3, wherein, responsive to a determination that the right lane markings determination is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the map-based road curvature.

9. The vision system of claim 8, wherein, responsive to a determination that the right lane markings determination is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the right lane markings determination.

10. The vision system of claim 1, wherein, responsive to a determination that a reliability level is below a threshold level for all of (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map data used to generate the map-based road curvature, said control determines an estimated path of travel of the equipped vehicle responsive at least in part to (i) a speed sensor of the equipped vehicle and (ii) a yaw rate sensor of the equipped vehicle.

11. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle equipped with said vision system, wherein said camera has a field of view forward of the equipped vehicle and is operable to capture image data;

a vehicle-based GPS system operable to determine a geographical location of the equipped vehicle;

a control including an image processor;

wherein said image processor processes image data captured by said camera;

wherein, responsive at least in part to processing of captured image data, said control is operable to determine (i) left lane markings along a left side of a lane in which the equipped vehicle is traveling on a road and (ii) right lane markings along a right side of the lane in which the equipped vehicle is traveling on the road;

wherein, responsive at least in part to said GPS system determining the geographic location of the equipped vehicle, said vision system generates a map-based road curvature for the road ahead of the equipped vehicle based on map data;

wherein, responsive at least in part to processing of captured image data by said image processor, said control determines a particular lane in which the equipped vehicle is traveling from a plurality of lanes of the road and generates a camera-based road curvature for the road ahead of the equipped vehicle;

wherein, responsive to generation of the map-based road curvature and the camera-based road curvature, said control fuses the map-based road curvature and the camera-based road curvature;

wherein, responsive to fusing of the map-based road curvature and the camera-based road curvature, said control is operable to estimate an actual road curvature of the road ahead of the equipped vehicle;

wherein, responsive to determined reliability levels of (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map data used to generate the map-based road curvature, said control determines weighting factors for (i) the camera-based road curvature and (ii) the map-based road curvature;

wherein, responsive at least in part to determined weighting factors, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling based on the weighted (i) camera-based road curvature and (ii) map-based road curvature;

wherein, in fusing of map-based road curvature and camera-based road curvature, said control diminishes the weighting factor of the camera-based road curvature when reliability level is below a threshold level for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the determined right lane markings; and

wherein, in fusing of map-based road curvature and camera-based road curvature, said control diminishes the weighting factor of the map-based road curvature when reliability level is below a threshold level for the map data.

12. The vision system of claim 11, wherein, responsive to a determination that the map data used in generating the map-based road curvature is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the determined right lane markings.

13. The vision system of claim 12, wherein, responsive to a determination that the map data used in generating the map-based road curvature is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the map data.

14. The vision system of claim 11, wherein, responsive to a determination that the left lane markings determination is compromised, said control determines a higher weighting factor for the map-based road curvature.

15. The vision system of claim 14, wherein, responsive to a determination that the left lane markings determination is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the left lane markings determination.

16. The vision system of claim 11, wherein, responsive to a determination that the right lane markings determination is compromised, said control determines a higher weighting factor for the map-based road curvature.

17. The vision system of claim 16, wherein, responsive to a determination that the right lane markings determination is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the right lane markings determination.

18. The vision system of claim 11, wherein, responsive to a determination that a reliability level is below a threshold level for all of (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map data used to generate the map-based road curvature, said control determines an estimated path of travel of the equipped vehicle responsive at least in part to (i) a speed sensor of the equipped vehicle and (ii) a yaw rate sensor of the equipped vehicle.

19. A vision system of a vehicle, said vision system comprising:

a camera disposed at a vehicle equipped with said vision system, wherein said camera has a field of view forward of the equipped vehicle and is operable to capture image data;

a vehicle-based GPS system operable to determine a geographical location of the equipped vehicle;

a control including an image processor;

wherein said image processor processes image data captured by said camera;

wherein, responsive at least in part to processing of captured image data, said control is operable to determine (i) left lane markings along a left side of a lane in which the equipped vehicle is traveling on a road and (ii) right lane markings along a right side of the lane in which the equipped vehicle is traveling on the road;

wherein, responsive at least in part to said GPS system determining the geographic location of the equipped vehicle, said vision system generates a map-based road curvature for the road ahead of the equipped vehicle based on map data;

wherein, responsive at least in part to processing of captured image data by said image processor, said control determines a particular lane in which the equipped vehicle is traveling from a plurality of lanes of the road and generates a camera-based road curvature for the road ahead of the equipped vehicle;

wherein, responsive to generation of the map-based road curvature and the camera-based road curvature, said control fuses the map-based road curvature and the camera-based road curvature;

wherein, responsive to fusing of the map-based road curvature and the camera-based road curvature, said control is operable to estimate an actual road curvature of the road ahead of the equipped vehicle;

wherein, responsive to determined reliability levels of (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map data used to generate the map-based road curvature, said control determines weighting factors for (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map-based road curvature;

wherein, responsive at least in part to determined weighting factors, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling based on the weighted (i) determined left lane markings, (ii) determined right lane markings and (iii) map-based road curvature;

wherein, in fusing of map-based road curvature and camera-based road curvature, said control diminishes the weighting factor of the camera-based road curvature when reliability level is below a threshold level for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the determined right lane markings; and

wherein, in fusing of map-based road curvature and camera-based road curvature, said control diminishes the weighting factor of the map-based road curvature when reliability level is below a threshold level for the map data;

wherein, responsive to a determination that the map data used to generate the map-based road curvature is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the map data;

wherein, responsive to a determination that the left lane markings determination is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the left lane markings determination;

wherein, responsive to a determination that the right lane markings determination is invalid, said control estimates the actual road curvature of the road ahead of the equipped vehicle along the particular lane in which the equipped vehicle is traveling without use of the right lane markings determination; and

wherein, responsive to a determination that reliability level is below a threshold level for all of (i) the determined left lane markings, (ii) the determined right lane markings and (iii) the map data used to generate the map-based road curvature, said control determines an estimated path of travel of the equipped vehicle responsive at least in part to (i) a speed sensor of the equipped vehicle and (ii) a yaw rate sensor of the equipped vehicle.

20. The vision system of claim 19, wherein, responsive to a determination that the map data used to generate the map-based road curvature is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the determined right lane markings, and wherein, responsive to a determination that a left lane markings determination is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined right lane markings and (ii) the map-based road curvature, and wherein, responsive to a determination that a right lane markings determination is compromised, said control determines a higher weighting factor for at least one selected from the group consisting of (i) the determined left lane markings and (ii) the map-based road curvature.