Distinguishing lane markings for a vehicle to follow

• US 9,829,888 B2
• Filed: 11/17/2015
• Issued: 11/28/2017
• Est. Priority Date: 11/17/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Reiff, Brielle, Jain, Jinesh J, Kadetotad, Sneha
• Primary Examiner(s): EL CHANTI, HUSSEIN A
• Application Number:
  US14/943,573
• Publication Number:
  US 20170139417A1
• Time in Patent Office: 742 Days
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60Y 2300/12 Lane keeping
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

The present invention extends to methods, systems, and computer program products for distinguishing lane markings for a vehicle to follow. Automated driving or driving assist vehicles utilize sensors to help the vehicle navigate on roadways or in parking areas. The sensors can utilize, for example, the painted surface markings to help guide the vehicle on its path. Aspects of the invention use a first type of sensor and at least a second different type of sensor to identify road surface markings. When ambiguity is detected between road surface markings, decision making algorithms identify the correct set of markings for a vehicle to abide by. The sensors also identify the location and trajectory of neighboring vehicles to increase confidence with respect to the identified road-surface markings.

Claims

1. A method for automated control of a vehicle based on roadway surface lane markings, the method comprising a processor:

accessing sensor data from a plurality of sensors at the vehicle, the plurality of sensors monitoring a roadway where the vehicle is traveling;

determining bounding boxes from the accessed sensor data, the bounding boxes representing regions of interest in the roadway for distinguishing roadway surface lane markings and neighboring vehicles;

processing the accessed sensor data within the bounding boxes to identify multiple sets of roadway surface lane markings;

utilizing the sensor data to identify a set of roadway surface lane markings, from among the multiple sets of roadway surface lane markings, the vehicle is to follow; and

controlling the vehicle within a lane of the roadway by following the set of roadway surface lane markings.

2. The method of claim 1, further comprising:

processing the accessed sensor data to identify location and trajectory of neighboring vehicles; and

using the location and trajectory of neighboring vehicles to increase confidence with respect to the identified set of roadway surface lane markings the vehicle is to follow.

3. The method of claim 1, wherein identifying a set of roadway surface lane markings comprises detecting at least two candidate lane markings of a different visual appearance.

4. The method of claim 3, wherein detecting at least two candidate lane markings of a different visual appearance comprise detecting at least two candidate lane markings that differ in on or more of:

intensity of color, reflectivity, presence of lane reflectors, paint cracking, and road tape peeling.

5. The method claim 1, wherein accessing sensor data from a plurality of sensors comprises accessing data from two or more of:

an image capture device, a lidar system, and a radar system.

6. A method for use at a computer system, the computer system including one or more processors and system memory, the method for automatically controlling a vehicle by distinguishing road-surface markings on a roadway, the method comprising the one or more processors:

accessing sensor data from a plurality of sensors, the plurality of sensors monitoring the roadway where the vehicle is traveling, the plurality of sensors including a first type of sensor and at least a second different type of sensor, the sensor data indicating road-surface markings, the road-surface markings including intensity of color and reflectivity, the sensor data also indicating the location and trajectory of neighboring vehicles;

determining bounding boxes from the accessed sensor data, the bounding boxes representing regions of interest for distinguishing road-surface markings and neighboring vehicles;

processing the accessed sensor data within the bounding boxes to determine that multiple road-surface markings are present;

in response to determining that multiple road-surface markings are present;

utilizing the sensor data to identify road-surface markings, from among the multiple road-surface markings, the vehicle is to follow;

using the location and trajectory of neighboring vehicles to increase confidence with respect to the identified road-surface markings; and

controlling the vehicle'"'"'s location and trajectory on the roadway by following the identified road-surface markings.

7. The method of claim 6, wherein determining that multiple road-surface markings are present comprises detecting at least two candidate lane markings of a different visual appearance.

8. The method of claim 7, further comprising, determining a visual appearance of each of the multiple road-surface markings from one or more of:

intensity of color, reflectivity, presence of lane reflectors, paint cracking, and road tape peeling.

9. The method of claim 6, wherein the sensor data comes from a plurality of sensors, the plurality of sensors selected from among:

lidar systems, image-capture devices, and radar systems.

10. The method of claim 9, further comprising, determining the location and trajectory of neighboring vehicles from the accessed sensor data.

11. The method of claim 6, wherein accessing sensor data comprises accessing sensor data indicating one or more of speed limit information and stop sign information painted onto the road surface.

12. The method of claim 6, wherein identifying the location and trajectories associated with neighboring vehicles on or near the road comprises identifying vehicles in the same lane as the vehicle.

13. The method of claim 6, wherein identifying the location and trajectories associated with neighboring vehicles on or near the road comprises identifying vehicles in a different lane than the vehicle.

14. The method of claim 6, wherein identifying the location and trajectories associated with neighboring vehicles comprises identifying vehicles traveling in the same direction as the vehicle.

15. The method of claim 6, wherein identifying the location and trajectories associated with neighboring vehicles comprises identifying vehicles traveling in the opposite direction as the vehicle.

16. A vehicle comprising:

one or more processors;

system memory, the system memory storing instructions that are executable by the one or more processors;

a plurality of sensors, the plurality of sensors including a first type of sensor and at least a second different type of sensor, the plurality of sensors monitoring a roadway where the vehicle is traveling;

the one or more processors executing the instructions stored in the system memory to automatically control the vehicle, including the following;

access sensor data from the plurality of sensors, the sensor data indicating road-surface markings the road-surface markings including intensity of color and reflectivity, the sensor data also indicating the location and trajectory of neighboring vehicles;

determine bounding boxes from the accessed sensor data, the bounding boxes representing regions of interest for distinguishing road-surface markings and neighboring vehicles;

process the accessed sensor data within the bounding boxes to determine that multiple road-surface markings are present;

in response to determining that multiple road-surface markings are present;

utilize the sensor data to identify road-surface markings, from among the multiple road-surface markings, the vehicle is to follow;

use the location and trajectory of neighboring vehicles to increase confidence with respect to the identified road-surface markings; and

control the vehicle'"'"'s location and trajectory on the roadway by following the identified road-surface markings.

17. The vehicle of claim 16, wherein the one or more processors executing the instructions stored in the system memory to sense sensor data comprise the one or more processors executing the instructions stored in the system memory to receive at least one type of data from one of the vehicle'"'"'s sensors and at least a second, different type of data from a different type of the vehicle'"'"'s sensors.

18. The vehicle of claim 16, further comprising the one or more processors executing the instructions stored in the system memory to query the different types of sensor data for road surface markings.

19. The vehicle of claim 16, further comprising the one or more processors executing the instructions stored in the system memory to query the different types of sensor data for neighboring vehicles.

20. The vehicle of claim 16, further comprising the one or more processors executing the instructions stored in the system memory to parse the sensor data for regions of interest, the regions of interest being bound by bounding boxes.