Distinguishing lane markings for a vehicle to follow

• US 10,048,691 B2
• Filed: 10/03/2017
• Issued: 08/14/2018
• Est. Priority Date: 11/17/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Reiff, Brielle, Jain, Jinesh J, Kadetotad, Sneha
• Primary Examiner(s): EL CHANTI, HUSSEIN A
• Application Number:
  US15/724,093
• Publication Number:
  US 20180024560A1
• Time in Patent Office: 315 Days
• CPC Class Codes: B60W 2420/403 Image sensing, e.g. optical camera
• CPC Class Codes: B60W 2420/408 Radar; Laser, e.g. lidar
• CPC Class Codes: B60W 2552/53 Road markings, e.g. lane marker or crosswalk
• CPC Class Codes: B60W 2554/4041 Position
• CPC Class Codes: B60W 2710/20 Steering systems
• CPC Class Codes: B60W 30/12 Lane keeping
• CPC Class Codes: B60Y 2300/12 Lane keeping
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road

Abstract

The present invention extends to methods, systems, and computer program products for distinguishing roadway surface lane markings for a vehicle to follow. Automated driving or driving assist vehicles utilize sensors to help the vehicle navigate on roadways or in parking areas. The sensors can utilize, for example, the painted surface markings to help guide the vehicle on its path. When ambiguity is detected between roadway surface markings, decision making algorithms identify a set of roadway surface markings for a vehicle to abide by. The sensors can also identify the location and trajectory of neighboring vehicles to increase confidence with respect to an identified set of roadway surface markings.

Claims

1. A method for automatically controlling a vehicle, the method comprising a processor:

accessing sensor data from a plurality of sensors at the vehicle, the plurality of sensors monitoring a roadway where the vehicle is located;

determining bounding boxes from the accessed sensor data, the bounding boxes representing regions of interest in the roadway for distinguishing roadway surface lane markings;

processing the accessed sensor data within the bounding boxes to identify a set of roadway surface lane markings the vehicle is to follow; and

controlling the vehicle within a lane of the roadway by following the set of roadway surface lane markings.

2. The method of claim 1, wherein accessing sensor data comprises accessing sensor data indicating one or more of:

speed limit information painted onto a surface of the roadway or stop sign information painted onto a surface of the roadway.

3. The method of claim 1, wherein determining bounding boxes from the accessed sensor data comprises determining bounding boxes for distinguishing neighboring vehicles.

4. The method of claim 3, further comprising:

processing the accessed sensor data to identify location and trajectory of the neighboring vehicles; and

using the location and trajectory of the neighboring vehicles to increase confidence with respect to identification of the identified set of roadway surface lane markings.

5. The method of claim 4, wherein identifying the location and trajectories associated with neighboring vehicles on or near the road comprises identifying vehicles in the same lane as the vehicle.

6. The method of claim 4, wherein identifying the location and trajectories associated with neighboring vehicles on or near the road comprises identifying vehicles in a different lane than the vehicle.

7. The method of claim 4, wherein identifying the location and trajectories associated with neighboring vehicles comprises at least one of:

identifying vehicles traveling in a same direction relative to a direction of the vehicle or identifying vehicles traveling in an opposite direction relative to the direction of the vehicle.

8. The method of claim 1, wherein identifying a set of roadway surface lane markings comprises detecting at least two candidate lane markings of a different visual appearance.

9. The method of claim 8, wherein detecting at least two lane markings of a different visual appearance comprises detecting at least two lane markings that differ in one or more of:

intensity of color, reflectivity, presence of lane reflectors, paint cracking, or road tape peeling.

10. The method of claim 1, wherein accessing sensor data from a plurality of sensors comprises accessing data from one or more of:

an image capture device, a lidar system, and a radar system.

11. A system, the system comprising:

one or more processors;

system memory coupled to one or more processors, the system memory storing instructions that are executable by the one or more processors; and

the one or more processors executing the instructions stored in the system memory to automatically control a vehicle, including the following;

access sensor data from a plurality of sensors at the vehicle, the plurality of sensors monitoring a roadway where the vehicle is located;

determine bounding boxes from the accessed sensor data, the bounding boxes representing regions of interest in the roadway for distinguishing roadway surface lane markings;

process the accessed sensor data within the bounding boxes to identify a set of roadway surface lane markings the vehicle is to follow; and

control the vehicle within a lane of the roadway by following the set of roadway surface lane markings.

12. The system of claim 11, wherein the one or more processors executing the instructions stored in the system memory to determine bounding boxes from the accessed sensor data comprises the one or more processors executing the instructions stored in the system memory to determine bounding boxes for distinguishing neighboring vehicles;

and

further comprising the one or more processors executing the instructions stored in the system memory to;

process the accessed sensor data to identify location and trajectory of the neighboring vehicles; and

use the location and trajectory of the neighboring vehicles to increase confidence with respect to identification of the identified set of roadway surface lane markings.

13. The system of claim 11, wherein the one or more processors executing the instructions stored in the system memory to identify a set of roadway surface lane markings comprises the one or more processors executing the instructions stored in the system memory to detect at least two lane markings of a different visual appearance.

14. The system of claim 13, wherein the one or more processors executing the instructions stored in the system memory to detect at least two candidate lane markings of a different visual appearance comprises detecting at least two candidate lane markings that differ in on or more of:

intensity of color, reflectivity, presence of lane reflectors, paint cracking, or road tape peeling.

15. The system of claim 11, wherein the one or more processors executing the instructions stored in the system memory to access sensor data from a plurality of sensors comprises the one or more processors executing the instructions stored in the system memory to access data from one or more of:

an image capture device, a lidar system, and a radar system.

16. A vehicle, the vehicle comprising:

one or more processors;

system memory coupled to one or more processors, the system memory storing instructions that are executable by the one or more processors;

a plurality of sensors monitoring a roadway where the vehicle is located; and

the one or more processors executing the instructions stored in the system memory to automatically control the vehicle, including the following;

access sensor data from the plurality of sensors;

determine bounding boxes from the accessed sensor data, the bounding boxes representing regions of interest in the roadway for distinguishing roadway surface lane markings;

process the accessed sensor data within the bounding boxes to identify a set of roadway surface lane markings the vehicle is to follow; and

control the vehicle within a lane of the roadway by following the set of roadway surface lane markings.

17. The vehicle of claim 16, wherein the one or more processors executing the instructions stored in the system memory to determine bounding boxes from the accessed sensor data comprises the one or more processors executing the instructions stored in the system memory to determine bounding boxes for distinguishing neighboring vehicles;

and

further comprising the one or more processors executing the instructions stored in the system memory to;

process the accessed sensor data to identify location and trajectory of the neighboring vehicles; and

use the location and trajectory of the neighboring vehicles to increase confidence with respect to identification of the identified set of roadway surface lane markings.

18. The vehicle of claim 16, wherein the one or more processors executing the instructions stored in the system memory to identify a set of roadway surface lane markings comprises the one or more processors executing the instructions stored in the system memory to detect at least two lane markings of a different visual appearance.

19. The vehicle of claim 18, wherein the one or more processors executing the instructions stored in the system memory to detect at least two candidate lane markings of a different visual appearance comprises detecting at least two candidate lane markings that differ in on or more of:

intensity of color, reflectivity, presence of lane reflectors, paint cracking, or road tape peeling.

20. The vehicle of claim 16, wherein the one or more processors executing the instructions stored in the system memory to access sensor data from a plurality of sensors comprises the one or more processors executing the instructions stored in the system memory to access data from one or more of:

an image capture device, a lidar system, and a radar system.