Enhancing basic roadway-intersection models using high intensity image data

• US 9,081,383 B1
• Filed: 01/22/2014
• Issued: 07/14/2015
• Est. Priority Date: 01/22/2014
• Status: Active Grant

• Current Assignee: Waymo LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Montemerlo, Michael Steven, Tisdale, John, Furman, Vadim
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): TESTARDI, DAVID A
• Application Number:
  US14/161,295
• Time in Patent Office: 538 Days
• Field of Search: 701/25
• US Class Current: 1/1
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/00 Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

Systems and methods are provided that may optimize basic models of an intersection in a roadway with high intensity image data of the intersection of the roadway. More specifically, parameters that define the basic model of the intersection in the roadway may be adjusted to more accurately define the intersection. For example, by comparing a shape of the intersection predicted by the basic model with extracted curbs and lane boundaries from elevation and intensity maps, the intersection parameters can be optimized to match real intersection-features in the environment. Once the optimal intersection parameters have been found, roadgraph features describing the intersection may be extracted.

Claims

1. A method comprising:

receiving, using a computing device, parameterized intersection data indicative of an intersection in a roadway of an environment, wherein the parameterized intersection data includes a plurality of parameters that define the intersection in the environment;

based on the plurality of parameters, determining a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted curb locations, predicted lane locations, or predicted pedestrian-control line locations;

receiving map data based on a detection of the roadway in the environment by sensors on a vehicle that traverses the environment, wherein the map data includes one or more of candidate curb locations of the intersection, candidate lane locations of the intersection, or candidate pedestrian-control lines of the intersection;

performing a first optimization of the predicted curb locations to the candidate curb locations to minimize correspondence error between the predicted curb locations and the candidate curb locations for optimal curb locations;

performing a second optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations;

performing a third optimization of the predicted pedestrian-control lines to the candidate pedestrian-control lines to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization, the second optimization, and the third optimization, determining a second configuration of the intersection that includes the optimal curb locations, the optimal lane locations, and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigating an autonomous vehicle through the intersection.

2. The method of claim 1, further comprising determining a roadgraph of the intersection that includes a symbolic representation of the second configuration of the intersection.

3. The method of claim 2, further comprising providing the roadgraph to the autonomous vehicle.

4. The method of claim 2, further comprising, based on the roadgraph, providing instructions to control the autonomous vehicle, wherein navigating the autonomous vehicle through the intersection is further based on the instructions.

5. The method of claim 1, wherein the plurality of parameters include one or more of parameters that define legs of the intersection, parameters that define lanes of the intersection, parameters that define medians in the intersection, and parameters that define lane connections of the intersection, wherein the lane connections define paths between lanes in the intersection.

6. The method of claim 1, wherein the map data includes at least one of an image map, an intensity map, or an elevation map.

7. The method of claim 1, wherein the predicted pedestrian-control lines and the candidate pedestrian-control lines include at least one of a stop line or a crosswalk.

8. The method of claim 1,

wherein performing the first optimization of the predicted curb locations to the candidate curb locations to determine the optimal curb locations of the intersection in the environment includes minimizing a distance between the predicted curb locations and the candidate curb locations by varying the plurality of parameters,

wherein performing the second optimization of the predicted lane locations to the candidate lane locations to determine the optimal lane locations of the intersection in the environment includes minimizing a distance between the predicted lane locations and the candidate lane locations by varying the plurality of parameters, and

wherein performing the third optimization of the predicted pedestrian-control lines to the candidate pedestrian-control lines to determine the optimal control line locations of the intersection in the environment includes minimizing a distance between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations by varying the plurality of parameters.

9. A system comprising:

sensors on one or more vehicles that are configured to traverse an environment; and

a computer system configured to;

receive parameterized intersection data indicative of an intersection in a roadway of the environment, wherein the parameterized intersection data includes a plurality of parameters that define the intersection in the environment;

based on the plurality of parameters, determine a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted curb locations, predicted lane locations, or predicted pedestrian-control line locations;

receive map data based on a detection of the roadway in the environment by the sensors on the one or more vehicles that traverse the environment, wherein the map data includes one or more of candidate curb locations of the intersection, candidate lane locations of the intersection, or candidate pedestrian-control lines of the intersection;

perform a first optimization of the predicted curb locations to the candidate curb locations to minimize correspondence error between the predicted curb locations and the candidate curb locations for optimal curb locations;

perform a second optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations;

perform a third optimization of the predicted pedestrian-control lines to the candidate pedestrian-control lines to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization, the second optimization, and the third optimization, determine a second configuration of the intersection that includes the optimal curb locations, the optimal lane locations, and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigate an autonomous vehicle through the intersection.

10. The system of claim 9, wherein the computer system is further configured to determine a roadgraph of the intersection that includes a symbolic representation of the second configuration of the intersection.

11. The system of claim 10, wherein the computer system is further configured to provide the roadgraph to the autonomous vehicle.

12. The system of claim 11, wherein the computer system is further configured to, based on the roadgraph, provide instructions to control the autonomous vehicle, wherein navigating the autonomous vehicle through the intersection is further based on the instructions.

13. The system of claim 9, wherein the plurality of parameters include one or more of parameters that define legs of the intersection, parameters that define lanes of the intersection, parameters that define medians in the intersection, and parameters that define lane connections of the intersection, wherein the lane connections define paths between lanes in the intersection.

14. The system of claim 9, wherein the map data includes at least one of an image map, an intensity map, or an elevation map.

15. The system of claim 9, wherein the predicted pedestrian-control lines and the candidate pedestrian-control lines include at least one of a stop line or a crosswalk.

16. The system of claim 9, wherein the computer system is further configured to:

minimize a distance between the predicted curb locations and the candidate curb locations by varying the plurality of parameters,

minimize a distance between the predicted lane locations and the candidate lane locations by varying the plurality of parameters, and

minimize a distance between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations by varying the plurality of parameters.

17. A non-transitory computer readable medium having stored therein instructions, that when executed by a computer system configured to control an autonomous vehicle, cause the computer system to perform functions comprising:

receiving, using the computer system, parameterized intersection data indicative of an intersection in a roadway of an environment, wherein the parameterized intersection data includes a plurality of parameters that define the intersection in the environment;

based on the plurality of parameters, determining a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted curb locations, predicted lane locations, or predicted pedestrian-control line locations;

receiving map data based on a detection of the roadway in the environment by sensors on a vehicle that traverses the environment, wherein the map data includes one or more of candidate curb locations of the intersection, candidate lane locations of the intersection, or candidate pedestrian-control lines of the intersection;

performing a first optimization of the predicted curb locations to the candidate curb locations to minimize correspondence error between the predicted curb locations and the candidate curb locations for optimal curb locations;

performing a second optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations;

performing a third optimization of the predicted pedestrian-control lines to the candidate pedestrian-control lines to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization, the second optimization, and the third optimization, determining a second configuration of the intersection that includes the optimal curb locations, the optimal lane locations, and the optimal control line locations; and

based on the second configuration of the intersection, navigating the autonomous vehicle through the intersection.

18. The non-transitory computer readable medium of claim 17, wherein the plurality of parameters include one or more of parameters that define legs of the intersection, parameters that define lanes of the intersection, parameters that define medians in the intersection, and parameters that define lane connections of the intersection, wherein the lane connections define paths between lanes in the intersection.

19. The non-transitory computer readable medium of claim 17, wherein the map data includes at least one of an image map, an intensity map, or an elevation map.

20. The non-transitory computer readable medium of claim 17, wherein the predicted pedestrian-control lines and the candidate pedestrian-control lines include at least one of a stop line or a crosswalk.