Enhancing basic roadway-intersection models using high intensity image data

• US 9,494,942 B1
• Filed: 05/28/2015
• Issued: 11/15/2016
• Est. Priority Date: 01/22/2014
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Montemerlo, Michael Steven, Tisdale, John, Furman, Vadim
• Primary Examiner(s): Badii, Behrang
• Assistant Examiner(s): Testardi, David
• Application Number:
  US14/724,243
• Time in Patent Office: 537 Days
• US Class Current: 1/1
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 30/18154 Approaching an intersection
• CPC Class Codes: G01C 21/005 with correlation of navigation data from several sources, e.g. map or contour matching G01C21/30 takes precedence
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0246 using a video camera in combination with image processing means
• CPC Class Codes: G05D 1/0248 in combination with a laser lasers per se H01S
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06V 20/588 Recognition of the road, e.g. of lane markings; Recognition of the vehicle driving pattern in relation to the road
• CPC Class Codes: G08G 1/00 Traffic control systems for road vehicles arrangement of road signs or traffic signals E01F9/00 ; automatic vehicle control B62D
• CPC Class Codes: G08G 1/164 Centralised systems, e.g. external to vehicles

Abstract

Systems and methods are provided that may optimize basic models of an intersection in a roadway with high intensity image data of the intersection of the roadway. More specifically, parameters that define the basic model of the intersection in the roadway may be adjusted to more accurately define the intersection. For example, by comparing a shape of the intersection predicted by the basic model with extracted curbs and lane boundaries from elevation and intensity maps, the intersection parameters can be optimized to match real intersection-features in the environment. Once the optimal intersection parameters have been found, roadgraph features describing the intersection may be extracted.

Claims

1. A method comprising:

based on a plurality of parameters that define an intersection in a roadway of an environment, determining, by a computing device, a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted locations of sides of the intersection and predicted pedestrian-control line locations of the intersection;

receiving map data based on a detection of the roadway in the environment by sensors on a vehicle that traverses the environment, wherein the map data includes one or more of candidate locations of sides of the intersection and candidate pedestrian-control line locations of the intersection;

performing a first optimization of the predicted locations of sides of the intersection to the candidate locations of sides of the intersection to minimize correspondence error between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection for optimal locations of sides of the intersection;

performing a second optimization of the predicted pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization and the second optimization, determining a second configuration of the intersection that includes the optimal locations of sides of the intersection and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigating an autonomous vehicle through the intersection.

2. The method of claim 1, further comprising determining a roadgraph of the intersection that includes a symbolic representation of the second configuration of the intersection.

3. The method of claim 2, further comprising providing the roadgraph to the autonomous vehicle.

4. The method of claim 2, further comprising, based on the roadgraph, providing instructions to control the autonomous vehicle, wherein navigating the autonomous vehicle through the intersection is further based on the instructions.

5. The method of claim 1, wherein the plurality of parameters include one or more of parameters that define legs of the intersection, parameters that define lanes of the intersection, parameters that define medians in the intersection, and parameters that define lane connections of the intersection, wherein the lane connections define paths between lanes in the intersection.

6. The method of claim 1, wherein the map data includes at least one of an image map, an intensity map, or an elevation map.

7. The method of claim 1, wherein performing the first optimization of the predicted locations of sides of the intersection to the candidate locations of sides of the intersection to minimize correspondence error between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection for the optimal locations of sides of the intersection comprises minimizing a distance between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection by varying the plurality of parameters, and

wherein performing the second optimization of the predicted pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for the optimal pedestrian-control line locations comprises minimizing a distance between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations by varying the plurality of parameters.

8. The method of claim 1, wherein the first configuration of the intersection further includes predicted lane locations, and wherein the map data further includes candidate lane locations of the intersection, the method further comprising:

performing a third optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations,

wherein determining the second configuration of the intersection is further based on the third optimization, and

wherein the second configuration further includes the optimal lane locations.

9. The method of claim 8, wherein performing the third optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for the optimal lane locations comprises minimizing a distance between the predicted lane locations and the candidate lane locations by varying the plurality of parameters.

10. The method of claim 1, wherein pedestrian-control line locations include locations of at least one of a stop line or a crosswalk.

11. The method of claim 1, wherein locations of sides of the intersection include locations of at least one of:

a curb and an edge of a surface of the intersection.

12. A system comprising:

sensors on one or more vehicles that are configured to traverse an environment; and

a computer system configured to;

based on a plurality of parameters that define an intersection in a roadway of an environment, determine a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted locations of sides of the intersection and predicted pedestrian-control line locations of the intersection;

receive map data based on a detection of the roadway in the environment by the sensors on the one or more vehicles that traverse the environment, wherein the map data includes one or more of candidate locations of sides of the intersection and candidate pedestrian-control line locations of the intersection;

perform a first optimization of the predicted locations of sides of the intersection to the candidate locations of sides of the intersection to minimize correspondence error between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection for optimal locations of sides of the intersection;

perform a second optimization of the predicted lane pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization and the second optimization, determine a second configuration of the intersection that includes the optimal locations of sides of the intersection and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigate an autonomous vehicle through the intersection.

13. The system of claim 12, wherein the computer system is further configured to determine a roadgraph of the intersection that includes a symbolic representation of the second configuration of the intersection.

14. The system of claim 13, wherein the computer system is further configured to:

provide the roadgraph to the autonomous vehicle; and

based on the roadgraph, provide instructions to control the autonomous vehicle, wherein navigating the autonomous vehicle through the intersection is further based on the instructions.

15. The system of claim 12, wherein the plurality of parameters include one or more of parameters that define legs of the intersection, parameters that define lanes of the intersection, parameters that define medians in the intersection, and parameters that define lane connections of the intersection, wherein the lane connections define paths between lanes in the intersection.

16. The system of claim 12, wherein the map data includes at least one of an image map, an intensity map, or an elevation map.

17. The system of claim 12, wherein the first configuration of the intersection further includes predicted lane locations, and wherein the map data further includes candidate lane locations of the intersection, and wherein the computer system is further configured to:

perform a third optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations,

wherein determining the second configuration of the intersection is further based on the third optimization,

wherein the second configuration further includes the optimal lane locations.

18. The system of claim 12, wherein performing the first optimization of the predicted locations of sides of the intersection to the candidate locations of sides of the intersection to minimize correspondence error between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection for the optimal locations of sides of the intersection comprises minimizing a distance between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection by varying the plurality of parameters, and

wherein performing the second optimization of the predicted pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for the optimal pedestrian-control line locations comprises minimizing a distance between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations by varying the plurality of parameters.

19. A non-transitory computer readable medium having stored therein instructions, that when executed by a computer system configured to control an autonomous vehicle, cause the computer system to perform functions comprising:

based on a plurality of parameters that define an intersection in a roadway of an environment, determining a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted locations of sides of the intersection and predicted pedestrian-control line locations of the intersection;

receiving map data based on a detection of the roadway in the environment by sensors on a vehicle that traverses the environment, wherein the map data includes one or more of candidate locations of sides of the intersection and candidate pedestrian-control line locations of the intersection;

performing a first optimization of the predicted locations of sides of the intersection to the candidate locations of sides of the intersection to minimize correspondence error between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection for optimal locations of sides of the intersection;

performing a second optimization of the predicted pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization and the second optimization, determining a second configuration of the intersection that includes the optimal locations of sides of the intersection and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigating the autonomous vehicle through the intersection.

20. The non-transitory computer readable medium of claim 19, wherein the plurality of parameters include one or more of parameters that define legs of the intersection, parameters that define lanes of the intersection, parameters that define medians in the intersection, and parameters that define lane connections of the intersection, wherein the lane connections define paths between lanes in the intersection.

21. The non-transitory computer readable medium of claim 19, wherein the first configuration of the intersection further includes predicted lane locations, and wherein the map data further includes candidate lane locations of the intersection, the functions further comprising:

performing a third optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations,

wherein determining the second configuration of the intersection is further based on the third optimization, and

wherein the second configuration further includes the optimal pedestrian-control line locations.

22. A method comprising:

based on a plurality of parameters that define an intersection in a roadway of an environment, determining, by a computing device, a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted lane locations of the intersection and predicted pedestrian-control line locations of the intersection;

receiving map data based on a detection of the roadway in the environment by sensors on a vehicle that traverses the environment, wherein the map data includes one or more of candidate lane locations of the intersection and candidate pedestrian-control line locations of the intersection;

performing a first optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations;

performing a second optimization of the predicted pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization and the second optimization, determining a second configuration of the intersection that includes the optimal lane locations and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigating an autonomous vehicle through the intersection.

23. A method comprising:

based on a plurality of parameters that define an intersection in a roadway of an environment, determining, by a computing device, a first configuration of the intersection, wherein the first configuration of the intersection includes one or more of predicted locations of sides of the intersection, predicted lane locations of the intersection, and predicted pedestrian-control line locations of the intersection;

receiving map data based on a detection of the roadway in the environment by sensors on a vehicle that traverses the environment, wherein the map data includes one or more of candidate locations of sides of the intersection, candidate lane locations of the intersection, and candidate pedestrian-control line locations of the intersection;

performing a first optimization of the predicted locations of sides of the intersection to the candidate locations of sides of the intersection to minimize correspondence error between the predicted locations of sides of the intersection and the candidate locations of sides of the intersection for optimal locations of sides of the intersection;

performing a second optimization of the predicted lane locations to the candidate lane locations to minimize correspondence error between the predicted lane locations and the candidate lane locations for optimal lane locations;

performing a third optimization of the predicted pedestrian-control line locations to the candidate pedestrian-control line locations to minimize correspondence error between the predicted pedestrian-control line locations and the candidate pedestrian-control line locations for optimal pedestrian-control line locations;

based on one or more of the first optimization, the second optimization, and the third optimization, determining a second configuration of the intersection that includes the optimal locations of sides of the intersection, the optimal lane locations, and the optimal pedestrian-control line locations; and

based on the second configuration of the intersection, navigating an autonomous vehicle through the intersection.