Relative atlas for autonomous vehicle and generation thereof

• US 10,521,913 B2
• Filed: 03/29/2018
• Issued: 12/31/2019
• Est. Priority Date: 03/29/2018
• Status: Active Grant

• Current Assignee: Aurora Operations Incorporated
• Original Assignee: Aurora Innovation, Inc.
• Inventors: Eade, Ethan, Bode, Michael
• Primary Examiner(s): Kan, Yuri
• Application Number:
  US15/940,516
• Publication Number:
  US 20190304097A1
• Time in Patent Office: 642 Days
• CPC Class Codes: G01C 21/20 Instruments for performing navigational calculations G01C21/24, G01C21/26 take precedence
• CPC Class Codes: G01C 21/3881 Tile-based structures
• CPC Class Codes: G06F 16/284 Relational databases
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06T 2207/20128 Atlas-based segmentation
• CPC Class Codes: G06T 7/12 Edge-based segmentation
• CPC Class Codes: G06T 7/162 involving graph-based methods

Abstract

A relative atlas graph is generated to store mapping data used by an autonomous vehicle. The relative atlas graph may be generated for a geographical area based on observations collected from the geographical area, and may include element nodes corresponding to elements detected from the observations along with edges that connect pairs of element nodes and define relative poses between the elements for connected pairs of element nodes.

Claims

1. A method of generating mapping data for use by an autonomous vehicle, the method comprising:

receiving a plurality of observations for a geographical area; and

generating a relative atlas graph for the geographical area based on the plurality of observations for the geographical area, including;

generating a plurality of element nodes for a plurality of elements detected from the plurality of observations, each element node having an associated element from among the plurality of elements;

generating a first plurality of edges, each edge in the first plurality of edges connecting a pair of element nodes from among the plurality of element nodes and defining a relative pose between the associated elements for the pair of element nodes;

generating a plurality of tile nodes for the geographical area, each tile node representing a region of the geographical area; and

generating a second plurality of edges, each edge in the second plurality of edges connecting an element node from among the plurality of element nodes to a tile node from among the plurality of tile nodes and defining a relative pose of the associated element for the connected element node within a frame of reference for the connected tile node;

wherein the first plurality of edges includes a first edge connecting a first element node among the plurality of element nodes to a second element node among the plurality of element nodes to define a relative pose between the associated elements for the first and second element nodes, and wherein the second plurality of edges includes a first edge connecting the first element node to a first tile node among the plurality of tile nodes and defining a relative pose of the associated element for the first element node within the frame of reference for the first tile node; and

storing the plurality of element nodes and the first and second pluralities of edges in the relative atlas graph.

2. The method of claim 1, wherein each edge includes a transformation field defining the relative pose between the associated elements of the connected pair of element nodes.

3. The method of claim 1, wherein storing the plurality of element nodes and the first plurality of edges in the relative atlas graph includes storing the plurality of element nodes as records in a node table and storing the first plurality of edges as records in an edge table.

4. The method of claim 3, wherein the node table and the edge table are maintained in a relational database.

5. The method of claim 1, wherein each of the plurality of element nodes includes an identifier field including an identifier for the associated element thereof, and wherein each of the first plurality of edges includes first and second identifier fields including identifiers for the associated elements for the pair of element nodes connected thereby.

6. The method of claim 1, wherein each of the plurality of element nodes includes a type field specifying a type that characterizes the respective element node and wherein each of the first plurality of edges includes a type field specifying a type that characterizes the respective edge.

7. The method of claim 6, wherein each of the plurality of element nodes includes a payload field including element-specific data, and wherein each of the first plurality of edges includes a payload field including edge-specific data.

8. The method of claim 7, wherein the element-specific data in the payload field for the first element node is structured based upon the type specified in the type field of the first element node, and wherein the edge-specific data in the payload field for the first edge among the first plurality of edges is structured based upon the type specified in the type field of the first edge.

9. The method of claim 1, wherein the second plurality of edges further includes a second edge connecting the first element node to a second tile node among the plurality of tile nodes and defining a relative pose of the associated element for the first element node within the frame of reference for the second tile node.

10. The method of claim 9, further comprising generating a tile-and-tile edge connecting the first and second tile nodes and defining a relative pose between the first and second tiles.

11. The method of claim 9, further comprising:

generating a fixed reference node defining a fixed reference point; and

generating first and second fixed reference edges respectively connecting the first and second tile nodes to the fixed reference node and respectively defining relative poses between the first and second tiles and the fixed reference point.

12. The method of claim 11, wherein the fixed reference point is an Earth Centered Earth Fixed (ECEF) reference point.

13. The method of claim 11, wherein the plurality of element nodes are not directly connected by edges to the fixed reference node such that no relative poses are defined between the plurality of elements and the fixed reference point.

14. The method of claim 1, further comprising storing in the first tile node geometry data defining a surface model for at least a portion of the region of the geographical area represented by the first tile node.

15. The method of claim 1, wherein the plurality of element nodes includes a plurality of gate nodes, each gate node defining a gate representing a permissible vehicle pose within a geographical area.

16. The method of claim 15, wherein generating the plurality of element nodes includes generating the plurality of gate nodes based upon one or more traversed paths in one or more vehicle logs collected by one or more vehicles traversing the geographical area.

17. The method of claim 15, wherein the first plurality of edges includes a plurality of gate-and-gate edges connecting pairs of gate nodes among the plurality of gate nodes, each gate-and-gate edge defining a permissible vehicle path between a pair of gates defined by the respective pair of gate nodes.

18. The method of claim 17, wherein a first gate-and-gate edge among the plurality of gate-and-gate edges connects first and second gate nodes among the plurality of gate nodes that represent a same lane of a roadway.

19. The method of claim 17, wherein a first gate-and-gate edge among the plurality of gate-and-gate edges connects first and second gate nodes among the plurality of gate nodes that represent different lanes of one or more roadways.

20. The method of claim 17, wherein the plurality of element nodes further includes a plurality of boundary nodes, each boundary node defining a physical or virtual boundary that restricts vehicle movement within a geographical area.

21. The method of claim 20, wherein generating the plurality of element nodes includes generating the plurality of boundary nodes based upon observations in one or more vehicle logs collected by one or more vehicles traversing the geographical area.

22. The method of claim 20, wherein the first plurality of edges further includes a plurality of boundary-and-boundary edges connecting pairs of boundary nodes among the plurality of boundary nodes, each boundary-and-boundary edge defining a relative pose between a pair of boundaries defined by the respective pair of boundary nodes.

23. The method of claim 20, wherein the first plurality of edges further includes a plurality of boundary-and-gate edges, each connecting a gate node from among the plurality of gate nodes with a boundary node from among the plurality of boundary nodes and defining a relative pose between a gate and boundary respectively defined by the gate node and boundary node connected thereby.

24. The method of claim 1, wherein the plurality of nodes includes a plurality of traffic nodes, each traffic node associated with a traffic control within a geographical area.

25. The method of claim 24, wherein generating the plurality of element nodes includes generating the plurality of traffic nodes based upon observations in one or more vehicle logs collected by one or more vehicles traversing the geographical area.

26. The method of claim 25, wherein generating the plurality of traffic nodes includes:

generating bounding boxes bounding traffic devices detected in images of the geographical area;

triangulating positions of traffic devices bounded by the bounding boxes within the geographical area; and

storing relative poses corresponding to the triangulated positions of the traffic devices in edges connected to one or more traffic nodes.

27. The method of claim 24, wherein the plurality of traffic nodes includes a control node defining a traffic control for a regulated geographical area, a plurality of traffic device nodes, each traffic device node defining a traffic device in the regulated geographical area, and a plurality of passage nodes defining passages through the regulated geographical area, and wherein the first plurality of edges includes traffic-device-and-control edges connecting the plurality of traffic device nodes to the control node and control-and-passage edges connecting the plurality of passage nodes to the control node.

28. The method of claim 27, wherein the first plurality of edges further includes traffic-device-and-tile edges connecting traffic device nodes from among the plurality of traffic device nodes to one or more tile nodes from among the plurality of nodes and respectively defining relative poses between traffic devices defined by respective traffic nodes and one or more tiles defined by the respective one or more tile nodes.

29. The method of claim 28, wherein the first plurality of edges further includes passage-and-gate edges connecting passage nodes from among the plurality of passage nodes to one or more gate nodes from among the plurality of element nodes to define gates that are controlled by the traffic control.

30. The method of claim 1, further comprising merging the generated relative atlas graph with another relative atlas graph for a second geographical area.

31. The method of claim 1, further comprising:

detecting a third element in the geographical area based on one or more observations among the plurality of observations;

determining that a third element node among the plurality of element nodes is for the third element; and

modifying a relative pose defined by a second edge connecting the third element node to a fourth element node for a fourth element based on one or more observations among the plurality of observations to reposition the third element relative to the fourth element.

32. The method of claim 31, wherein the fourth element node is connected to a fifth element node for a fifth element by a third edge, wherein modifying the relative pose defined by the second edge is performed without modifying a relative pose defined by the third edge such that modifying the relative pose defined by the second edge repositions the third element relative to the fifth element without modifying the relative pose defined by the third edge.

33. The method of claim 1, further comprising collecting the plurality of observations within one or more autonomous vehicles.

34. The method of claim 1, wherein generating the relative atlas graph is performed by an autonomous vehicle.

35. The method of claim 1, wherein generating the relative atlas graph is performed by a computer system that is remote from and in communication with one or more autonomous vehicles.

36. The method of claim 35, further comprising deploying at least a portion of the relative atlas graph to at least one of the one or more autonomous vehicles over a wireless communication network.

37. The method of claim 35, further comprising:

modifying the relative atlas graph in response to one or more observations; and

propagating one or more changes in the modified relative atlas graph to one or more of the plurality of autonomous vehicles.

38. The method of claim 1, further comprising importing external data into the relative atlas graph by:

converting the external data to a fixed frame of reference;

identifying one or more tile nodes in the relative atlas graph that respectively define one or more tiles proximate the external data in the fixed frame of reference;

projecting the external data into a frame of reference for each of the one or more tiles;

generating one or more element nodes for one or more elements represented by the external data;

generating one or more edges connecting the one or more element nodes to the one or more tile nodes, each of the one or more edges defining a relative pose for an element among the one or more elements in a frame of reference for a tile among the one or more tiles; and

adding the generated one or more element nodes and one or more edges to the relative atlas graph.

39. The method of claim 38, wherein importing the external data further includes performing a corrective warp of the external data based upon sparse correspondences between the external data and one or more existing elements defined in the relative atlas graph.

40. An apparatus, comprising:

a memory;

one or more processors coupled to the memory; and

computer instructions executable by the one or more processors to generate mapping data for use by an autonomous vehicle, the computer instructions configured to generate a relative atlas graph for a geographical area in the memory based on a plurality of observations for the geographical area by;

generating a plurality of element nodes for a plurality of elements detected from the plurality of observations, each element node having an associated element from among the plurality of elements;

generating a first plurality of edges, each edge in the first plurality of edges connecting a pair of element nodes from among the plurality of element nodes and defining a relative pose between the associated elements for the pair of element nodes;

generating a plurality of tile nodes for the geographical area, each tile node representing a region of the geographical area; and

generating a second plurality of edges, each edge in the second plurality of edges connecting an element node from among the plurality of element nodes to a tile node from among the plurality of tile nodes and defining a relative pose of the associated element for the connected element node within a frame of reference for the connected tile node;

wherein the first plurality of edges includes a first edge connecting a first element node among the plurality of element nodes to a second element node among the plurality of element nodes to define a relative pose between the associated elements for the first and second element nodes, and wherein the second plurality of edges includes a first edge connecting the first element node to a first tile node among the plurality of tile nodes and defining a relative pose of the associated element for the first element node within the frame of reference for the first tile node; and

storing the plurality of element nodes and the plurality of edges in the relative atlas graph in the memory.

41. The apparatus of claim 40, wherein the apparatus comprises an autonomous vehicle.

42. The apparatus of claim 40, wherein the apparatus comprises a computer system that is remote from and in communication with one or more autonomous vehicles.

43. A non-transitory computer readable storage medium storing computer instructions executable by one or more processors to perform a method of generating mapping data for use by an autonomous vehicle, the method comprising:

receiving a plurality of observations for a geographical area; and

generating a relative atlas graph for the geographical area based on the plurality of observations for the geographical area, including;

generating a plurality of element nodes for a plurality of elements detected from the plurality of observations, each element node having an associated element from among the plurality of elements;

generating a first plurality of edges, each edge in the first plurality of edges connecting a pair of element nodes from among the plurality of element nodes and defining a relative pose between the associated elements for the pair of element nodes;

generating a plurality of tile nodes for the geographical area, each tile node representing a region of the geographical area; and

generating a second plurality of edges, each edge in the second plurality of edges connecting an element node from among the plurality of element nodes to a tile node from among the plurality of tile nodes and defining a relative pose of the associated element for the connected element node within a frame of reference for the connected tile node;

wherein the first plurality of edges includes a first edge connecting a first element node among the plurality of element nodes to a second element node among the plurality of element nodes to define a relative pose between the associated elements for the first and second element nodes, and wherein the second plurality of edges includes a first edge connecting the first element node to a first tile node among the plurality of tile nodes and defining a relative pose of the associated element for the first element node within the frame of reference for the first tile node; and

storing the plurality of element nodes and the plurality of edges in the relative atlas graph.

44. A method of propagating mapping data changes to an autonomous vehicle, the method comprising:

receiving a plurality of observations for a geographical area;

modifying a relative atlas graph for the geographical area based on the plurality of observations for the geographical area, the relative atlas graph including a plurality of element nodes for a plurality of elements and a plurality of edges, each edge connecting a pair of element nodes from among the plurality of element nodes and defining a relative pose between the associated elements for the pair of element nodes, the relative atlas graph further includes a plurality of tile nodes for the geographical area, each tile node representing a region of the geographical area, wherein modifying the relative atlas graph includes;

detecting a changed element in the geographical area from the plurality of observations;

modifying a first relative pose defined by a first edge connecting a first element node for the changed element to a second element node for another element in the relative atlas graph to reposition the changed element relative to the other element; and

modifying a second relative pose defined by a second edge connecting the first element node to a first tile node among the plurality of tile nodes to reposition the changed element within the frame of reference for the first tile node; and

propagating the modification of the first and second relative poses to the autonomous vehicle.

45. The method of claim 44, wherein modifying the relative atlas graph further includes:

detecting a new element in the geographical area from the plurality of observations;

generating a new element node for the new element;

generating a third edge connecting the new element node to an existing element node in the relative atlas graph, the third edge defining a relative pose between the new element and an existing element defined by the existing element node;

storing the new element node and the third edge in the relative atlas graph; and

propagating the new element node and the third edge to the autonomous vehicle.

46. A method of generating mapping data for use by an autonomous vehicle, the method comprising:

receiving a vehicle log for a geographical area;

determining bounding boxes around a plurality of elements of interest in the vehicle log;

triangulating a location of each element of interest among the plurality of elements of interest using the determined bounding boxes;

generating in a relative atlas graph a plurality of element nodes defining the plurality of elements of interest;

generating a first plurality of edges in the relative atlas graph, each edge in the first plurality of edges connected to an element node among the plurality of element nodes and defining a relative pose for an element of interest defined thereby based on the triangulated location therefor;

generating a plurality of tile nodes for the geographical area, each tile node representing a region of the geographical area; and

generating a second plurality of edges, each edge in the second plurality of edges connecting an element node from among the plurality of element nodes to a tile node from among the plurality of tile nodes and defining a relative pose of the associated element for the connected element node within a frame of reference for the connected tile node;

wherein the first plurality of edges includes a first edge connecting a first element node among the plurality of element nodes to a second element node among the plurality of element nodes to define a relative pose between the associated elements for the first and second element nodes, and wherein the second plurality of edges includes a first edge connecting the first element node to a first tile node among the plurality of tile nodes and defining a relative pose of the associated element for the first element node within the frame of reference for the first tile node.

47. The method of claim 46, wherein determining the bounding boxes is performed automatically or manually with human assistance.

48. The method of claim 46, wherein the elements of interest are traffic devices.

49. The method of claim 48, wherein generating the second plurality of edges includes generating third and fourth edges respectively connecting a third element node for a first traffic device to second and third tile nodes among the plurality of tile nodes, the third and fourth edges respectively defining first and second relative poses for the first traffic device.

50. A method of deploying mapping data to an autonomous vehicle, the mapping data stored in a relative atlas graph including a plurality of element nodes for a plurality of elements, a first plurality of edges, each edge among the plurality of edges connecting a pair of element nodes from among the plurality of element nodes and defining a relative pose between the associated elements for the pair of element nodes, a plurality of tile nodes for a geographical area, each tile node representing a tile corresponding to a region of the geographical area, a second plurality of edges, each edge among the second plurality of edges connecting an element node from among the plurality of element nodes to a tile node from among the plurality of tile nodes and defining a relative pose of the associated element for the connected element node within a frame of reference for the connected tile node, and a third plurality of edges, each edge among the third plurality of edges connecting a pair of tile nodes from among the plurality of tile nodes and defining a relative pose between the associated tiles for the pair of tile nodes, the method comprising:

determining a location of the autonomous vehicle;

accessing the relative atlas graph to identify a root tile among the tiles represented by the plurality of tile nodes that is proximate the location of the autonomous vehicle;

identifying one or more additional tiles proximate to the root tile by accessing one or more edges among the third plurality of edges;

extracting a portion of the relative atlas graph, the portion including a root tile node for the root tile, one or more additional tile nodes for the one or more additional tiles, a subset of element nodes from among the plurality of element nodes, each element node is the subset connected by an edge from the second plurality of edges to the root tile or one of the one or more additional tile nodes, and each edge among the first, second and third pluralities of edges connected to any of the root tile node, one or more additional tile nodes, and subset of element nodes; and

communicating the portion of the relative atlas graph to the autonomous vehicle.

51. A method of importing external data into a relative atlas graph used by an autonomous vehicle, the relative atlas graph including a plurality of element nodes for a plurality of elements, a first plurality of edges, each edge among the plurality of edges connecting a pair of element nodes from among the plurality of element nodes and defining a relative pose between the associated elements for the pair of element nodes, a plurality of tile nodes for a geographical area, each tile node representing a tile corresponding to a region of the geographical area, and a second plurality of edges, each edge among the second plurality of edges connecting an element node from among the plurality of element nodes to a tile node from among the plurality of tile nodes and defining a relative pose of the associated element for the connected element node within a frame of reference for the connected tile node, the method comprising:

converting the external data to a fixed frame of reference;

identifying one or more tile nodes from among the plurality of tile nodes in the relative atlas graph that represent one or more tiles proximate the external data in the fixed frame of reference;

projecting the external data into a frame of reference for each of the one or more tiles;

generating one or more element nodes for one or more elements represented by the external data;

generating one or more edges connecting the one or more element nodes to the one or more tile nodes, each of the one or more edges defining a relative pose for an element among the one or more elements in a frame of reference for a tile among the one or more tiles; and

adding the generated one or more element nodes and one or more edges to the relative atlas graph.