High definition map and route storage management system for autonomous vehicles

• US 10,572,514 B2
• Filed: 05/09/2019
• Issued: 02/25/2020
• Est. Priority Date: 12/30/2016
• Status: Active Grant

• Current Assignee: NVIDIA Corporation
• Original Assignee: DeepMap, Inc. (NVIDIA Corporation)
• Inventors: Wheeler, Mark Damon
• Primary Examiner(s): Black, Thomas G
• Assistant Examiner(s): Smith-Stewart, Demetra R
• Application Number:
  US16/408,270
• Publication Number:
  US 20190266179A1
• Time in Patent Office: 292 Days
• Field of Search: None
• CPC Class Codes: G01C 21/3881 Tile-based structures
• CPC Class Codes: G01C 21/3889 Transmission of selected map data, e.g. depending on route
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0212 with means for defining a desired trajectory involving a plurality of land vehicles G05D1/0287
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G06F 12/0811 with multilevel cache hierarchies
• CPC Class Codes: G06F 12/0875 with dedicated cache, e.g. instruction or stack
• CPC Class Codes: G06F 16/29 Geographical information databases
• CPC Class Codes: G06F 2212/1016 Performance improvement
• CPC Class Codes: G06F 2212/1024 Latency reduction
• CPC Class Codes: G06F 2212/283 Plural cache memories
• CPC Class Codes: G06F 9/54 Interprogram communication

Abstract

High definition maps for autonomous vehicles are very high resolution and detailed, and hence require storage of a great deal of data. A vehicle computing system provides multi-layered caching makes this data usable in a system that requires very low latency on every operation. The system determines which routes are most likely to be driven in the near future by the car, and ensures that the route is cached on the vehicle before beginning the route. The system provides efficient formats for moving map data from server to car and for managing the on-car disk. The system further provides real-time accessibility of nearby map data as the car moves, while providing data access at optimal speeds.

Claims

1. A method of caching high-definition (HD) map data by a vehicle comprising:

storing, by a vehicle, a plurality of map tiles, each map tile representing a portion of the HD map data;

identifying a route to be travelled by the vehicle to reach a destination;

receiving sensor data from one or more sensors of the vehicle;

performing localization to determine a first location of the vehicle based on the received sensor data;

identifying a first map tile of the HD map, the first map tile representing a geographical region including the current location of the vehicle;

loading the first map tile in a random-access memory (RAM), wherein the RAM stores map tiles for utilization in driving the vehicle along the route;

predicting movement of the vehicle within a threshold time interval from the first location;

determining a first subset of map tiles based on a combination of the first location and the predicted movement, each map tile of the first subset of map tiles corresponding to a portion of a route that the vehicle is likely to drive through within the threshold time interval;

loading the first subset of map tiles in the RAM; and

navigating the vehicle along the route by accessing the first subset of map tiles loaded in the RAM.

2. The method of claim 1, further comprising:

receiving the plurality of map tiles from an online system.

3. The method of claim 1, wherein the sensors comprise a detection and ranging sensor mounted on the vehicle, and wherein the sensor data collected by the detection and ranging sensor comprises a depth map of an environment surrounding the vehicle.

4. The method of claim 1, wherein the sensors comprise a Global Positioning System (GPS) receiver, and wherein the sensor data collected by the GPS receiver comprises geolocation coordinates of the vehicle.

5. The method of claim 1, wherein the sensors comprise a camera, and wherein the sensor data collected by the camera comprises images of an environment surrounding the vehicle.

6. The method of claim 1, wherein the sensors comprise an inertial measurement unit (IMU), and wherein the sensor data collected by the IMU comprises motion data of the vehicle.

7. The method of claim 1, wherein performing localization further determines a velocity of the vehicle, and wherein the predicted movement is based on the velocity of the vehicle.

8. The method of claim 1, wherein determining the first subset of map tiles based on the combination of the first location and the predicted movement further comprises:

identifying a first map tile in which the vehicle is currently located based on the first location of the vehicle; and

determining one or more other map tiles adjacent to the first map tile based on the predicted movement, wherein the first subset of map tiles is inclusive of the first map tile and the other map tiles.

9. The method of claim 8, wherein determining the first subset of map tiles based on the combination of the first location and the predicted movement further comprises:

identifying a lane within the first map tile, wherein determining the other map tiles adjacent to the first map tile is further based the identified lane.

10. The method of claim 1, further comprising:

performing localization to determine a second location of the vehicle at a second time based on the received sensor data;

predicting subsequent movement of the vehicle within the threshold time interval from the second location;

determining a second subset of map tiles based on a combination of the second location and the predicted subsequent movement, each map tile of the second subset of map tiles corresponding to a subsequent portion of the route that the vehicle is likely to drive through within the threshold time interval;

loading the second subset of map tiles in the RAM; and

navigating the vehicle along the route by accessing the second subset of map tiles loaded in the RAM.

11. A non-transitory computer-readable storage medium with encoded instructions that, when executed by a processor, cause the processor to perform steps of:

storing, by a vehicle, a plurality of map tiles, each map tile representing a portion of the HD map data;

identifying a route to be travelled by the vehicle to reach a destination;

receiving sensor data from one or more sensors of the vehicle;

performing localization to determine a first location of the vehicle based on the received sensor data;

identifying a first map tile of the HD map, the first map tile representing a geographical region including the current location of the vehicle;

loading the first map tile in a random-access memory (RAM), wherein the RAM stores map tiles for utilization in driving the vehicle along the route;

predicting movement of the vehicle within a threshold time interval from the first location;

determining a first subset of map tiles based on a combination of the first location and the predicted movement, each map tile of the first subset of map tiles corresponding to a portion of a route that the vehicle is likely to drive through within the threshold time interval;

loading the first subset of map tiles in the RAM; and

navigating the vehicle along the route by accessing the first subset of map tiles loaded in the RAM.

12. The storage medium of claim 11, the steps further comprising:

receiving the plurality of map tiles from an online system.

13. The storage medium of claim 11, wherein the sensors comprise a detection and ranging sensor mounted on the vehicle, and wherein the sensor data collected by the detection and ranging sensor comprises a depth map of an environment surrounding the vehicle.

14. The storage medium of claim 11, wherein the sensors comprise a Global Positioning System (GPS) receiver, and wherein the sensor data collected by the GPS receiver comprises geolocation coordinates of the vehicle.

15. The storage medium of claim 11, wherein the sensors comprise a camera, and wherein the sensor data collected by the camera comprises images of an environment surrounding the vehicle.

16. The storage medium of claim 11, wherein the sensors comprise an inertial measurement unit (IMU), and wherein the sensor data collected by the IMU comprises motion data of the vehicle.

17. The storage medium of claim 11, wherein performing localization further determines a velocity of the vehicle, and wherein the predicted movement is based on the velocity of the vehicle.

18. The storage medium of claim 11, wherein determining the first subset of map tiles based on the combination of the first location and the predicted movement further comprises:

identifying a first map tile in which the vehicle is currently located based on the first location of the vehicle; and

determining one or more other map tiles adjacent to the first map tile based on the predicted movement, wherein the first subset of map tiles is inclusive of the first map tile and the other map tiles.

19. The storage medium of claim 18, wherein determining the first subset of map tiles based on the combination of the first location and the predicted movement further comprises:

identifying a lane within the first map tile, wherein determining the other map tiles adjacent to the first map tile is further based the identified lane.

20. The storage medium of claim 11, the steps further comprising:

performing localization to determine a second location of the vehicle at a second time based on the received sensor data;

predicting subsequent movement of the vehicle within the threshold time interval from the second location;

determining a second subset of map tiles based on a combination of the second location and the predicted subsequent movement, each map tile of the second subset of map tiles corresponding to a subsequent portion of the route that the vehicle is likely to drive through within the threshold time interval;

loading the second subset of map tiles in the RAM; and

navigating the vehicle along the route by accessing the second subset of map tiles loaded in the RAM.

21. A system on a vehicle comprising:

a processor; and

a computer-readable storage medium with encoded instructions that, when executed by the processor, cause the processor to perform steps of;

storing a plurality of map tiles, each map tile representing a portion of the HD map data;

identifying a route to be travelled by the vehicle to reach a destination;

receiving sensor data from one or more sensors of the vehicle;

performing localization to determine a first location of the vehicle based on the received sensor data;

identifying a first map tile of the HD map, the first map tile representing a geographical region including the current location of the vehicle;

loading the first map tile in a random-access memory (RAM), wherein the RAM stores map tiles for utilization in driving the vehicle along the route;

predicting movement of the vehicle within a threshold time interval from the first location;

determining a first subset of map tiles based on a combination of the first location and the predicted movement, each map tile of the first subset of map tiles corresponding to a portion of a route that the vehicle is likely to drive through within the threshold time interval;

loading the first subset of map tiles in the RAM; and

navigating the vehicle along the route by accessing the first subset of map tiles loaded in the RAM.

22. The system of claim 21, the steps further comprising:

receiving the plurality of map tiles from an online system.

23. The system of claim 21, wherein performing localization further determines a velocity of the vehicle, and wherein the predicted movement is based on the velocity of the vehicle.

24. The storage medium of claim 21, wherein determining the first subset of map tiles based on the combination of the first location and the predicted movement further comprises:

identifying a first map tile in which the vehicle is currently located based on the first location of the vehicle; and

determining one or more other map tiles adjacent to the first map tile based on the predicted movement, wherein the first subset of map tiles is inclusive of the first map tile and the other map tiles.

25. The storage medium of claim 21, the steps further comprising:

performing localization to determine a second location of the vehicle at a second time based on the received sensor data;

predicting subsequent movement of the vehicle within the threshold time interval from the second location;

determining a second subset of map tiles based on a combination of the second location and the predicted subsequent movement, each map tile of the second subset of map tiles corresponding to a subsequent portion of the route that the vehicle is likely to drive through within the threshold time interval;

loading the second subset of map tiles in the RAM; and

navigating the vehicle along the route by accessing the second subset of map tiles loaded in the RAM.