AUTOMATIC GENERATION OF DIMENSIONALLY REDUCED MAPS AND SPATIOTEMPORAL LOCALIZATION FOR NAVIGATION OF A VEHICLE

• US 20200133272A1
• Filed: 10/18/2019
• Published: 04/30/2020
• Est. Priority Date: 10/29/2018
• Status: Active Application

• Current Assignee: Motional AD, LLC (Aptiv PLC)
• Original Assignee: Aptiv Technologies Limited (BorgWarner Incorporated)
• Inventors: Chong, Zhuang Jie
• Application Number:
  US16/656,948
• Publication Number:
  US 20200133272A1
• Time in Patent Office: Days
• CPC Class Codes: G01C 21/3602 Input other than that of destination using image analysis, e.g. detection of road signs, lanes, buildings, real preceding vehicles using a camera
• CPC Class Codes: G01C 21/3848 Data obtained from both position sensors and additional sensors
• CPC Class Codes: G01C 21/387 Organisation of map data, e.g. version management or database structures
• CPC Class Codes: G01S 17/89 for mapping or imaging
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G07C 5/02 Registering or indicating driving, working, idle, or waiting time only apparatus forming part of taximeters G07B13/00

Abstract

Among other things, we describe techniques for generation of dimensionally reduced maps and localization in an environment for navigation of vehicles. The techniques include generating, using one or more sensors of a vehicle located at a spatiotemporal location within an environment, M-dimensional sensor data representing the environment, wherein M is greater than 2. Odometry data is generated representing an operational state of the vehicle. The odometry data is associated with the spatiotemporal location. An N-dimensional map is generated of the environment from the M-dimensional sensor data, wherein N is less than M. The generating of the N-dimensional map comprises extracting an M-dimensional environmental feature of the spatiotemporal location from the M-dimensional sensor data. The M-dimensional environmental feature is associated with the odometry data. An N-dimensional version of the M-dimensional environmental feature is generated. The N-dimensional version of the M-dimensional environmental feature is embedded within the N-dimensional map.

Claims

1. A method comprising:

generating, using one or more sensors of a vehicle located at a spatiotemporal location within an environment, M-dimensional sensor data representing the environment, wherein M is greater than 2;

generating, using the one or more sensors, odometry data representing an operational state of the vehicle, the odometry data associated with the spatiotemporal location; and

generating, using one or more processors of the vehicle, an N-dimensional map of the environment based on the M-dimensional sensor data, wherein N is less than M, the generating of the N-dimensional map comprising;

extracting, using the one or more processors, an M-dimensional environmental feature based on the M-dimensional sensor data;

associating, using the one or more processors, the M-dimensional environmental feature with the odometry data;

generating, using the one or more processors, an N-dimensional version of the M-dimensional environmental feature; and

embedding, using the one or more processors, the N-dimensional version of the M-dimensional environmental feature within the N-dimensional map.

2. The method of claim 1, wherein the embedding of the N-dimensional version of the M-dimensional environmental feature comprises embedding, using the one or more processors, a computer-readable semantic annotation corresponding to the N-dimensional version of the M-dimensional environmental feature within the N-dimensional map.

3. The method of claim 1, wherein the generating of the N-dimensional version of the M-dimensional environmental feature comprises mapping, using the one or more processors, the three-dimensional LiDAR point cloud data to a two-dimensional image, the M-dimensional sensor data comprising three-dimensional LiDAR point cloud data.

4. The method of any of claim 1, wherein the M-dimensional sensor data comprises (M-1)-dimensional LiDAR point cloud data indexed in accordance with time, the N-dimensional map comprising (N-1)-dimensional image data indexed in accordance with time.

5. The method of claim 1, further comprising mapping, using the one or more processors, the M-dimensional sensor data to a plurality of planes, each plane of the plurality of planes corresponding to a distinct N-dimensional map.

6. The method of claim 1, further comprising determining, using the one or more processors, location coordinates of the vehicle based on the odometry data, the location coordinates being relative to the N-dimensional map.

7. The method of claim 6, further comprising transmitting, using the one or more processors, the determined location coordinates to at least one of a remote server or another vehicle to provide navigation assistance to the other vehicle.

8. The method of claim 1, further comprising determining, using the one or more processors, a directional orientation of the vehicle based on the odometry data, the directional orientation being relative to the N-dimensional map.

9. The method of claim 1, wherein the one or more sensors comprise a controller area network (CAN) bus of the vehicle, the odometry data generated using the CAN bus.

10. The method of claim 1, wherein the one or more sensors comprise an inertial measurement unit (IMU) of the vehicle, the odometry data generated using the IMU.

11. The method of claim 1, wherein the odometry data comprises at least one of a speed, a steering angle, a longitudinal acceleration, or a lateral acceleration.

12. The method of claim 1, wherein the extracting of the M-dimensional environmental feature comprises:

generating, using the one or more processors, a plurality of pixels based on the M- dimensional sensor data, a number of the plurality of pixels corresponding to a number of LiDAR beams of the one or more sensors; and

analyzing, using the one or more processors, a depth difference between a first pixel of the plurality of pixels and neighboring pixels of the first pixel to extract the M-dimensional environmental feature.

13. The method of claim 1, wherein the extracting of the M-dimensional environmental feature comprises:

generating, using the one or more processors, a plurality of pixels based on the M- dimensional sensor data; and

extracting, using the one or more processors, the M-dimensional environmental feature from the plurality of pixels responsive to a depth difference between a first pixel of the plurality of pixels and neighboring pixels of the first pixel being less than a threshold.

14. The method of claim 1, wherein the associating of the M-dimensional environmental feature with the odometry data comprises at least one of correlating or probabilistic matching of the M-dimensional environmental feature to the spatiotemporal location.

15. The method of claim 1, further comprising:

generating, using the one or more sensors of the vehicle, second M-dimensional sensor data representing the environment, the vehicle located at a second spatiotemporal location within the environment; and

associating, using the one or more processors, the second M-dimensional sensor data with the second spatiotemporal location to make the second M-dimensional sensor data distinct from the M-dimensional sensor data.

16. The method of claim 1, wherein the generating of the N-dimensional version of the M-dimensional environmental feature comprises counting, using the one or more processors, instances of N-dimensional location coordinates of the M-dimensional environmental feature.

17. The method of claim 1, wherein the embedding of the N-dimensional version of the M-dimensional environmental feature comprises:

responsive to a count of instances of N-dimensional location coordinates of the M-dimensional environmental feature exceeding a threshold, embedding, using the one or more processors, the N-dimensional location coordinates of the M-dimensional environmental feature within the N-dimensional map.

18. The method of claim 1, wherein the M-dimensional environmental feature represents at least one of a road segment, a construction zone, a curb, a building, a parking space located on a road segment, a highway exit or entrance ramp, or a parking lot.

19. An autonomous vehicle comprising:

one or more computer processors; and

one or more non-transitory storage media storing instructions which, when executed by the one or more computer processors, cause the one or more computer processors to;

generate, using one or more sensors of the vehicle, M-dimensional sensor data representing an environment, wherein M is greater than 2, the vehicle located at a spatiotemporal location within the environment;

generate, using the one or more sensors, odometry data representing an operational state of the vehicle, the odometry data associated with the spatiotemporal location; and

generate an N-dimensional map of the environment based on the M-dimensional sensor data, wherein N is less than M, the generating of the N-dimensional map comprising;

extracting an M-dimensional environmental feature based on the M- dimensional sensor data;

associating the M-dimensional environmental feature with the odometry data;

generating an N-dimensional version of the M-dimensional environmental feature; and

embedding the N-dimensional version of the M-dimensional environmental feature within the N-dimensional map.

20. One or more non-transitory storage media storing instructions which, when executed by one or more computing devices, cause the one or more computing devices to:

generate, using one or more sensors of the vehicle, M-dimensional sensor data representing an environment, wherein M is greater than 2, the vehicle located at a spatiotemporal location within the environment;

generate, using the one or more sensors, odometry data representing an operational state of the vehicle, the odometry data associated with the spatiotemporal location; and

generate an N-dimensional map of the environment based on the M-dimensional sensor data, wherein N is less than M, the generating of the N-dimensional map comprising;

extracting an M-dimensional environmental feature based on the M-dimensional sensor data;

associating the M-dimensional environmental feature with the odometry data;

generating an N-dimensional version of the M-dimensional environmental feature; and

embedding the N-dimensional version of the M-dimensional environmental feature within the N-dimensional map.