AUTOMATIC DATA LABELLING FOR AUTONOMOUS DRIVING VEHICLES

• US 20190317507A1
• Filed: 04/13/2018
• Published: 10/17/2019
• Est. Priority Date: 04/13/2018
• Status: Active Grant

• Current Assignee: Baidu USA LLC (Baidu Incorporated)
• Original Assignee: Baidu USA LLC (Baidu Incorporated)
• Inventors: Zhang, Liangliang, Li, Dong, Hu, Jiangtao, Tao, Jiaming, Jiang, Yifei
• Granted Patent: US 10,816,984 B2
• Time in Patent Office: Days
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0214 in accordance with safety or protection criteria, e.g. avoiding hazardous areas monitoring the location of vehicles within a certain area, e.g. forbidden or allowed areas, in traffic control systems for road vehicles G08G1/13
• CPC Class Codes: G05D 1/0238 using obstacle or wall sensors G05D1/0246 and G05D1/0289 take precedence; lidar systems designed for anti-collision purposes G01S17/93
• CPC Class Codes: G05D 1/027 comprising intertial navigation means, e.g. azimuth detector inertial navigation G01C21/16; inertial navigation combined with non-inertial navigation instruments G01C21/165
• CPC Class Codes: G05D 1/0274 using mapping information stored in a memory device navigation using map-matching G01C21/30
• CPC Class Codes: G05D 1/0278 using satellite positioning signals, e.g. GPS

Abstract

In one embodiment, sensor data are collected from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a number of obstacles. The collected sensor data are operated on to obtain obstacle data associated with the obstacles, location data, and a number of timestamps that correspond to the obstacle data and the location data. For each of the timestamps, positions of the obstacles are mapped to some of the obstacle data that correspond to the timestamp based on the location data, thereby generating mapped information of the obstacles. The mapped information is automatically labelled to generate labelled data, where the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of an ADV.

Claims

1. A computer-implemented method for automatic generation of labelled data, comprising:

collecting sensor data from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a plurality of obstacles;

operating on the collected sensor data to obtain obstacle data associated with the obstacles, location data, and a plurality of timestamps that correspond to the obstacle data and the location data;

for each of the timestamps, mapping positions of the obstacles to some of the obstacle data that correspond to the timestamp based on the location data, thereby generating mapped information of the obstacles; and

automatically labelling the mapped information to generate labelled data, wherein the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of an ADV.

2. The method of claim 1, further comprising prior to collecting the sensor data from the one or more sensors mounted on the ADV while the ADV is moving within the ROI, pre-marking the ROI and the plurality of obstacles from an initial position of the ADV to generate pre-marked data;

wherein mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp is further based on the pre-marked data.

3. The method of claim 1, further comprising prior to mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp, aligning the plurality of timestamps and the positions of the obstacles.

4. The method of claim 1, wherein the obstacle data comprises light detection and range (LIDAR) data captured at different points in time that are included in the timestamps.

5. The method of claim 4, wherein the location data comprises the positions of the obstacles captured at the different points in time.

6. The method of claim 5, wherein the location data is obtained from an inertial measurement unit (IMU) or a global positioning system (GPS).

7. The method of claim 2, wherein the pre-marked data comprises boundary information of the ROI, and initial position and boundary information of the plurality of obstacles within the ROI.

8. A non-transitory machine-readable medium having instructions stored therein, which when executed by a processor, cause the processor to perform operations, the operations comprising:

collecting sensor data from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a plurality of obstacles;

operating on the collected sensor data to obtain obstacle data associated with the obstacles, location data, and a plurality of timestamps that correspond to the obstacle data and the location data;

for each of the timestamps, mapping positions of the obstacles to some of the obstacle data that correspond to the timestamp based on the location data, thereby generating mapped information of the obstacles; and

automatically labelling the mapped information to generate labelled data, wherein the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of an ADV.

9. The non-transitory machine-readable medium of claim 8,

wherein the operations further comprise:

prior to collecting the sensor data from the one or more sensors mounted on the ADV while the ADV is moving within the ROI, pre-marking the ROI and the plurality of obstacles from an initial position of the ADV to generate pre-marked data;

wherein mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp is further based on the pre-marked data.

10. The non-transitory machine-readable medium of claim 8, wherein the operations further comprise:

prior to mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp, aligning the plurality of timestamps and the positions of the obstacles.

11. The non-transitory machine-readable medium of claim 8, wherein the obstacle data comprises light detection and range (LIDAR) data captured at different points in time that are included in the timestamps.

12. The non-transitory machine-readable medium of claim 11, wherein the location data comprises the positions of the obstacles captured at the different points in time.

13. The non-transitory machine-readable medium of claim 12, wherein the location data is obtained from an inertial measurement unit (IMU) or a global positioning system (GPS).

14. The non-transitory machine-readable medium of claim 9, wherein the pre-marked data comprises boundary information of the ROI, and initial position and boundary information of the plurality of obstacles within the ROI.

15. A data processing system, comprising:

a processor; and

a memory coupled to the processor to store instructions, which when executed by the processor, cause the processor to perform operations, the operations including

collecting sensor data from one or more sensors mounted on an autonomous driving vehicle (ADV) while the ADV is moving within a region of interest (ROI) that includes a plurality of obstacles;

operating on the collected sensor data to obtain obstacle data associated with the obstacles, location data, and a plurality of timestamps that correspond to the obstacle data and the location data;

for each of the timestamps, mapping positions of the obstacles to some of the obstacle data that correspond to the timestamp based on the location data, thereby generating mapped information of the obstacles; and

automatically labelling the mapped information to generate labelled data, wherein the labelled data is utilized to subsequently train a machine learning algorithm to recognize obstacles during autonomous driving of an ADV.

16. The data processing system of claim 15,

wherein the operations further include:

prior to collecting the sensor data from the one or more sensors mounted on the ADV while the ADV is moving within the ROI, pre-marking the ROI and the plurality of obstacles from an initial position of the ADV to generate pre-marked data;

wherein mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp is further based on the pre-marked data.

17. The data processing system of claim 15, wherein the operations further include:

prior to mapping the positions of the obstacles to some of the obstacle data that correspond to the timestamp, aligning the plurality of timestamps and the positions of the obstacles.

18. The data processing system of claim 15, wherein the obstacle data comprises light detection and range (LIDAR) data captured at different points in time that are included in the timestamps.

19. The data processing system of claim 18, wherein the location data comprises the positions of the obstacles captured at the different points in time.

20. The data processing system of claim 19, wherein the location data is obtained from an inertial measurement unit (IMU) or a global positioning system (GPS).

21. The data processing system of claim 16, wherein the pre-marked data comprises boundary information of the ROI, and initial position and boundary information of the plurality of obstacles within the ROI.