Perception simulation for improved autonomous vehicle control

• US 10,830,669 B2
• Filed: 10/22/2019
• Issued: 11/10/2020
• Est. Priority Date: 05/18/2017
• Status: Active Grant

• Current Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Original Assignee: TuSimple, Inc. (TuSimple Holdings, Inc.)
• Inventors: Sun, Xing, Lin, Wutu, Zhao, Yufei, Liu, Liu
• Primary Examiner(s): Dager, Jonathan M
• Application Number:
  US16/660,244
• Publication Number:
  US 20200049592A1
• Time in Patent Office: 385 Days
• Field of Search: None
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: B60W 50/0205 Diagnosing or detecting failures; Failure detection models
• CPC Class Codes: G01M 17/00 Testing of vehicles testing fluid tightness G01M3/00; testing elastic properties of bodies or chassis, e.g. torsion-testing, G01M5/00; testing alignment of vehicle headlight devices G01M11/06; testing of engines G01M15/00

Abstract

A system and method for real world autonomous vehicle perception simulation are disclosed. A particular embodiment includes: receiving perception data from a plurality of sensors of an autonomous vehicle; configuring the perception simulation operation based on a comparison of the perception data against ground truth data; generating simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors; and providing the simulated perception data to a motion planning system for the autonomous vehicle.

Claims

1. A system comprising:

a data processor; and

a perception simulation module, executable by the data processor, the perception simulation module being configured to perform a perception simulation operation for autonomous vehicles, the perception simulation operation being configured to;

receive perception data from a plurality of sensors of an autonomous vehicle;

configure the perception simulation operation based on a comparison of the perception data against ground truth data;

generate simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors, wherein simulating errors related to the physical constraints of one or more of the plurality of sensors includes applying a maximum range beyond which the one or more of the plurality of sensors cannot detect objects; and

provide the simulated perception data to a motion planning system for the autonomous vehicle.

2. The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating a false positive or false negative object detection.

3. The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating noise in vehicle localization.

4. The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating noise in lane detection.

5. The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in object type classification.

6. The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in object location.

7. The system of claim 1 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in data produced by an image processing module.

8. A method comprising:

receiving perception data from a plurality of sensors of an autonomous vehicle;

configuring the perception simulation operation based on a comparison of the perception data against ground truth data;

generating simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors, wherein simulating errors related to the physical constraints of one or more of the plurality of sensors includes applying a maximum range beyond which the one or more of the plurality of sensors cannot detect objects; and

providing the simulated perception data to a motion planning system for the autonomous vehicle.

9. The method of claim 8 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating a false positive or false negative object detection.

10. The method of claim 8 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating noise in vehicle localization.

11. The method of claim 8 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating noise in lane detection.

12. The method of claim 8 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in object type classification.

13. The method of claim 8 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in object location.

14. The method of claim 8 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in data produced by an image processing module.

15. A non-transitory machine-useable storage medium embodying instructions which, when executed by a machine, cause the machine to:

receive perception data from a plurality of sensors of an autonomous vehicle;

configure the perception simulation operation based on a comparison of the perception data against ground truth data;

generate simulated perception data by simulating errors related to the physical constraints of one or more of the plurality of sensors, and by simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors, wherein simulating errors related to the physical constraints of one or more of the plurality of sensors includes applying a maximum range beyond which the one or more of the plurality of sensors cannot detect objects; and

provide the simulated perception data to a motion planning system for the autonomous vehicle.

16. The non-transitory machine-useable storage medium of claim 15 wherein the ground truth data is produced by a ground truth data simulation system.

17. The non-transitory machine-useable storage medium of claim 15 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating a false positive or false negative object detection.

18. The non-transitory machine-useable storage medium of claim 15 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating noise in vehicle localization.

19. The non-transitory machine-useable storage medium of claim 15 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating noise in lane detection.

20. The non-transitory machine-useable storage medium of claim 15 wherein simulating noise in data provided by a sensor processing module corresponding to one or more of the plurality of sensors includes simulating errors in object type classification.