Method and device for generating test cases for autonomous vehicles

• US 10,255,168 B2
• Filed: 11/01/2016
• Issued: 04/09/2019
• Est. Priority Date: 11/06/2015
• Status: Active Grant

• Current Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Original Assignee: Ford Global Technologies, LLC (Ford Motor Company)
• Inventors: Stefan, Frederic, Chevalier, Alain Marie Roger, Marbaix, Michael, Bitsanis, Evangelos
• Primary Examiner(s): Duncan, Marc
• Application Number:
  US15/340,856
• Publication Number:
  US 20170132117A1
• Time in Patent Office: 889 Days
• CPC Class Codes: G06F 11/3604 Software analysis for verifying properties of programs byte-code verification G06F9/44589
• CPC Class Codes: G06F 11/3684 for test design, e.g. generating new test cases

Abstract

Test cases for autonomous vehicles are generated automatically by using data which have been collected from vehicles participating in public road traffic. A test planning system for autonomous vehicle includes defined application cases for autonomous vehicles. The vehicles are configured to identify test cases with prediction analyzes of a reference catalog of driving situations and the defined application cases, and compare, via a comparative analyzes, the test cases and the defined application cases to compile an expanded set of test cases, wherein the expanded set of test cases are compared to the defined application cases to output a complete set of test cases. The system also includes a central database configured to query the complete set of test cases.

Claims

1. A method for testing autonomous vehicles comprising:

collecting data from vehicles participating in public road traffic;

analyzing the data to identify a plurality of classifications representative of driving situations and a frequency of occurrence of each of the driving situations;

generating a first set of real-world test cases from the analyzed data;

generating a second set of test cases using predefined system application cases;

performing a comparative analysis between the first set of real-world test cases and the second set of test cases to identify at least one test case from the first set that is comparable with at least one test case from the second set; and

merging the a least one test case from the first set with its comparable at least one test case from the second set to form a combined test case.

2. The method as claimed in claim 1, further comprising:

generating a third set of test cases using safety- and reliability-related application cases; and

performing a second comparative analysis between the third set and a combined set made up of a plurality of combined test cases to identify comparable members thereof; and

merging the comparable test case members identified to form a further combined set of test cases.

3. The method as claimed in claim 1, wherein the collected data from the vehicles participating in public road traffic represents a real driver experience of drivers of the vehicles.

4. The method as claimed in claim 1 further comprising identifying average test steps and acceptance criteria for all test cases based on real world scenarios by analyzing the collected data.

5. The method as claimed in claim 1, wherein, during the merging, the single case based on the comparable test cases is preserved and modified on the basis of an application case identified from data from real world scenarios.

6. The method as claimed in claim 1 further comprising storing the test cases for test documentation and conduct, wherein the test cases are obtained automatically at a system level.

7. The method as claimed in claim 1 further comprising:

applying a rule for continuous integration to update the test cases from real-world scenarios; and

outputting a signal when new test cases from the real-world scenarios are identified.

8. A tool for project life cycle management of autonomous vehicles:

a test planning system having continuous integration rules and including defined application cases for autonomous vehicles configured to identify test cases with prediction analyses of a reference catalogue of driving situations and the defined application cases, and compare, via a comparative analyses, the test cases and the defined application cases to merge comparable cases with predetermined rules to compile an expanded set of test cases, wherein the expanded set of test cases are compared to the defined application cases to output a complete set of test cases; and

a central database configured to query the test planning system.

9. The tool as claimed in claim 8, wherein the expanded set of test cases comprises a single case.

10. The tool as claimed in claim 8, wherein the continuous integration rules are configured to regularly update the test cases.

11. The tool as claimed in claim 8, wherein the defined application cases include system and safety and reliability application cases.

12. The tool as claimed in claim 8, wherein the reference catalogue of driving situations includes data from real world scenarios.

13. A test planning system for autonomous vehicle comprising:

defined application cases for autonomous vehicles configured to identify test cases with prediction analyses of a reference catalogue of driving situations and the defined application cases, and compare, via a comparative analyses, the test cases and the defined application cases to compile an expanded set of test cases, wherein the expanded set of test cases are compared to the defined application cases to output a complete set of test cases; and

a central database configured to query the complete set of test cases.

14. The test planning system as claimed in claim 13 further comprising continuous integrations rules configured to regularly update the test case.

15. The test planning system as claimed in claim 13, wherein the comparative analyses is further configure to merge comparable cases using predetermined rules such that the expanded set of test cases defines a single case.

16. The test planning system as claimed in claim 13, wherein the reference catalogue of driving situations includes data from real world scenarios.

17. The test planning system as claimed in claim 13, wherein the complete set of test cases is a test scenario to be defined in order to achieve a representative set of test cases.