Self-diagnosis of faults with a secondary system in an autonomous driving system

• US 10,553,044 B2
• Filed: 01/31/2018
• Issued: 02/04/2020
• Est. Priority Date: 01/31/2018
• Status: Active Grant

• Current Assignee: Siemens Electronic Design Automation GmbH
• Original Assignee: Mentor Graphics Development (Deutschland) GmbH (Siemens AG)
• Inventors: Mercep, Ljubo, Pollach, Matthias
• Primary Examiner(s): Shafi, Muhammad
• Application Number:
  US15/885,023
• Publication Number:
  US 20190236862A1
• Time in Patent Office: 734 Days
• Field of Search: 701 291
• CPC Class Codes: B60W 2050/0215 Sensor drifts or sensor failures
• CPC Class Codes: B60W 2520/10 Longitudinal speed
• CPC Class Codes: B60W 2554/00 Input parameters relating to objects
• CPC Class Codes: B60W 30/09 Taking automatic action to avoid collision, e.g. braking and steering
• CPC Class Codes: B60W 30/0956 the prediction being responsive to traffic or environmental parameters
• CPC Class Codes: B60W 40/04 Traffic conditions
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G06F 18/251 of input or preprocessed data
• CPC Class Codes: G06V 10/803 of input or preprocessed data
• CPC Class Codes: G06V 10/993 Evaluation of the quality of the acquired pattern
• CPC Class Codes: G06V 20/58 Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
• CPC Class Codes: G07C 5/0808 Diagnosing performance data testing of vehicles G01M17/00; testing of electrical installation on vehicles G01R31/005
• CPC Class Codes: G07C 5/0816 Indicating performance data, e.g. occurrence of a malfunction
• CPC Class Codes: G08G 1/16 Anti-collision systems road vehicle drive control systems for predicting or avoiding probable or impending collision otherwise than by control of a particular sub-unit B60W30/08
• CPC Class Codes: G08G 1/165 for passive traffic, e.g. including static obstacles, trees
• CPC Class Codes: G08G 1/166 for active traffic, e.g. moving vehicles, pedestrians, bikes

Abstract

This application discloses self-diagnosis of faults for an assisted or automated driving system of a vehicle. A primary computing system can generate an environmental model populated with objects detected from measurement data from sensors. A secondary computing system can generate a secondary system data structure configured to identify objects located around the vehicle that were detected from measurement data from the sensors. The secondary computing system can identify a fault in the sensors based on a comparison of the secondary system data structure with the environmental model. The secondary computing system also can estimate an amount of time before the vehicle crashes from a vehicle velocity vector and free space in the environment around the vehicle, and identify the fault in the sensors based on the estimated amount of time before the vehicle crashes.

Claims

1. A method comprising:

generating, by a sensor fusion computing system, an environmental model populated with objects detected from measurement data collected by sensors mounted in a vehicle;

generating, by a self-diagnosis computing system, a secondary system data structure based, at least in part, on measurement data collected by the sensors, wherein the secondary system data structure is configured to identify objects located around the vehicle that were detected from the measurement data collected by the sensors;

identifying, by the self-diagnosis computing system, a fault in the sensors based, at least in part, on a comparison of the secondary system data structure to the environmental model;

determining, by the self-diagnosis computing system, a velocity vector for the vehicle based on the measurement data collected by the sensors;

identifying, by the self-diagnosis computing system, free space in the environment around the vehicle based on locations of the detected objects in the secondary system data structure relative to the vehicle; and

estimating, by the self-diagnosis computing system, an amount of time before the vehicle crashes based, at least in part, on the velocity vector and the free space in the environment around the vehicle, wherein identifying the fault in the sensors is based, at least in part, on the estimated amount of time before the vehicle crashes.

2. The method of claim 1, wherein identifying the fault in the sensors is performed when locations of the measurement data or the detected objects in the environmental model differ from locations of the measurement data or the detected objects in the secondary system data structure.

3. The method of claim 1, wherein generating the secondary system data structure further comprises tracking movement of the detected objects, and wherein estimating the amount of time before the vehicle crashes is based, at least in part, on the velocity vector, an expected driving corridor for the vehicle, the free space in the environment around the vehicle, and the tracked movement of the detected objects.

4. The method of claim 1, wherein the velocity vector includes an estimate of a speed of the vehicle and a directionality of motion of the vehicle motion based, at least in part, on the measurement data collected by the sensors.

5. The method of claim 1, wherein identifying the fault in the sensors is performed when the estimated amount of time before the vehicle crashes falls below a preset threshold level of time.

6. The method of claim 1, wherein the secondary system data structure is configured to identify a physical border of the vehicle and locations of the detected objects, and wherein identifying the free space in the environment around the vehicle is performed by determining a distance between the detected objects in the secondary system data structure and the physical border of the vehicle in the secondary system data structure.

7. An apparatus comprising at least one memory device storing instructions configured to cause one or more processing devices to perform operations comprising:

generating a secondary system data structure configured to identify objects located around a vehicle that were detected from measurement data collected by sensors mounted in the vehicle;

comparing the objects detected in the secondary system data structure to an environmental model populated with objects detected from measurement data collected by the sensors;

identifying a fault in the sensors based, at least in part, on the comparison of the secondary system data structure to the environmental model;

determining a velocity vector for the vehicle based on the measurement data collected by the sensors;

identifying free space in the environment around the vehicle based on locations of the detected objects in the secondary system data structure relative to the vehicle; and

estimating an amount of time before the vehicle crashes based, at least in part, on the velocity vector and the free space in the environment around the vehicle, wherein identifying the fault in the sensors is based, at least in part, on the estimated amount of time before the vehicle crashes.

8. The apparatus of claim 7, wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising identifying the fault in the sensors when locations of the measurement data or the detected objects in the environmental model differ from locations of the measurement data or the detected objects in the secondary system data structure.

9. The apparatus of claim 7, wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising:

tracking movement of the detected objects; and

estimating the amount of time before the vehicle crashes based, at least in part, on the velocity vector, an expected driving corridor for the vehicle, the free space in the environment around the vehicle, and the tracked movement of the detected objects.

10. The apparatus of claim 7, wherein the velocity vector includes an estimate of a speed of the vehicle and a directionality of motion of the vehicle motion based, at least in part, on the measurement data collected by the sensors.

11. The apparatus of claim 7, wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising identifying the fault in the sensors when the estimated amount of time before the vehicle crashes falls below a preset threshold level of time.

12. The apparatus of claim 7, wherein the secondary system data structure is configured to identify a physical border of the vehicle and locations of the detected objects, and wherein the instructions are further configured to cause the one or more processing devices to perform operations comprising identifying the free space in the environment around the vehicle by determining a distance between the detected objects in the secondary system data structure and the physical border of the vehicle in the secondary system data structure.

13. A system comprising:

a memory device configured to store machine-readable instructions; and

a computing system including one or more processing devices, in response to executing the machine-readable instructions, configured to;

generate a secondary system data structure configured to identify objects located around a vehicle that were detected from measurement data collected by sensors mounted in the vehicle;

compare the objects detected in the secondary system data structure to an environmental model populated with objects detected from measurement data collected by the sensors;

identify a fault in the sensors based, at least in part, on the comparison of the secondary system data structure to the environmental model;

determine a velocity vector for the vehicle based on the measurement data collected by the sensors;

identify free space in the environment around the vehicle based on locations of the detected objects in the secondary system data structure relative to the vehicle; and

estimate an amount of time before the vehicle crashes based, at least in part, on the velocity vector and the free space in the environment around the vehicle, wherein the fault in the sensors is identified based, at least in part, on the estimated amount of time before the vehicle crashes.

14. The system of claim 13, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify the fault in the sensors when locations of the measurement data or the detected objects in the environmental model differ from locations of the measurement data or the detected objects in the secondary system data structure.

15. The system of claim 13, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to:

track movement of the detected objects; and

estimate the amount of time before the vehicle crashes based, at least in part, on the velocity vector, an expected driving corridor for the vehicle, the free space in the environment around the vehicle, and the tracked movement of the detected objects.

16. The system of claim 13, wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify the fault in the sensors when the estimated amount of time before the vehicle crashes falls below a preset threshold level of time.

17. The system of claim 13, wherein the secondary system data structure is configured to identify a physical border of the vehicle and locations of the detected objects, and wherein the one or more processing devices, in response to executing the machine-readable instructions, are configured to identify the free space in the environment around the vehicle by determining a distance between the detected objects in the secondary system data structure and the physical border of the vehicle in the secondary system data structure.