Intervention in operation of a vehicle having autonomous driving capabilities

• US 10,599,141 B2
• Filed: 06/16/2017
• Issued: 03/24/2020
• Est. Priority Date: 06/16/2017
• Status: Active Grant

• Current Assignee: Motional AD, LLC (Aptiv PLC)
• Original Assignee: nuTonomy, Inc. (BorgWarner Incorporated)
• Inventors: Liu, Shih-Yuan, Ravichandran, Harshavardhan, Iagnemma, Karl, Chang, Hsun-Hsien
• Primary Examiner(s): Tissot, Adam D
• Application Number:
  US15/624,819
• Publication Number:
  US 20180364701A1
• Time in Patent Office: 1,012 Days
• Field of Search: None
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0816 Indicating performance data, e.g. occurrence of a malfunction

Abstract

Among other things, a determination is made that intervention in an operation of one or more autonomous driving capabilities of a vehicle is appropriate. Based on the determination, a person is enabled to provide information for an intervention. The intervention is caused in the operation of the one or more autonomous driving capabilities of the vehicle.

Claims

1. A vehicle having autonomous driving capabilities and comprising:

steering, acceleration, and deceleration devices that respond to control signals from a driving control system to drive the vehicle autonomously on a road network;

a monitoring element on the vehicle that;

determines, using a model of the steering, acceleration, and deceleration devices, an estimated output of the steering, acceleration, and deceleration devices;

receives, from the steering, acceleration, and deceleration devices, a signal;

estimates a failure in the steering, acceleration, and deceleration devices based on the estimated output and the signal; and

generates an intervention request for the vehicle to engage in an intervention, the intervention comprising;

treating a current location of the vehicle as a non-deterministic location having a conditional probability; and

using probabilistic reasoning to identify a true geolocation of the vehicle based on the conditional probability; and

a communication element that receives the intervention to be implemented by the driving control system by issuing control signals to the steering, acceleration, and deceleration devices to cause the vehicle to maneuver from the true geolocation to a goal location.

2. The vehicle of claim 1, comprising a processor that receives information about a status or environment of the vehicle to determine that the intervention is appropriate.

3. The vehicle of claim 2, in which the status or the environment of the vehicle comprises a functionality of a hardware component or a software process of the vehicle.

4. The vehicle of claim 2, in which the information about the status or the environment of the vehicle comprises a signal from a hardware component or a software process of the vehicle.

5. The vehicle of claim 4, in which determining that intervention is appropriate comprises using pattern recognition to evaluate an abnormal pattern in the signal.

6. The vehicle of claim 5, in which the abnormal pattern is learned from a machine learning algorithm.

7. The vehicle of claim 2, in which determining that intervention is appropriate comprises detecting presence of unexpected data or absence of expected data in the received information.

8. The vehicle of claim 2, in which determining that intervention is appropriate comprises evaluating a mismatch between a measured quantity and a model-estimated quantity for a hardware component or a software process.

9. The vehicle of claim 2, in which determining that intervention is appropriate comprises inferring a malfunction in a hardware component or a software process.

10. The vehicle of claim 2, in which determining that intervention is appropriate comprises detecting an unknown object present in the environment of the vehicle.

11. The vehicle of claim 2, in which determining that intervention is appropriate comprises inferring an event that is or will be happening in the environment of the vehicle.

12. The vehicle of claim 1, in which the intervention request comprises data associated with status or environment of a vehicle.

13. The vehicle of claim 1, in which the intervention request comprises one or more signals from one or more hardware components or one or more software processes of the vehicle.

14. The vehicle of claim 1, comprising a processor that causes a fallback intervention in the driving control system.

15. The vehicle of claim 14, in which the fallback intervention comprises causing the vehicle to enter a fully autonomous driving mode, a semi-autonomous driving mode, or a fully manual driving mode.

16. The vehicle of claim 14, in which the fallback intervention comprises causing the vehicle to operate at a reduced velocity.

17. The vehicle of claim 14, in which the fallback intervention comprises identifying a safe-to-stop location and generating a new trajectory to the safe-to-stop location.

18. The vehicle of claim 14, in which the fallback intervention comprises invoking a backup hardware component or a backup software process.

19. The vehicle of claim 14, in which the fallback intervention comprises evaluating a functional hardware component or a functional software process required to operate the vehicle.

20. The vehicle of claim 1, comprising a processor that evaluates one or more active intervention requests associated with the vehicle or with the environment of the vehicle.

21. The vehicle of claim 20, in which evaluating one or more active events comprises merging two or more intervention requests.

22. The vehicle of claim 20, in which evaluating one or more active events comprises prioritizing an intervention request using one or more of the following:

a decision tree, a combinatorial optimization, a machine algorithm, and a past intervention.

23. The vehicle of claim 1, comprising a processor that treats a current location specified in an intervention as prior knowledge and using an inference algorithm to update the current location.

24. The vehicle of claim 1, comprising a processor that treats a goal location specified in an intervention as prior knowledge and using an inference algorithm to update the goal location.

25. The vehicle of claim 1, comprising a processor that treats a trajectory specified in an intervention as prior knowledge and using an inference algorithm to update the trajectory.

26. The vehicle of claim 1, comprising a processor that treats one or more trajectory sampling points specified in an intervention as prior knowledge and using an inference algorithm to update the one or more trajectory sampling points.

27. The vehicle of claim 26, comprising a processor that infers a trajectory or a trajectory segment based on the one or more trajectory sampling points.

28. The vehicle of claim 27, in which inferring a trajectory or a trajectory segment is based on one or more trajectory primitives.

29. The vehicle of claim 27, comprising a processor that concatenates two trajectory segments, the concatenating comprises smoothing the trajectory segments and smoothing speed profiles across the trajectory segments.

30. The vehicle of claim 1, in which the intervention comprises specifying one or more un-traversable road segments.

31. The vehicle of claim 1, comprising a processor that treats a speed profile specified in an intervention as prior knowledge and using an inference algorithm to update the speed profile.

32. The vehicle of claim 1, comprising a processor that executes the intervention to enable, edit or disable a hardware component or a software process.

33. The vehicle of claim 1, comprising a processor that executes the intervention to overwrite a travel preference or a travel rule.

34. The vehicle of claim 1, comprising a processor that executes the intervention to edit data, the data comprising one or more of the following:

a map, sensor data, trajectory data, vision data, or any past data.