Intervention in operation of a vehicle having autonomous driving capabilities

• US 10,317,899 B2
• Filed: 06/16/2017
• Issued: 06/11/2019
• Est. Priority Date: 06/16/2017
• Status: Active Grant

• Current Assignee: Motional AD, LLC (Aptiv PLC)
• Original Assignee: nuTonomy, Inc. (Aptiv PLC)
• Inventors: Liu, Shih-Yuan, Ravichandran, Harshavardhan, Iagnemma, Karl, Chang, Hsun-Hsien
• Primary Examiner(s): Kan, Yuri
• Application Number:
  US15/624,857
• Publication Number:
  US 20180364704A1
• Time in Patent Office: 725 Days
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/0055 with safety arrangements
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G07C 5/008 communicating information to a remotely located station transmission systems for measured values G08C
• CPC Class Codes: G07C 5/0816 Indicating performance data, e.g. occurrence of a malfunction

Abstract

Among other things, a determination is made that intervention in an operation of one or more autonomous driving capabilities of a vehicle is appropriate. Based on the determination, a person is enabled to provide information for an intervention. The intervention is caused in the operation of the one or more autonomous driving capabilities of the vehicle.

Claims

1. A method comprising:

on a vehicle configured to receive a teleoperation command in the form of natural language instructions or machine-readable instructions, wherein received natural language instructions are converted to machine-readable instructions;

receiving, from a remote operator, a teleoperation command regarding an operation of the vehicle;

identifying an abnormal condition in one or more systems of the vehicle;

in response to identifying the abnormal condition, determining that the received teleoperation command includes machine-readable instructions; and

in response to the determination, configuring the vehicle to execute the machine-readable instructions.

2. The method of claim 1, in which the vehicle comprises one or more autonomous driving capabilities.

3. The method of claim 1, in which the machine-readable instructions represent a current location.

4. The method of claim 1, in which the machine-readable instructions represent a goal location.

5. The method of claim 1, in which the machine-readable instructions represent one or more trajectories.

6. The method of claim 1, in which the machine-readable instructions represent one or more trajectory segments.

7. The method of claim 1, in which the machine-readable instructions represent one or more speed profiles.

8. The method of claim 1, in which the machine-readable instructions represent one or more un-traversable road segments.

9. The method of claim 1, in which the machine-readable instructions comprise enabling, editing or disabling a hardware component or a software process.

10. The method of claim 1, in which the machine-readable instructions comprise enabling, editing or disabling a subcomponent of a hardware component or a processing step of a software process.

11. The method of claim 1, in which the machine-readable instructions comprise overwriting a travel preference or a travel rule.

12. The method of claim 1, in which the machine-readable instructions comprise editing data, the data comprising one or more of the following:

a map, sensor data in the vehicle or a related AV system, trajectory data in the vehicle or a related AV system, vision data in the vehicle or a related AV system, or any past data in the vehicle or a related AV system.

13. The method of claim 1, in which the machine-readable instructions include at least one of an assembly language and a low-level language.

14. The method of claim 1, in which determining that the teleoperation command includes machine-readable instructions comprises determining that the teleoperation command is not expressed in a natural language.

15. The method of claim 1, in which the machine-readable instructions include alphanumeric strings or computer code.

16. The method of claim 1, in which configuring the vehicle to execute the machine-readable instructions comprises delivering the machine-readable instructions to corresponding hardware components or software processes of the vehicle.

17. The method of claim 1, in which the machine-readable instructions are executed if a teleoperation command handling process is in an abnormal condition.

18. The method of claim 1, in which the abnormal condition comprises a malfunction of a mechanical component of the vehicle.

19. The method of claim 1, in which the abnormal condition is a blockage of a field of view of a vision sensor of the vehicle.

20. The method of claim 1, in which the abnormal condition is a movement of the vehicle that does not match an expected movement of the vehicle.