Remote operation of autonomous vehicle in unexpected environment

• US 10,248,116 B2
• Filed: 10/28/2016
• Issued: 04/02/2019
• Est. Priority Date: 11/13/2014
• Status: Active Grant

• Current Assignee: Toyota Jidosha Kabushiki Kaisha (Toyota Motor Corporation)
• Original Assignee: Toyota Motor Engineering & Manufacturing North America Incorporated (Toyota Motor Corporation)
• Inventors: Okumura, Bunyo, Prokhorov, Danil V.
• Primary Examiner(s): Tissot, Adam D
• Application Number:
  US15/337,008
• Publication Number:
  US 20170045885A1
• Time in Patent Office: 886 Days
• Field of Search: 701 36, 701 23, 701 41, 701 70, 180443, 180444
• CPC Class Codes: B60W 30/00 Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00

Abstract

Computer devices, systems, and methods for an autonomous passenger vehicle are described. An unexpected driving environment can be identified, such as by using one or more processors. Information based on the unexpected driving environment received from one or more sensors disposed on the vehicle can be sent to a remote operator using a remote server. A command sent by the remote operator relating to one or more vehicle systems can be received. The command can be sent to the one or more vehicle systems for execution.

Claims

1. A computing device for an autonomous passenger vehicle comprising:

one or more processors for controlling operations of the computing device;

a communications interface configured to communicate with a remote server over a network; and

a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to;

identify an unexpected driving environment;

send information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using the remote server, the information sent over the network being based at least in part on a quality metric of the network;

receive a command sent by the remote operator relating to one or more vehicle systems; and

send the command to the one or more vehicle systems for execution.

2. The computing device of claim 1, wherein identify an unexpected driving environment includes identify the unexpected driving environment if the vehicle is located in a region that is predesignated as a challenging environment.

3. The computing device of claim 1, wherein identify an unexpected driving environment includes identify the unexpected driving environment if information received from one or more sensors about a current driving environment of the vehicle differs from expected data about the current driving environment.

4. The computing device of claim 1, wherein the unexpected driving environment is identified autonomously based at least in part on information received from the one or more sensors disposed on the vehicle.

5. The computing device of claim 1, wherein the command received from the remote operator is used to directly control the one or more vehicle systems.

6. The computing device of claim 1, wherein the vehicle systems are controlled autonomously based on the command received from the remote operator.

7. The computing device of claim 1, wherein the information received from the one or more sensors is sent to the remote operator upon receiving an indication of approval by an occupant of the vehicle.

8. The computing device of claim 1, wherein the information received from the one or more sensors is automatically sent to the remote operator responsive to identifying the unexpected driving environment.

10. The computing device of claim 1, wherein additional information relating to the unexpected driving environment is received from other vehicles located at the unexpected driving environment.

12. The computing device of claim 1, wherein the network is selected based on quality of service.

9. A computing device for a vehicle comprising:

one or more processors for controlling operations of the computing device;

a communications interface configured to communicate with a remote server over a network; and

a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to;

identify an unexpected driving environment;

selectively send information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using the remote server, the information received from the one or more sensors being selectively sent to the remote operator based on the remote operator'"'"'s need for the information in remotely operating the vehicle, the remote operator'"'"'s need for the information in remotely operating the vehicle being based on the unexpected driving environment, the information including;

only sensor data from the portion of the external environment in which the unexpected driving environment is located;

or

only sensor data from the portion of the external environment in which the unexpected driving environment is located and sensor data from a forward portion of the vehicle relative to the travel direction of the vehicle;

receive a command sent by the remote operator relating to one or more vehicle systems; and

send the command to the one or more vehicle systems for execution.

11. A computing device for a vehicle comprising:

one or more processors for controlling operations of the computing device;

a communications interface configured to communicate with a remote server over a network; and

a memory for storing data and program instructions used by the one or more processors, wherein the one or more processors are configured to execute instructions stored in the memory to;

identify an unexpected driving environment;

send information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using the remote server;

receive a command sent by the remote operator relating to one or more vehicle systems; and

send the command to one or more other vehicles located at the unexpected driving environment for autonomously controlling the one or more other vehicles.

13. A computer-implemented method for an autonomous passenger vehicle, the method comprising:

identifying, using one or more processors, an unexpected driving environment;

sending information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using a remote server, the information received from the one or more sensors being selectively sent to the remote operator based on the remote operator'"'"'s need for the information in remotely operating the vehicle, the remote operator'"'"'s need for the information in remotely operating the vehicle being based on the unexpected driving environment, the information including;

only sensor data from the portion of the external environment in which the unexpected driving environment;

or

only sensor data from the portion of the external environment in which the unexpected driving environment is located and sensor data from a forward portion of the vehicle relative to the travel direction of the vehicle;

receiving a command sent by the remote operator relating to one or more vehicle systems; and

sending the command to the one or more vehicle systems for execution.

14. The method of claim 13, wherein identifying an unexpected driving environment includes identifying the unexpected driving environment if the vehicle is located in a region that is predesignated as a challenging environment.

15. The method of claim 13, wherein identifying an unexpected driving environment includes identifying the unexpected driving environment if information received from one or more sensors about a current driving environment of the vehicle differs from expected data about the current driving environment.

16. The method of claim 13, wherein the unexpected driving environment is identified autonomously based at least in part on information received from the one or more sensors disposed on the vehicle.

17. The method of claim 13, wherein the command received from the remote operator is used to directly control the one or more vehicle systems.

18. The method of claim 13, wherein the vehicle systems are controlled autonomously based on the command received from the remote operator.

19. A computer-implemented method for a vehicle, the method comprising:

identifying, using one or more processors, an unexpected driving environment;

sending information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote operator using a remote server;

receiving at the vehicle a command sent by the remote operator relating to one or more vehicle systems; and

sending the command from the vehicle to one or more other vehicles located at the unexpected driving environment for autonomously controlling the one or more other vehicles.

20. A computer-implemented method for a vehicle, the method comprising:

identifying, using one or more processors, an unexpected driving environment;

sending information based on the unexpected driving environment received from one or more sensors disposed on the vehicle to a remote server over a network, the information sent being based at least in part on a quality metric of the network;

receiving a command sent by a remote operator using the remote server, the command relating to one or more vehicle systems; and

sending the command to the one or more vehicle systems for execution.