System for asymmetric just-in-time human intervention in automated vehicle fleets

• US 10,775,783 B2
• Filed: 08/04/2016
• Issued: 09/15/2020
• Est. Priority Date: 08/04/2016
• Status: Active Grant

• Current Assignee: Kevin Lawler
• Original Assignee: Kevin Lawler
• Inventors: Lawler, Kevin Christopher, Locklin, Scott Christopher
• Primary Examiner(s): To, Tuan C
• Assistant Examiner(s): Williams, Kelly D
• Application Number:
  US15/229,059
• Publication Number:
  US 20180039265A1
• Time in Patent Office: 1,503 Days
• Field of Search: None
• CPC Class Codes: B60W 50/14 Means for informing the driver, warning the driver or prompting a driver intervention
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G06Q 10/083 Shipping
• CPC Class Codes: G06Q 50/40 Business processes related to the transportation industry shipping G06Q10/083
• CPC Class Codes: H04L 67/125 involving control of end-device applications over a network

Abstract

A bank of available remote human vehicle operators receives service requests to assume control of an automated vehicle via a service queue. The service queue triages remote vehicle control requests based on factors including skill level of available human vehicle operators, urgency of service requests, capabilities of available human vehicle operators, etc. Additional human vehicle operators may join a control session of an automated vehicle as observers and/or assistants to a controlling human vehicle operator. Human vehicle operators may be triaged among ongoing vehicle control sessions depending on the completion of vehicle control tasks and/or emergence of new requests from other automatic vehicles. Automatic vehicles may make service requests to the bank of available remote human vehicle operators including requested capabilities or operator skills based on data sensed regarding the environment of the vehicle.

Claims

1. A method of triaging control requests of an automated vehicle, the method comprising:

receiving, from a vehicle under automatic control, a service request to assume remote manual control of the vehicle, the service request including an assistance message, the assistance message including a requested operator capability;

pairing a human vehicle operator in a bank of available human vehicle operators with the vehicle under automatic control based at least in part on the assistance message and on a comparison of a capability attribute score of the human vehicle operator to other human vehicle operators in the bank of available human vehicle operators with respect to the requested operator capability;

routing an assignment of manual control of the vehicle to the human vehicle operator in the bank of available human vehicle operators; and

transferring manual control of the vehicle to the human vehicle operator in the bank of available human vehicle operators during a manual control period.

2. The method of claim 1, further comprising:

ending participation in transmission of the remote vehicle navigation signals to the automated vehicle and ending manual control by the human vehicle operator over the automated vehicle upon compliance with the assistance instructions.

3. The method of claim 1, further comprising:

monitoring control signals sent from the human vehicle operator to the vehicle;

monitoring vehicle data received from the vehicle during the manual control period; and

updating a capability attribute score of the human vehicle operator based at least in part on the control signals and the vehicle data.

4. The method of claim 1, wherein the pairing operation is based at least in part on an urgency level of the service request.

5. The method of claim 4, wherein the urgency level includes a threat indication to the vehicle in the service request.

6. The method of claim 1, wherein the bank of available human vehicle operators a calls out to a re-insurance bank of human vehicle operators and the operation that routes the assignment of manual control of the vehicle to the human vehicle operator in the bank of human vehicle operators further routes the assignment of manual control of the vehicle to another human vehicle operator in the re-insurance bank of human vehicle operators.

7. The method of claim 1, further comprising:

pairing a second human vehicle operator from the bank of available human vehicle operators with the vehicle, the second human vehicle operator being an observer of telemetric data of the vehicle and having a communications channel with the human vehicle operator.

8. An automated vehicle system, the system comprising:

an automated vehicle navigation system for determining navigation control signals to a vehicle based on sensor data reflecting an environment of the automated vehicle;

a manual assistance arbiter to determine the automated vehicle is in need of manual control based on the environment of the automated vehicle during a service period; and

a logic bridge to transmit a service request to a bank of available manual human vehicle operators, the logic bridge transmitting a another service request to another bank of available human vehicle operators based on an elapsed time from transmission of the service request, the logic bridge further receiving an assignment of the service request to a human vehicle operator in the another bank of human vehicle operators and vehicle control signals from the human vehicle operator during the service period.

9. The automated vehicle system of claim 8, wherein the service request includes a request for verbal interaction between the human vehicle operator and a human in the environment of the vehicle.

10. The automated vehicle system of claim 8, wherein the service request includes a vehicle operation capability of the human vehicle operator.

11. The automated vehicle system of claim 8, wherein the automated vehicle system has a role that is nonhomogeneous with roles of other automated vehicle systems transmitting service requests to the bank of available manual human vehicle operators.

12. A method of remote automatic vehicle control, the method comprising:

communicating an availability message to a service queue provider, the service queue provider triaging remote manual vehicle control requests;

receiving, by a matching engine of the service queue provider, a service request to assume remote manual control of an automated vehicle, the service request including assistance instructions;

communicating an acceptance of the service request to the service queue provider;

accepting remote manual control of the automated vehicle;

transmitting remote vehicle navigation signals to the automated vehicle in accordance with the assistance instructions; and

ending participation in transmission of the remote vehicle navigation signals to the automated vehicle and ending manual control by the human vehicle operator over the automated vehicle before compliance with the assistance instructions.

13. The method of claim 12, wherein the availability message includes available vehicle control signal input hardware.

14. The method of claim 12, wherein the availability message includes one or more vehicle operator capabilities, the one or more vehicle operator capabilities being associated with a task included in the service request.

15. The method of claim 12, wherein the assistance instructions include a request for remote verbal communication with a person in the environment of the automated vehicle.

16. A method of triaging control requests of an automated vehicle, the method comprising:

receiving, from a vehicle under automatic control, a service request to assume remote manual control of the vehicle, the service request including an assistance message;

increasing a number of potential human vehicle operators by calling out to a re-insurance bank of human vehicle operators;

pairing a human vehicle operator in the re-insurance bank of human vehicle operators with the vehicle under automatic control based at least in part on the assistance message;

routing an assignment of manual control of the vehicle to the human vehicle operator in the re-insurance bank of available human vehicle operators; and

transferring manual control of the vehicle to the human vehicle operator in the re-insurance bank of available human vehicle operators during a manual control period.

17. The method of claim 16, wherein the denial of the request to pair a human vehicle operator is based on a lack of availability of human vehicle operators in the bank of human vehicle operators.