System and method for human operator intervention in autonomous vehicle operations

• US 10,365,645 B1
• Filed: 03/05/2018
• Issued: 07/30/2019
• Est. Priority Date: 04/14/2015
• Status: Active Grant

• Current Assignee: Zipline International Inc.
• Original Assignee: Zipline International Inc.
• Inventors: Chambers, Andrew, Wyrobek, Keenan, Rinaudo, Keller, Oksenhorn, Ryan, Hetzler, William
• Primary Examiner(s): Shafi, Muhammad
• Application Number:
  US15/912,311
• Time in Patent Office: 512 Days
• Field of Search: 701 3, 701 4, 701 5, 701 6, 701 7, 701 8, 701 10, 701 15, 701 16, 701 18, 701 24, 701 25, 701 26, 701 27, 701 38, 244 23 A, 244 23 B, 244 36, 244211, 244214, 244215, 244 56, 244179, 244180, 244181, 244 80, 244 90 R, 244 90 A, 340963, 340967-970, 340973, 340974, 340975, 340977, 340978, 342 38, 342 63, 342176, 3423572, 34235733, 34235736, 342462
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/60 for transporting passengers; for transporting goods other than weapons
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0022 characterised by the communication link data switching networks in general H04L12/00
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/1064 specially adapted for avoiding collisions with other aircraft

Abstract

An autonomous vehicle system is configured to receive vehicle commands from one or more parties and to execute those vehicle commands in a way that prevents the execution of stale commands. The autonomous vehicle system includes a finite state machine and a command counter or stored vehicle timestamp, which are used to help reject invalid or stale vehicle commands.

Claims

1. A method for piloting an autonomous vehicle, comprising:

selecting, at a computer server, a remote vehicle operator from a group of remote vehicle operators, the remote vehicle operator associated with a control device;

receiving, at the computer server, a situation report associated with the autonomous vehicle;

sending to the control device, from the computer server, information from the situation report;

receiving, at the computer server, at least one operator command from the control device; and

sending, from the computer server, a vehicle command to the autonomous vehicle, the vehicle command based at least in part on the at least one operator command and including a command verification value;

causing, with the vehicle command, a finite state machine of the autonomous vehicle to transition from an active vehicle state of a plurality of vehicle states to a second vehicle state of the plurality of vehicle states after the autonomous vehicle determines that the vehicle command has precedence.

2. The method of claim 1, further comprising:

receiving, at the computer server, a status message from the autonomous vehicle, the status message including a command verification value; and

including the command verification value in the at least one vehicle command;

wherein the autonomous vehicle is configured to transition from the active vehicle state to the second vehicle state only if the command verification value in the vehicle command is valid when compared to a stored command verification value at a time that the vehicle command is received.

3. The method of claim 2, wherein:

the stored command verification value is a command counter indicating a number of commands executed by the autonomous vehicle; and

the autonomous vehicle is configured to transition from the active vehicle state to the second vehicle state only if the command verification value in the vehicle command is equal to the command counter at a time that the vehicle command is received.

4. The method of claim 1, further comprising:

receiving, at the computer server, a status message from the autonomous vehicle, the status message including a vehicle timestamp value indicating a time that a last command executed by the autonomous vehicle was issued; and

including a command timestamp in the vehicle command, the command timestamp indicating a time when the vehicle command was issued;

wherein the autonomous vehicle is configured to transition from the active vehicle state to the second vehicle state only if the command timestamp in the vehicle command is later than the vehicle timestamp value at a time that the vehicle command is received.

5. The method of claim 1, wherein selecting the remote vehicle operator from the group of remote vehicle operators further comprises selecting the remote vehicle operator based on at least one of:

a measure of a quality of a communication link between the control device associated with the remote vehicle operator and the autonomous vehicle;

a score associated with the remote vehicle operator; and

a price associated with the remote vehicle operator.

6. The method of claim 1, wherein the information from the situation report includes information useable by the control device to display, to the remote vehicle operator, a representation of an environment around the autonomous vehicle.

7. The method of claim 1, wherein the finite state machine further comprises a plurality of valid state transitions, the method further comprising, at the autonomous vehicle:

in response to the vehicle command, transitioning the finite state machine from the active vehicle state to the second vehicle state only if a transition from the active vehicle state to the second vehicle state is one of the plurality of valid state transitions.

8. The method of claim 7, wherein:

the vehicle command comprises a command timestamp, the command timestamp indicating a time when the vehicle command was issued;

the autonomous vehicle comprises a stored vehicle timestamp, the stored vehicle timestamp indicating a time when a vehicle command that was previously executed by the autonomous vehicle was issued; and

transitioning the finite state machine from the active vehicle state to the second vehicle state comprises transitioning the finite state machine from the active vehicle state to the second vehicle state only if the command timestamp of the vehicle command is later than the stored vehicle timestamp.

9. The method of claim 8, further comprising, at the autonomous vehicle, sending a status message toward the computer server, the status message including a value of the stored vehicle timestamp.

10. The method of claim 7, further comprising, at the autonomous vehicle:

storing a command count value, the stored command count value indicating an amount of previous vehicle commands executed by the autonomous vehicle;

receiving the vehicle command, the vehicle command including a command verification value; and

transitioning the finite state machine from the active vehicle state to the second vehicle state only if the command verification value is equal to the stored command count value.

11. The method of claim 10, further comprising, at the autonomous vehicle, sending a status message toward the computer server, the status message including the value of the stored command count value.

12. The method of claim 11, wherein:

the command verification value corresponds to a previous command count value received by the computer server in a status message from the autonomous vehicle, and the status message is received prior to the sending of the vehicle command.

13. The method of claim 1, wherein:

the active vehicle state is a nominal flight state; and

the second vehicle state is a drop payload state.

14. A method for piloting an autonomous vehicle, comprising:

storing, in a computer memory of an autonomous vehicle, a first command verification value;

broadcasting, from a communications system of the autonomous vehicle, a status message comprising the first command verification value;

receiving a vehicle command via the communications system, the vehicle command comprising a second command verification value;

determining, using a processor of at the autonomous vehicle, if the second command verification value is valid when compared to the first command verification value;

responsive to determining that the second command verification value is valid, transitioning a finite state machine of the autonomous vehicle from an active vehicle state of a plurality of vehicle states to a second vehicle state of the plurality of vehicle states; and

responsive to determining that the second command verification value is not valid, not transitioning the finite state machine from the active vehicle state of the plurality of vehicle states to the second vehicle state of the plurality of vehicle states.

15. The method of claim 14, wherein:

transitioning the finite state machine from the active vehicle state to the second vehicle state results in the autonomous vehicle changing from a first path to a second path different from the first path.

16. The method of claim 15, wherein:

the first command verification value comprises a command counter indicating a number of commands executed by the autonomous vehicle; and

determining if the second command verification value is valid comprises checking if the first command verification value equals the second command verification value.

17. The method of claim 15, wherein:

the first command verification value comprises a stored vehicle timestamp indicating a time a last command executed by the autonomous vehicle was issued;

the second command verification value comprises a command timestamp indicating a time that the vehicle command was issued; and

determining if the second command verification value is valid comprises checking if the second command verification value is later than the first command verification value.

18. A method for piloting an autonomous vehicle, comprising:

receiving, at a control device associated with a vehicle operator and remote from an autonomous aircraft, information associated with the autonomous aircraft, the information including at least one of a speed or a direction of travel of the autonomous aircraft;

sending, from the control device to the autonomous aircraft while the autonomous aircraft is in flight, a command including a command verification value and vehicle command; and

causing, with the vehicle command, a finite state machine of the autonomous aircraft to transition from an active vehicle state of a plurality of vehicle states to a second vehicle state of the plurality of vehicle states after the autonomous aircraft determines, using the command verification value, that the vehicle command has precedence.

19. The method of claim 18, wherein the operation of causing the finite state machine of the autonomous aircraft to transition from the active vehicle state to the second vehicle state causes the autonomous aircraft to drop a payload from the autonomous aircraft.

20. The method of claim 18, wherein the operation of causing the finite state machine of the autonomous aircraft to transition from the active vehicle state to the second vehicle state causes the autonomous aircraft to return to a location from which the autonomous aircraft was launched.