Method for determining fleet control policies to satisfy a sequence of coverage requirements

• US 9,635,706 B1
• Filed: 01/02/2013
• Issued: 04/25/2017
• Est. Priority Date: 01/02/2013
• Status: Active Grant

• Current Assignee: SOFTBANK Corporation (SoftBank Group Corp.)
• Original Assignee: X Development LLC (Alphabet Inc.)
• Inventors: Bonawitz, Keith Allen
• Primary Examiner(s): Smith, Jelani
• Assistant Examiner(s): Ismail, Mahmoud
• Application Number:
  US13/732,546
• Time in Patent Office: 1,574 Days
• Field of Search: 701117, 701200, 705 712, 340902, 455431
• CPC Class Codes: H04B 7/18504 Aircraft used as relay or high altitude atmospheric platform
• CPC Class Codes: H04W 84/18 Self-organising networks, e.g. ad-hoc networks or sensor networks

Abstract

Methods and systems for determining control policies for a fleet of vehicles are provided. In one example, a method is provided that comprises receiving a sequence of coverage requirements for a region and an associated period of time, and receiving an initial location of one or more vehicles of a fleet of vehicles. The method may further include determining a control policy for each of the one or more vehicles. Additionally, based on the determined control policies and the initial locations, one or more estimated distributions of the fleet of vehicles at respective phases within the period of time may be determined. According to the method, a score associated with the control policies may be determined based on a comparison between the estimated distributions and corresponding desired distributions of the sequence of coverage requirements. In some examples, the control policies may also be revised using an optimization technique.

Claims

1. A method comprising:

receiving a sequence of individual coverage requirements for a region and an associated period of time, wherein the region is divided into a plurality of cells, and wherein each individual coverage requirement of the sequence of individual coverage requirements indicates a desired distribution of vehicles of a fleet of vehicles among the plurality of cells at a given phase within the period of time;

receiving an initial location of one or more vehicles of the fleet of vehicles;

determining, by a processor, a control policy for each of the one or more vehicles for the period of time, wherein the control policy comprises, for each of multiple respective cells of the plurality of cells, information indicative of a corresponding direction for the vehicle to travel while the vehicle is located in the cell;

based on the control policies and the initial locations of the one or more vehicles, determining, by the processor, one or more estimated distributions of vehicles in the fleet of vehicles at respective phases within the period of time;

determining, by the processor, a score associated with the determined control policies based on a comparison between the one or more estimated distributions and corresponding desired distributions of the sequence of individual coverage requirements, wherein the score quantifies a difference between the one or more estimated distributions and the corresponding desired distributions of the sequence of individual coverage requirements;

revising, by the processor, at least one of the determined control policies to obtain a revised set of control policies that reduces the score; and

instructing the one or more vehicles of the fleet of vehicles to execute the revised set of control policies.

2. The method of claim 1, wherein revising at least one of the determined control policies comprises revising, by the processor, the determined control policies using an optimization technique to determine an optimal score that minimizes the difference between the one or more estimated distributions and the corresponding desired distributions of the sequence of individual coverage requirements.

3. The method of claim 2, wherein the optimization technique comprises a stochastic optimization technique.

4. The method of claim 2, wherein revising the control policy for a given vehicle using an optimization technique to determine an optimal score comprises:

determining one or more alternative directions of travel within an individual cell of a control policy associated with the given vehicle;

for each of the one or more alternative directions of travel, determining a set of estimated distributions of the fleet of vehicles;

determining a score associated with each set of estimated distributions; and

revising the control policy for the given vehicle based on the score associated with each set of estimated distributions.

5. The method of claim 1, wherein determining one or more estimated distributions of the fleet of vehicles at respective phases within the period of time comprises:

determining, for each of the one or more vehicles, a trajectory for the vehicle over the period of time given the initial location of the vehicle and the control policy for the vehicle; and

based on the determined trajectories for the one or more vehicles, determining an estimated distribution of the fleet of vehicles among the plurality of cells at the respective phases.

6. The method of claim 5, further comprising:

determining the score associated with the determined control policies based on an estimated amount of fuel consumed by the one or more vehicles executing the determined control policies.

7. The method of claim 1, wherein the control policies for the one or more vehicles vary over time.

8. The method of claim 7, wherein the information indicative of the direction for the vehicle to travel is indicative of a same direction of travel for each cell of the plurality of cells during a given time period.

9. The method of claim 1, further comprising determining the score associated with the determined control policies based on an amount of separation between the one or more vehicles during the period of time.

10. The method of claim 1, further comprising determining the score associated with the determined control policies based on a number of neighboring vehicles within a predetermined distance of each vehicle during the period of time.

11. The method of claim 1, wherein vehicles of the fleet of vehicles include a balloon within a data network that is operable to provide data communication via optical or radio-frequency (RF) links.

12. The method of claim 11, wherein a direction of travel within a given cell is constrained by estimated winds at one or more altitudes for the cell.

13. A non-transitory computer-readable storage medium having stored therein instructions, that when executed by a computing device, cause the computing device to perform functions comprising:

receiving a sequence of individual coverage requirements for a region and an associated period of time, wherein the region is divided into a plurality of cells, and wherein each individual coverage requirement of the sequence of coverage requirements indicates a desired distribution of vehicles of a fleet of vehicles among the plurality of cells at a given phase within the period of time;

receiving an initial location of one or more vehicles of the fleet of vehicles;

determining a control policy for each of the one or more vehicles for the period of time, wherein the control policy comprises, for each of multiple respective cells of the plurality of cells, information indicative of a direction for the vehicle to travel while the vehicle is located in the cell;

based on the control policies and the initial locations of the one or more vehicles, determining one or more estimated distributions of vehicles in the fleet of vehicles at respective phases within the period of time;

determining a score associated with the determined control policies based on a comparison between the one or more estimated distributions and corresponding desired distributions of the sequence of individual coverage requirements, wherein the score quantifies a difference between the one or more estimated distributions and the corresponding desired distributions of the sequence of individual coverage requirements;

revising at least one of the determined control policies to obtain a revised set of control policies that reduces the score; and

instructing the one or more vehicles of the fleet of vehicles to execute the revised set of control policies.

14. The non-transitory computer-readable storage medium of claim 13, wherein revising at least one of the determined control polices comprises revising the determined control policies using an optimization technique to determine an optimal score that minimizes the difference between the one or more estimated distributions and the corresponding desired distributions of the sequence of individual coverage requirements.

15. The non-transitory computer-readable storage medium of claim 14, wherein revising the control policy for a given vehicle using an optimization technique to determine an optimal score comprises:

determining one or more alternative directions of travel within an individual cell of a control policy associated with the given vehicle;

for each of the one or more alternative directions of travel, determining a set of estimated distributions of the fleet of vehicles;

determining a score associated with each set of estimated distributions; and

revising the control policy for the given vehicle based on the score associated with each set of estimated distributions.

16. The non-transitory computer-readable storage medium of claim 13, wherein vehicles of the fleet of vehicles include a balloon within a data network that is operable to provide data communication via optical or radio-frequency (RF) links.

17. A system, comprising:

at least one processor; and

data storage comprising program instructions executable by the at least one process to cause the system to perform functions comprising;

receiving a sequence of individual coverage requirements for a region and an associated period of time, wherein the region is divided into a plurality of cells, and wherein each individual coverage requirement indicates a desired distribution of vehicles of a fleet of vehicles among the plurality of cells at a given phase within the period of time;

receiving an initial location of one or more vehicles of the fleet of vehicles;

determining a control policy for each of the one or more vehicles for the period of time, wherein the control policy comprises, for each of multiple respective cells of the plurality of cells, information indicative of a direction for the vehicle to travel while the vehicle is located in the cell;

based on the control policies and the initial locations of the one or more vehicles, determining one or more estimated distributions of vehicles in the fleet of vehicles at respective phases within the period of time;

determining a score associated with the determined control policies based on a comparison between the one or more estimated distributions and corresponding desired distributions of the sequence of individual coverage requirements, wherein the score quantifies a difference between the one or more estimated distributions and the corresponding desired distributions of the sequence of individual coverage requirements;

revising at least one of the determined control policies to obtain a revised set of control policies that reduces the score; and

instructing the one or more vehicles of the fleet of vehicles to execute the revised set of control policies.

18. The system of claim 17, wherein revising at least one of the determined control policies comprises revising the determined control policies for the one or more vehicles to determine an optimal score that minimizes the difference between the one or more estimated distributions and the corresponding desired distributions of the sequence of individual coverage requirements by:

determining one or more alternative directions of travel within an individual cell of a control policy associated with a given vehicle;

for each of the one or more alternative directions of travel, determining a set of estimated distributions of the fleet of vehicles;

determining a score associated with each set of estimated distributions; and

revising the control policy for the given vehicle based on the score associated with each set of estimated distributions.