Multiple unmanned aerial vehicle autonomous coordination

• US 10,120,377 B2
• Filed: 08/13/2016
• Issued: 11/06/2018
• Est. Priority Date: 08/13/2016
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Lai, Jui-Hsin, Ma, Yu, Shi, Conglei, Xia, Yinglong
• Primary Examiner(s): Figueroa, Jaime
• Application Number:
  US15/236,409
• Publication Number:
  US 20180046181A1
• Time in Patent Office: 815 Days
• Field of Search: 701 1, 701 2
• CPC Class Codes: G05D 1/0027 involving a plurality of vehicles, e.g. fleet or convoy travelling fleet control of land vehicles from a control room G05D1/0297; traffic control systems for road vehicles G08G1/00; for marine craft G08G3/00; for aircraft G08G5/00
• CPC Class Codes: G05D 1/104 involving a plurality of aircrafts, e.g. formation flying traffic control systems for aircraft G08G5/00
• CPC Class Codes: G08G 5/0013 with a ground station
• CPC Class Codes: G08G 5/0039 Modification of a flight plan
• CPC Class Codes: G08G 5/0043 Traffic management of multiple aircrafts from the ground G08G5/003 takes precedence; data processing specially designed for resource management, e.g. scheduling or allocating time, human or machine resources G06Q10/06
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933
• CPC Class Codes: H04W 4/023 using mutual or relative location information between multiple location based services [LBS] targets or of distance thresholds

Abstract

At each of a plurality of unmanned aerial vehicles operating in a three-dimensional space, current relative locations of other ones of the plurality of unmanned aerial vehicles which are visible to each of the plurality of unmanned aerial vehicles are determined based on two-dimensional camera images from on-board dual-lens cameras. The current relative locations are wirelessly transmitted to a controller. At the plurality of unmanned aerial vehicles, specification of a path to be followed by each of the plurality of unmanned aerial vehicles is wirelessly received from the controller. At each of the plurality of unmanned aerial vehicles, on-board collision avoidance is carried out based on the two-dimensional camera images from the on-board dual-lens cameras, while following the path specification.

Claims

1. A method comprising:

at each of a plurality of unmanned aerial vehicles operating in a three-dimensional space, determining, based on two-dimensional camera images from at least an on-board double lens camera of that unmanned aerial vehicle, current relative locations with reference to that same unmanned aerial vehicle of other ones of said plurality of unmanned aerial vehicles which are visible to that same unmanned aerial vehicle, wherein determining said current relative locations comprises applying multidimensional scaling to said two-dimensional camera images;

wirelessly transmitting said current relative locations from at least one of said plurality of unmanned aerial vehicles to a controller;

wirelessly receiving, at each of said plurality of unmanned aerial vehicles, from said controller, a specification of a path to be followed by that unmanned aerial vehicle; and

at each of said plurality of unmanned aerial vehicles, carrying out on-board collision avoidance based on said two-dimensional camera images, while following said path.

2. The method of claim 1, further comprising, with said controller, determining said specifications of said paths to be followed by each of said plurality of unmanned aerial vehicles, based at least in part on said current relative locations of said plurality of unmanned aerial vehicles.

3. The method of claim 2, further comprising, with said controller, constructing enhanced-accuracy 3D coordinates and velocities for each of said plurality of unmanned aerial vehicles by aggregating current relative locations determined at each of said plurality of unmanned aerial vehicles.

4. The method of claim 2, wherein determining said specification of said path to be followed includes constructing 3D coordinates of each of said plurality of unmanned aerial vehicles from said current relative locations of said plurality of unmanned aerial vehicles and applying a spring-force model to said 3D coordinates.

5. The method of claim 1, wherein carrying out said on-board collision avoidance includes constructing 3D coordinates of said other ones of said plurality of unmanned aerial vehicles from said current relative locations and applying a spring-force model to said 3D coordinates.

6. The method of claim 1, further comprising locating said controller on a control unmanned aerial vehicle.

7. The method of claim 1, further comprising receiving, at each of said plurality of unmanned aerial vehicles, specifications of paths to be followed by other ones of said plurality of unmanned aerial vehicles.

19. The method of claim 1, wherein determining said current relative locations comprises applying multidimensional scaling to two-dimensional camera images obtained from a plurality of on-board double lens cameras each mounted on a respective one of said plurality of unmanned aerial vehicles.

8. A system comprising:

a plurality of unmanned aerial vehicles operating in a three-dimensional space, each of said unmanned aerial vehicles in turn comprising a memory, at least one processor coupled to said memory, an on-board double lens camera coupled to said processor, and a wireless interface to a remote controller;

wherein each of said processors of said plurality of unmanned aerial vehicles is configured to;

determine, based on two-dimensional camera images from at least said on-board double lens camera of the unmanned aerial vehicle carrying said processor, current relative locations of other ones of said plurality of unmanned aerial vehicles which are visible to that unmanned aerial vehicle, wherein determining said current relative locations comprises applying multidimensional scaling to said two-dimensional camera images;

wirelessly transmit said current relative locations to the remote controller;

wirelessly receive, from the remote controller, a specification of a path to be followed by the unmanned aerial vehicle carrying said processor; and

carry out on-board collision avoidance based on said two-dimensional camera images from said on-board double lens camera, while following said path.

9. The system of claim 8, further comprising said remote controller, wherein said remote controller determines specifications of paths to be followed by each of said plurality of unmanned aerial vehicles, based at least in part on said current relative locations.

10. The system of claim 8, wherein carrying out said on-board collision avoidance constructing 3D coordinates of said other ones of said plurality of unmanned aerial vehicles from said current relative locations and applying a spring-force model to said 3D coordinates.

11. The system of claim 9, wherein said remote controller is located on a control unmanned aerial vehicle that is one of said plurality of unmanned aerial vehicles.

12. The system of claim 9, further comprising a control unmanned aerial vehicle separate and distinct from said plurality of unmanned aerial vehicles, wherein said remote controller is located on said control unmanned aerial vehicle.

13. The system of claim 9, wherein determining said specification of said path to be followed includes constructing 3D coordinates of each of said plurality of unmanned aerial vehicles from said current relative locations of said plurality of unmanned aerial vehicles and applying a spring-force model to said 3D coordinates.

14. The system of claim 9, wherein said remote controller constructs enhanced-accuracy 3D coordinates and velocities for each of said plurality of unmanned aerial vehicles by aggregating current relative locations determined at each of said plurality of unmanned aerial vehicles.

20. The system of claim 8, wherein determining said current relative locations comprises applying multidimensional scaling to two-dimensional camera images obtained from a plurality of on-board double lens cameras each mounted on a respective one of said plurality of unmanned aerial vehicles.

15. A non-transitory computer readable medium comprising computer executable instructions which when executed by a computer cause the computer to perform the method of:

at each of a plurality of unmanned aerial vehicles operating in a three-dimensional space, determining, based on two-dimensional camera images from at least an on-board double lens camera of that unmanned aerial vehicle, current relative locations of other ones of said plurality of unmanned aerial vehicles which are visible to that unmanned aerial vehicle, wherein determining said current relative locations comprises applying multidimensional scaling to said two-dimensional camera images;

wirelessly transmitting said current relative locations from each of said plurality of unmanned aerial vehicles to a controller;

wirelessly receiving, at each of said plurality of unmanned aerial vehicles, from said controller, a specification of a path to be followed by that unmanned aerial vehicle; and

at each of said plurality of unmanned aerial vehicles, carrying out on-board collision avoidance based on said two-dimensional camera images, while following said path.

16. The non-transitory computer readable medium of claim 15, wherein said computer executable instructions cause said controller to determine said specifications of said paths to be followed by each of said plurality of unmanned aerial vehicles, based at least in part on said current relative locations of said plurality of unmanned aerial vehicles.

17. The non-transitory computer readable medium of claim 15, wherein said computer executable instructions cause said controller to construct enhanced-accuracy 3D coordinates and velocities for each of said plurality of unmanned aerial vehicles by aggregating current relative locations determined at each of said plurality of unmanned aerial vehicles.

18. The non-transitory computer readable medium of claim 16, wherein determining said specification of said path to be followed includes constructing 3D coordinates of each of said plurality of unmanned aerial vehicles from said current relative locations of said plurality of unmanned aerial vehicles and applying a spring-force model to said 3D coordinates.