AUTONOMOUS MISSION ACTION ALTERATION

• US 20180122246A1
• Filed: 11/23/2016
• Published: 05/03/2018
• Est. Priority Date: 11/23/2015
• Status: Active Grant

• Current Assignee: Firmatek Software LLC
• Original Assignee: Kespry, Inc.
• Inventors: Clark, Robert Parker
• Granted Patent: US 10,126,126 B2
• Time in Patent Office: Days
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 31/06 actuated automatically
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/32 for cartography or topography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: B64U 50/13 using external fans or propellers
• CPC Class Codes: G01C 7/02 of land surfaces
• CPC Class Codes: G01C 7/04 involving a vehicle which moves along the profile to be traced
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G08G 5/0034 Assembly of a flight plan
• CPC Class Codes: G08G 5/0039 Modification of a flight plan
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

An unmanned aerial vehicle responds to mission cues during a mission. The mission cues are characteristics of image and/or sensor data. The unmanned aerial vehicle may change data gathering operations or may perform sub-missions within a mission in response to the mission cues.

Claims

1. An unmanned aerial vehicle comprising:

a camera configured to generate photographic images;

memory storing first travel instructions that define a first flight path for the unmanned aerial vehicle;

memory storing one or more pre-defined image characteristics that correspond to one or more image characteristics that are exhibited by images of one or more pre-defined objects and/or one or more pre-defined objects having one or more pre-defined conditions; and

one or more processors configured to;

control flight of the unmanned aerial vehicle to perform a first part of the first flight path by executing some of the first travel instructions;

control the camera to generate one or more photographic images with the camera while performing the first part of the first flight path;

process the one or more photographic images generated by the camera during the first part of the first flight path to determine whether or not any of the one or more pre-defined image characteristics are present in the one or more photographic images;

in response at least to detecting some or all of the pre-defined image characteristics in at least one of the one or more photographic images generated by the camera during the first part of the first flight path, generate and/or retrieve second travel instructions that define a second flight path for the unmanned aerial vehicle;

after performing the first part of the first flight path, control flight of the unmanned aerial vehicle to perform the second flight path by executing the second travel instructions; and

control the camera to generate one or more photographic images while performing the second flight path, wherein the one or more photographic images generated during the second flight path including at least one image of a pre-defined object or pre-defined condition that corresponds to the pre-defined image characteristics detected in the at least one of the one or more photographic images generated by the camera during the first part of the first flight path.

2. The unmanned aerial vehicle of claim 1, wherein the one or more processors are configured to perform a second part of the first flight path by executing some of the first travel instructions after performing the second flight path by executing the second instructions.

3. The unmanned aerial vehicle of claim 1, wherein the first travel instructions include instructions to hover in a pre-defined location that is part of the first flight path and take at least one photographic image with the camera while the unmanned aerial vehicle is hovering in the pre-defined location.

4. The unmanned aerial vehicle of claim 3, wherein the second travel instructions include instructions to move from the pre-defined location in a direction the same as or derived from the direction the camera is pointing when the at least one photographic image is taken while the unmanned aerial vehicle is hovering at the pre-defined location.

5. The unmanned aerial vehicle of claim 3 wherein the unmanned aerial vehicle comprises a distance detector, and the second travel instructions include instructions to move from the pre-defined location by an amount that is based at least in part on an output of the distance detector.

6. The unmanned aerial vehicle of claim 4, wherein the unmanned aerial vehicle comprises a distance detector, and the second travel instructions include instructions to move from the pre-defined location in the direction by an amount that is based at least in part on an output of the distance detector.

7. The unmanned aerial vehicle of claim 5 or 6, wherein the distance detector comprises LIDAR.

8. The unmanned aerial vehicle of claim 5 or 6, wherein the distance detector comprises a stereo camera.

9. The unmanned aerial vehicle of claim 5, wherein the unmanned aerial vehicle comprises a location detector, and the second travel instructions include instructions to move from the pre-defined location to a second different location, wherein the second different location is based at least in part on an output of the location detector.

10. The unmanned aerial vehicle of claim 6, wherein the unmanned aerial vehicle comprises a location detector, and the second travel instructions include instructions to move from the pre-defined location in the direction to a second different location, wherein the second different location is based at least in part on an output of the distance detector.

11. The unmanned aerial vehicle of claim 5 or 6, wherein the location detector comprises a GPS receiver.

12. The unmanned aerial vehicle of claim 5, wherein the second flight path begins and ends at the pre-defined location.

13. The unmanned aerial vehicle of claim 12, wherein the second flight path comprises a series of linear path segments.

14. The unmanned aerial vehicle of claim 7, wherein the second flight path consists essentially of two linear path segments.

15. The unmanned aerial vehicle of any of the preceding claims, wherein at least one pre-defined object with at least one pre-defined condition is a roof of a structure having storm damage.

16. The unmanned aerial vehicle of any of the preceding claims, wherein at least one pre-defined object is a bridge truss joint.

17. The unmanned aerial vehicle of any of the preceding claims, wherein at least one pre-defined object is a bridge truss joint bolt.

18. The unmanned aerial vehicle of any of the preceding claims, wherein at least one pre-defined object with at least one pre-defined condition is a bridge truss joint bolt having rust thereon.

19. An unmanned aerial vehicle comprising:

a camera configured to generate image data;

one or more sensors configured to generate sensor data;

memory storing travel instructions that define a mission for the unmanned aerial vehicle;

memory storing one or more mission cues comprising one or more pre-defined image data characteristics and/or sensor data characteristics; and

one or more processors configured to;

execute the travel instructions to control the unmanned aerial vehicle to perform the mission;

process the image data and/or sensor data generated during the mission to detect the presence of some or all of the mission cues;

in response to detecting a mission cue, changing image generation operations performed by the camera such that the amount of image data generated during the mission is dependent on the number of mission cues detected during the mission.

20. The unmanned aerial vehicle of claim 19, wherein the one or more processors are configured to change the resolution of image data stored in or transmitted from the unmanned aerial vehicle in response to detecting one or more of the mission cues during the mission.

21. The unmanned aerial vehicle of claim 19, wherein one of the sensors comprises a distance detector configured to gather distance data, and wherein the one or more processors is configured to detect a mission cue based at least in part on the distance data, and wherein the distance data indicates a change in a topology of the area under the unmanned aerial vehicle.

22. The unmanned aerial vehicle of claim 19, wherein the one or more processors are configured to change the rate per travel distance of the unmanned aerial vehicle at which image data is stored in memory on the unmanned aerial vehicle or transmitted from the unmanned aerial vehicle in response to detecting one or more of the mission cues.

23. The method of any one of claims 19 through 22, wherein a mission cue comprises pre-defined image characteristics corresponding to image characteristics of a roof structure with storm damage.

24. The method of any one of claims 19 through 23, wherein a mission cue comprises image characteristics and/or sensor data characteristics corresponding to a topology change of the area under the unmanned aerial vehicle.

25. An unmanned aerial vehicle comprising:

a camera configured to generate image data;

one or more sensors configured to generate sensor data;

memory storing first travel instructions that define an overall mission for the unmanned aerial vehicle;

memory storing second travel instructions that define at least a first sub-mission of the overall mission;

memory storing one or more mission cues comprising one or more pre-defined image data characteristics and/or sensor data characteristics; and

one or more processors configured to;

execute the first travel instructions to control the unmanned aerial vehicle to perform the overall mission;

process the image data and/or sensor data generated during the overall mission to detect the presence of some or all of the mission cues;

in response to detecting a mission cue, interrupting execution of the first travel instructions that define the overall mission and executing the second travel instructions to control the unmanned aerial vehicle to perform the first sub-mission of the overall mission; and

after executing the second travel instructions, continuing execution of the first travel instructions to continue performing the overall mission.

26. The unmanned aerial vehicle of claim 25, comprising memory storing third travel instructions that define at least a first sub-mission of at least the first sub-mission of the overall mission defined by the second travel instructions.

27. The unmanned aerial vehicle of claim 26, wherein the one or more processors are configured to:

process the image data and/or sensor data generated during the first sub-mission defined by the second travel instructions to detect the presence of some or all of the mission cues;

in response to detecting a mission cue, interrupting execution of the second travel instructions that define the first sub-mission of the overall mission and executing the third travel instructions to control the unmanned aerial vehicle to perform the first sub-mission of the first sub-mission of the overall mission; and

after executing the third instructions, continuing execution of the second instructions to continue performing the first sub-mission of the overall mission.

28. The unmanned aerial vehicle of any one of claims 25 through 27, wherein the first sub-mission of the first submission of the overall mission has the same starting and ending location, and wherein the starting and ending location of the of the first sub-mission of the first sub-mission of the overall mission is a location that is on the flight path of the first sub-mission of the overall mission.

29. The unmanned aerial vehicle of any one of claims 25 through 28, wherein the first sub-mission of the overall mission has the same starting and ending location, and wherein the starting and ending location of the of the first sub-mission of the overall mission is a location that is on the flight path of the overall mission.

30. In an unmanned aerial vehicle, a method of gathering data comprising:

executing at least some first travel instructions to perform a first part of a first flight path;

generating photographic images during the first part of the first flight path;

detecting one or more pre-defined image characteristics that correspond to one or more image characteristics that are exhibited by images of one or more pre-defined objects and/or one or more pre-defined objects having one or more pre-defined conditions in at least one of the one or more photographic images generated by the camera during the first part of the first flight path;

in response at least to the detecting, generating and/or retrieving second travel instructions that define a second flight path for the unmanned aerial vehicle;

after performing the first part of the first flight path, executing the second travel instructions to perform the second flight path; and

generating one or more photographic images while performing the second flight path, wherein the one or more photographic images generated during the second flight path include at least one image of a pre-defined object or pre-defined condition that corresponds to the pre-defined image characteristics detected in the at least one of the one or more photographic images generated by the camera during the first part of the first flight path.

31. The method of claim 30, comprising performing a second part of the first flight path by executing some of the first travel instructions after performing the second flight path by executing the second instructions.

32. The method of claim 30, comprising hovering in a pre-defined location that is part of the first flight path and taking at least one photographic image with the camera while the unmanned aerial vehicle is hovering in the pre-defined location.

33. The method of claim 32, comprising moving from the pre-defined location in a direction the same as or derived from the direction the camera is pointing when the at least one photographic image is taken while the unmanned aerial vehicle is hovering at the pre-defined location.

34. The method of claim 32, comprising moving from the pre-defined location by a defined amount.

35. The method of claim 34, comprising moving the defined amount back to the pre-defined location.

36. The method of claim 35, wherein the moving and the moving back are performed as two overlapping linear path segments.

37. The method of any of claims 30 through 36, wherein at least one pre-defined object with at least one pre-defined condition is a roof of a structure having storm damage.

38. The method of any of claims 30 through 36, wherein at least one pre-defined object is a bridge truss joint.

39. The method of any of claims 30 through 36, wherein at least one pre-defined object is a bridge truss joint bolt.

40. The method of any of claims 30 through 36, wherein at least one pre-defined object with at least one pre-defined condition is a bridge truss joint bolt having rust thereon.

41. In an unmanned aerial vehicle, a method of gathering data comprising:

executing travel instructions to control the unmanned aerial vehicle to perform a mission;

detecting one or more mission cues during the mission, the mission cues comprising one or more pre-defined image data characteristics and/or sensor data characteristics;

in response to detecting a mission cue, changing image generation operations performed by the camera such that the amount of image data generated during the mission is dependent on the number of mission cues detected during the mission.

42. The method of claim 41, wherein the changing comprises changing the resolution of image data stored in or transmitted from the unmanned aerial vehicle.

43. The method of claim 41, wherein the changing comprises changing the rate per travel distance of the unmanned aerial vehicle at which image data is stored in memory on the unmanned aerial vehicle or transmitted from the unmanned aerial vehicle.

44. In an unmanned aerial vehicle, a method of gathering data comprising:

executing first travel instructions to control the unmanned aerial vehicle to perform an overall mission;

processing image data and/or sensor data generated during the overall mission to detect the presence of one or more mission cues, the mission cues comprising one or more pre-defined image data characteristics and/or sensor data characteristics;

in response to detecting a mission cue, interrupting execution of the first travel instructions that define the overall mission and executing second travel instructions to perform a first sub-mission of the overall mission; and

after executing the second travel instructions, continuing execution of the first travel instructions to continue performing the overall mission.

45. The method of claim 44 comprising:

processing image data and/or sensor data generated during the first sub-mission defined by the second travel instructions to detect the presence of one or more mission cues;

in response to detecting a mission cue, interrupting execution of the second travel instructions that define the first sub-mission of the overall mission and executing third travel instructions to perform a first sub-mission of the first sub-mission of the overall mission; and

after executing the third travel instructions, continuing execution of the second travel instructions to continue performing the first sub-mission of the overall mission.