System and method for controlling a remote aerial device for up-close inspection

• US 10,281,911 B1
• Filed: 03/24/2017
• Issued: 05/07/2019
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Tofte, Nathan L., Freeman, James M., Harvey, Brian N.
• Primary Examiner(s): Edwards, Jerrah
• Application Number:
  US15/468,209
• Time in Patent Office: 774 Days
• CPC Class Codes: B64C 19/00 Aircraft control not otherwise provided for
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G05D 1/12 Target-seeking control
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The method and system may be used to control the movement of a remote aerial device in an incremental step manner during a close inspection of an object or other subject matter. At the inspection location, a control module “stabilizes” the remote aerial device in a maintained, consistent hover while maintaining a close distance to the desired object. The control module may retrieve proximal sensor data that indicates possible nearby obstructions to the remote aerial device and may transmit the data to a remote control client. The remote control module may determine and display the possible one or more non-obstructed directions that the remote aerial device is capable of moving by an incremental distance. In response to receiving a selection of one of the directions, the remote control module may transmit the selection to the remote aerial device to indicate the next movement for the remote aerial device.

Claims

1. A computer-implemented method for remotely controlling a remote aerial device for up-close inspection of an object, the method comprising:

controlling, by a remote control module, the remote aerial device to move to an initial target location in proximity to a portion of the object to be inspected;

receiving, at the remote control module from one or more sensors of the remote aerial device, proximal sensor data indicating one or more proximal obstructions located within a distance threshold of the remote aerial device;

determining, by the remote control module, a plurality of non-obstructed incremental movement directions based at least in part upon the proximal sensor data, wherein each non-obstructed incremental movement direction specifies a potential movement for the remote aerial device of a fixed distance in a direction that has been determined by the remote control module to be unobstructed;

receiving, at the remote control module, a user selection of one of the plurality of non-obstructed incremental movement directions; and

controlling, by the remote control module, the remote aerial device to move to a new non-obstructed location by a movement of the fixed distance in the direction of the selected incremental movement direction.

2. The computer-implemented method of claim 1, wherein:

each non-obstructed incremental movement direction is selected from a finite set of predetermined movements of predetermined distances in limited directions relative to the orientation of the remote aerial device that form a specific movement pattern for the remote aerial device; and

the plurality of non-obstructed incremental movement directions are selected from the finite set by removing elements of the finite set that indicate predetermined movements associated with locations determined to be obstructed by the one or more proximal obstructions based upon the sensor data.

3. The computer-implemented method of claim 1, further comprising:

presenting, by a display of the remote control module, indications of the plurality of non-obstructed incremental movement directions to the user.

4. The computer-implemented method of claim 1, wherein controlling the remote aerial device to move to the initial target location comprises receiving, via a user interface, an indication of global positioning system (GPS) coordinates and an altitude from the user.

5. The computer-implemented method of claim 1, wherein the fixed distance of each non-obstructed incremental movement direction is determined in advance of remote aerial device control based upon a type of the object to be inspected.

6. The computer-implemented method of claim 1, further comprising:

controlling, by the remote control module, the remote aerial device to hover at the initial target location until the user selection of one of the plurality of non-obstructed incremental movement directions is received; and

controlling, by the remote control module, the remote aerial device to hover at the new target location until an additional instruction is received.

7. The computer-implemented method of claim 1, further comprising:

causing, by the remote control module, a camera of the remote aerial device to capture images of the object after moving to the initial target location.

8. A computer system for remotely controlling a remote aerial device for up-close subject inspection, comprising:

a remote aerial device having one or more sensors; and

a remote control module having one or more processors communicatively connected to the remote aerial device and a program memory storing executable instructions that, when executed by the one or more processors, cause the remote control module to;

control the remote aerial device to move to an initial target location in proximity to a portion of the object to be inspected;

receive proximal sensor data from the one or more sensors of the remote aerial device, the proximal sensor data indicating one or more proximal obstructions located within a distance threshold of the remote aerial device;

a plurality of non-obstructed incremental movement directions based at least in part upon the proximal sensor data, wherein each non-obstructed incremental movement direction specifies a potential movement for the remote aerial device of a fixed distance in a direction that has been determined by the remote control module to be unobstructed;

receive a user selection of one of the plurality of non-obstructed incremental movement directions; and

the remote aerial device to move to a new non-obstructed location by a movement of the fixed distance in the direction of the selected incremental movement direction.

9. The computer system of claim 8, wherein:

each non-obstructed incremental movement direction is selected from a finite set of predetermined movements of predetermined distances in limited directions relative to the orientation of the remote aerial device that form a specific movement pattern for the remote aerial device; and

the plurality of non-obstructed incremental movement directions are selected from the finite set by removing elements of the finite set that indicate predetermined movements associated with locations determined to be obstructed by the one or more proximal obstructions based upon the sensor data.

10. The computer system of claim 8, wherein:

the remote control module further comprises a display; and

the executable instructions further cause the remote control module to present indications of the plurality of non-obstructed incremental movement directions to the user via the display.

11. The computer system of claim 8, wherein the executable instructions that cause the remote control module to control the remote aerial device to move to the initial target location further cause the remote control module to receive an indication of global positioning system (GPS) coordinates and an altitude via a user interface.

12. The computer system of claim 8, wherein the fixed distance of each non-obstructed incremental movement direction is determined in advance of remote aerial device control based upon a type of the object to be inspected.

13. The computer system of claim 8, wherein the executable instructions further cause the remote control module to:

control the remote aerial device to hover at the initial target location until the user selection of one of the plurality of non-obstructed incremental movement directions is received; and

control the remote aerial device to hover at the new target location until an additional instruction is received.

14. The computer system of claim 8, wherein:

the remote aerial vehicle includes a camera; and

the executable instructions further cause the remote control module to cause the camera to capture images of the object after moving to the initial target location.

15. A tangible, non-transitory computer-readable medium storing executable instructions for remotely controlling a remote aerial device for up-close subject inspection that, when executed by one or more processors of a computer system, cause the computer system to:

control the remote aerial device to move to an initial target location in proximity to a portion of the object to be inspected;

receive proximal sensor data from one or more sensors of the remote aerial device, the proximal sensor data indicating one or more proximal obstructions located within a distance threshold of the remote aerial device;

determine a plurality of non-obstructed incremental movement directions based at least in part upon the proximal sensor data, wherein each non-obstructed incremental movement direction specifies a potential movement for the remote aerial device of a fixed distance in a direction that has been determined by the remote control module to be unobstructed;

receive a user selection of one of the plurality of non-obstructed incremental movement directions; and

control the remote aerial device to move to a new non-obstructed location by a movement of the fixed distance in the direction of the selected incremental movement direction.

16. The tangible, non-transitory computer-readable medium of claim 15, wherein:

each non-obstructed incremental movement direction is selected from a finite set of predetermined movements of predetermined distances in limited directions relative to the orientation of the remote aerial device that form a specific movement pattern for the remote aerial device; and

the plurality of non-obstructed incremental movement directions are selected from the finite set by removing elements of the finite set that indicate predetermined movements associated with locations determined to be obstructed by the one or more proximal obstructions based upon the sensor data.

17. The tangible, non-transitory computer-readable medium of claim 15, further storing instructions that cause the computer system to:

present indications of the plurality of non-obstructed incremental movement directions to the user via the display.

18. The tangible, non-transitory computer-readable medium of claim 15, wherein the executable instructions that cause the computer system to control the remote aerial device to move to the initial target location further cause the computer system to receive an indication of global positioning system (GPS) coordinates and an altitude via a user interface.

19. The tangible, non-transitory computer-readable medium of claim 15, wherein the fixed distance of each non-obstructed incremental movement direction is determined in advance of remote aerial device control based upon a type of the object to be inspected.

20. The tangible, non-transitory computer-readable medium of claim 15, further storing instructions that cause the computer system to:

control the remote aerial device to hover at the initial target location until the user selection of one of the plurality of non-obstructed incremental movement directions is received; and

control the remote aerial device to hover at the new target location until an additional instruction is received.