System and method for controlling a remote aerial device for up-close inspection

• US 9,682,777 B2
• Filed: 04/18/2016
• Issued: 06/20/2017
• Est. Priority Date: 03/15/2013
• Status: Active Grant

• Current Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Original Assignee: State Farm Mutual Automobile Insurance Company (State Farm)
• Inventors: Tofte, Nathan L., Freeman, James M., Harvey, Brian N.
• Primary Examiner(s): SMITH, JELANI A
• Application Number:
  US15/131,220
• Publication Number:
  US 20160231739A1
• Time in Patent Office: 428 Days
• Field of Search: None
• CPC Class Codes: B64C 19/00 Aircraft control not otherwise provided for
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64D 47/08 Arrangements of cameras
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0011 associated with a remote control arrangement
• CPC Class Codes: G05D 1/0016 characterised by the operator's input device input arrangements for computing systems in general G06F3/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G05D 1/12 Target-seeking control
• CPC Class Codes: G06Q 40/00 Finance; Insurance; Tax strategies; Processing of corporate or income taxes
• CPC Class Codes: G06Q 40/08 Insurance
• CPC Class Codes: G06V 20/176 Urban or other man-made structures
• CPC Class Codes: H04N 7/183 for receiving images from a single remote source

Abstract

The method and system may be used to control the movement of a remote aerial device in an incremental step manner during a close inspection of an object or other subject matter. At the inspection location, a control module “stabilizes” the remote aerial device in a maintained, consistent hover while maintaining a close distance to the desired object. The control module may retrieve proximal sensor data that indicates possible nearby obstructions to the remote aerial device and may transmit the data to a remote control client. The remote control module may determine and display the possible one or more non-obstructed directions that the remote aerial device is capable of moving by an incremental distance. In response to receiving a selection of one of the directions, the remote control module may transmit the selection to the remote aerial device to indicate the next movement for the remote aerial device.

Claims

1. A computer-implemented method for remotely controlling a remote aerial device for up-close inspection of an object, the method comprising:

controlling the remote aerial device to move to a target location in proximity to a portion of the object to be inspected;

receiving, from one or more sensors of the remote aerial device, proximal sensor data indicating one or more directions toward one or more proximal obstructions, each proximal obstruction being located within a distance threshold of the remote aerial device;

determining a plurality of incremental movement directions based at least in part upon the proximal sensor data, wherein each incremental movement direction is selected from a finite set of predetermined movements of predetermined distances in limited directions relative to the orientation of the remote aerial device that form a specific movement pattern for the remote aerial device, and wherein each incremental movement direction indicates a potential non-obstructed movement of the remote aerial device, wherein the plurality of incremental movement directions are selected from the finite set by removing elements of the finite set that indicate predetermined movements associated with locations determined to be obstructed by the one or more proximal obstructions based upon the sensor data;

receiving, at a remote control module, a selection of one of the plurality of incremental movement directions; and

controlling the remote aerial device to move to a new target location based upon the selected incremental movement direction.

2. The computer-implemented method of claim 1, wherein each element in the finite set of predetermined movements represents a distinct predetermined movement that is either parallel or orthogonal to predetermined movements represented by every other element in the finite set.

3. The computer-implemented method of claim 1, wherein the finite set of predetermined movements contains six elements, representing movements of predetermined distances in the following directions relative to the orientation of the remote aerial device:

up, down, left, right, forward, and backward.

4. The computer-implemented method of claim 1, wherein the predetermined distances of the predetermined movements of the finite set are equivalent.

5. The computer-implemented method of claim 1, wherein the predetermined distances are determined in advance of remote aerial device control based upon a type of the object to be inspected.

6. The computer-implemented method of claim 1, further comprising:

stabilizing the remote aerial device to hover at the target location until the selection of one of the plurality of incremental movement directions is received; and

stabilizing the remote aerial device to hover at the new target location until an additional instruction is received.

7. A computer system for remotely controlling a remote aerial device for up-close subject inspection, comprising:

a remote aerial device having one or more sensors;

a remote control module having one or more processors; and

a program memory coupled to the one or more processors and storing executable instructions that, when executed by the one or more processors, cause the computer system to;

control the remote aerial device to move to a target location in proximity to a portion of the object to be inspected;

receive proximal sensor data indicating one or more directions toward one or more proximal obstructions from the one or more sensors of the remote aerial device, each proximal obstruction being located within a distance threshold of the remote aerial device;

determine a plurality of incremental movement directions based at least in part upon the proximal sensor data, wherein each incremental movement direction is selected from a finite set of predetermined movements of predetermined distances in limited directions relative to the orientation of the remote aerial device that form a specific movement pattern for the remote aerial device, and wherein each incremental movement direction indicates a potential non-obstructed movement of the remote aerial device, wherein the plurality of incremental movement directions are selected from the finite set by removing elements of the finite set that indicate predetermined movements associated with locations determined to be obstructed by the one or more proximal obstructions based upon the sensor data;

receive a selection of one of the plurality of incremental movement directions at the remote control module; and

controlling the remote aerial device to move to a new target location based upon the selected incremental movement direction.

8. The computer system of claim 7, wherein each element in the finite set of predetermined movements represents a distinct predetermined movement that is either parallel or orthogonal to predetermined movements represented by every other element in the finite set.

9. The computer system of claim 7, wherein the finite set of predetermined movements contains six elements, representing movements of predetermined distances in the following directions relative to the orientation of the remote aerial device:

up, down, left, right, forward, and backward.

10. The computer system of claim 7, wherein the predetermined distances of the predetermined movements of the finite set are equivalent.

11. The computer system of claim 7, wherein the predetermined distances are determined in advance of remote aerial device control based upon a type of the object to be inspected.

12. The computer system of claim 7, wherein the program memory further stores instructions that cause the computer system to:

stabilize the remote aerial device to hover at the target location until the selection of one of the plurality of incremental movement directions is received; and

stabilize the remote aerial device to hover at the new target location until an additional instruction is received.

13. A tangible, non-transitory computer-readable medium storing instructions for remotely controlling a remote aerial device for up-close subject inspection that, when executed by one or more processors of a computer system, cause the computer system to:

control the remote aerial device to move to a target location in proximity to a portion of the object to be inspected;

receive, from one or more sensors of the remote aerial device, proximal sensor data indicating one or more directions toward one or more proximal obstructions, each proximal obstruction being located within a distance threshold of the remote aerial device;

determine a plurality of incremental movement directions based at least in part upon the proximal sensor data, wherein each incremental movement direction is selected from a finite set of predetermined movements of predetermined distances in limited directions relative to the orientation of the remote aerial device that form a specific movement pattern for the remote aerial device, and wherein each incremental movement direction indicates a potential non-obstructed movement of the remote aerial device, wherein the plurality of incremental movement directions are selected from the finite set by removing elements of the finite set that indicate predetermined movements associated with locations determined to be obstructed by the one or more proximal obstructions based upon the sensor data;

receive, at a remote control module, a selection of one of the plurality of incremental movement directions; and

control the remote aerial device to move to a new target location based upon the selected incremental movement direction.

14. The tangible, non-transitory computer-readable medium of claim 13, wherein each element in the finite set of predetermined movements represents a distinct predetermined movement that is either parallel or orthogonal to predetermined movements represented by every other element in the finite set.

15. The tangible, non-transitory computer-readable medium of claim 13, wherein the finite set of predetermined movements contains six elements, representing movements of predetermined distances in the following directions relative to the orientation of the remote aerial device:

up, down, left, right, forward, and backward.

16. The tangible, non-transitory computer-readable medium of claim 13, wherein the predetermined distances of the predetermined movements of the finite set are equivalent.

17. The tangible, non-transitory computer-readable medium of claim 13, wherein the predetermined distances are determined in advance of remote aerial device control based upon a type of the object to be inspected.

18. The tangible, non-transitory computer-readable medium of claim 13, further storing instructions that cause the computer system to:

stabilize the remote aerial device to hover at the target location until the selection of one of the plurality of incremental movement directions is received; and

stabilize the remote aerial device to hover at the new target location until an additional instruction is received.