Optical state estimation and simulation environment for unmanned aerial vehicles

• US 8,942,964 B2
• Filed: 06/08/2010
• Issued: 01/27/2015
• Est. Priority Date: 06/08/2010
• Status: Active Grant

• Current Assignee: Southwest Research Institute
• Original Assignee: Southwest Research Institute
• Inventors: McWilliams, George T. III, Alley, Kevin J., Mentzer, Christopher I.
• Primary Examiner(s): Thangavelu, Kandasamy
• Assistant Examiner(s): PIERRE LOUIS, ANDRE
• Application Number:
  US12/796,429
• Publication Number:
  US 20110301925A1
• Time in Patent Office: 1,694 Days
• Field of Search: 703/6, 703/7, 703/8, 434/11, 434/14, 434/27, 434/30, 434/37, 434/38, 701/11, 701/23, 701/24, 701/30.9, 701/32.3, 701/32.4, 701/400, 701/436
• US Class Current: 703/8
• CPC Class Codes: G01C 21/00 Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 measuring distance traversed on the ground by a vehicle G01C22/00; control of position, course, altitude or attitude of vehicles G05D1/00; traffic control systems for road vehicles involving transmission of navigation instructions to the vehicle G08G1/0968
• CPC Class Codes: G09B 9/08 for teaching control of aircraft, e.g. Link trainer
• CPC Class Codes: G09B 9/48 a model being viewed and manoeuvred from a remote point

Abstract

The present disclosure relates to a method and system for simulating an unmanned aerial vehicle (UAV) and simulating an environment in which the UAV may be flying. A plurality of visual sensors, e.g., cameras, positioned on the UAV, may be simulated. A UAV simulator is configured to simulate the UAV and a graphical simulator is configured to simulate the environment. The UAV simulator may be configured to: estimate pitch, roll and/or groundspeed based, at least in part, on outputs from the visual sensors, determine a position and/or an orientation of the simulated UAV in the simulated environment based, at least in part, on the estimate(s) and provide the position and/or orientation to the graphical simulator. The graphical simulator may be configured to display the simulated UAV at the position and/or orientation in the simulated environment and/or to display the simulated camera view(s).

Claims

1. A method for simulating an unmanned aerial vehicle (UAV) in a simulated environment, said method comprising:

(1) receiving a first simulated view of said simulated environment from a graphical simulator configured to simulate at least one visual sensor positioned on said simulated UAV;

(2) estimating pitch, roll and groundspeed associated with said simulated UAV, based at least in part, on said first simulated view from said simulated visual sensor, using a first processor, wherein said pitch and said roll are based on an angle of said simulated UAV relative to a simulated horizon in said simulated environment, said simulated horizon determined by a Hough transformation of an output of a Sobel edge detection filter applied to said first simulated view, and wherein said groundspeed is based on a block matching algorithm configured to match a region of said first simulated view to a region of a previous simulated view and to estimate motion based on a difference in position of said region of said first simulated view and said region of said previous simulated view, wherein a size of said region is based on a maximum groundspeed, altitude and field of view of said simulated visual sensor;

(3) comparing said estimated pitch, roll and groundspeed, to data generated by a simulated global positioning system (GPS), said comparison to evaluate said simulated visual sensor;

(4) determining at least one of a position and an orientation of said simulated UAV, based at least in part, on said estimated pitch, roll and groundspeed, using said first processor;

(5) simulating said UAV in said simulated environment at said at least one of said position and said orientation, using a second processor;

(6) generating a second simulated view based on at least one of said position and said orientation; and

(7) repeating steps 1 through 5 using said second simulated view.

2. The method of claim 1 wherein said first simulated view from said at least one simulated visual sensor is generated using said second processor based, at least in part, on said at least one of said position and said orientation of said simulated UAV in said simulated environment.

3. The method of claim 1 wherein said first simulated view from said at least one simulated visual sensor is generated using at least one other processor based, at least in part, on said at least one of said position and said orientation of said simulated UAV in said simulated environment.

4. The method of claim 1 wherein the first processor is included in a first computer system and the second processor is included in a second computer system.

5. The method of claim 1 further comprising displaying said simulated UAV in said simulated environment.

6. The method of claim 1 further comprising displaying at least one simulated view of said at least one simulated visual sensor.

7. A system comprising:

a first processor configured to;

simulate an unmanned aerial vehicle (UAV),

estimate pitch, roll and groundspeed associated with said UAV, based at least in part, on a first simulated view from a graphical simulator configured to simulate at least one visual sensor, wherein said pitch and said roll are based on an angle of said simulated UAV relative to a simulated horizon, said simulated horizon determined by a Hough transformation of an output of a Sobel edge detection filter applied to said first simulated view, and wherein said groundspeed is based on a block matching algorithm configured to match a region of said first simulated view to a region of a previous simulated view and to estimate motion based on a difference in position of said region of said first simulated view and said region of said previous simulated view, wherein a size of said region is based on a maximum groundspeed, altitude and field of view of said simulated visual sensor,

compare said estimated pitch, roll and groundspeed, to data generated by a simulated global positioning system (GPS), said comparison to evaluate said simulated visual sensor, and

determine at least one of a position and an orientation of said UAV, based at least in part, on said estimated pitch, roll and groundspeed; and

a second processor configured to;

simulate an environment,

simulate said UAV in said environment at said at least one of said position and said orientation;

generate a second simulated view based on at least one of said position and said orientation; and

provide said second simulated view to said first processor to repeat said estimation of said pitch, roll and groundspeed using said second simulated view.

8. The system of claim 7 wherein the first processor is included in a first computer system and the second processor is included in a second computer system.

9. The system of claim 7 wherein said second processor is further configured to display said simulated UAV in said simulated environment.

10. The system of claim 7 wherein said second processor is further configured to display at least one simulated view of said at least one simulated visual sensor.

11. An article comprising a non-transitory storage medium having stored thereon instructions that when executed by a machine result in the following operations for simulating an unmanned aerial vehicle (UAV) in a simulated environment:

(1) receiving a first simulated view of said simulated environment from a graphical simulator configured to simulate at least one visual sensor positioned on said simulated UAV;

(2) estimating pitch, roll and groundspeed associated with said simulated UAV, based at least in part, on said first simulated view from said simulated visual sensor, using a first processor, wherein said pitch and said roll are based on an angle of said simulated UAV relative to a simulated horizon in said simulated environment, said simulated horizon determined by a Hough transformation of an output of a Sobel edge detection filter applied to said first simulated view, and wherein said groundspeed is based on a block matching algorithm configured to match a region of said first simulated view to a region of a previous simulated view and to estimate motion based on a difference in position of said region of said first simulated view and said region of said previous simulated view, wherein a size of said region is based on a maximum groundspeed, altitude and field of view of said simulated visual sensor;

(3) comparing said estimated pitch, roll and groundspeed, to data generated by a simulated global positioning system (GPS), said comparison to evaluate said simulated visual sensor;

(4) determining at least one of a position and an orientation of said simulated UAV, based at least in part, on said estimated pitch, roll and groundspeed, using said first processor;

(5) simulating said UAV in said simulated environment at said at least one of said position and said orientation, using a second processor;

(6) generating a second simulated view based on at least one of said position and said orientation; and

(7) repeating steps 1 through 5 using said second simulated view.

12. The article of claim 11 wherein said simulated view from said at least one simulated visual sensor is generated using said second processor based, at least in part, on said at least one of said position and said orientation of said simulated UAV in said simulated environment.

13. The article of claim 11 wherein said simulated view from said at least one simulated visual sensor is generated using at least one other processor based, at least in part, on said at least one of said position and said orientation of said simulated UAV in said simulated environment.

14. The article of claim 11 wherein the first processor is included in a first computer system and the second processor is included in a second computer system.

15. The article of claim 11 wherein the instructions further result in the following operations:

displaying said simulated UAV in said simulated environment.

16. The article of claim 11 wherein the instructions further result in the following operations:

displaying at least one simulated view of said at least one simulated visual sensor.