VISION-BASED AUTOMATED LANDING SYSTEM FOR UNMANNED AERIAL VEHICLES

• US 20090306840A1
• Filed: 04/07/2009
• Published: 12/10/2009
• Est. Priority Date: 04/08/2008
• Status: Abandoned Application

• Current Assignee: 21st Century Screens Incorporated
• Original Assignee: 21st Century Screens Incorporated
• Inventors: O'Hara, Stephen V., BLENKHORN, Kevin P.
• Application Number:
  US12/419,975
• Publication Number:
  US 20090306840A1
• Time in Patent Office: Days
• US Class Current: 701/16
• CPC Class Codes: G05D 1/0676 specially adapted for landing

Abstract

The invention relates generally to the control and landing of unmanned aerial vehicles. More specifically, the invention relates to systems, methods, devices, and computer readable media for landing unmanned aerial vehicles using sensor input and image processing techniques.

Claims

1. A vision-based automated system for landing unmanned aerial vehicles, said system comprising:

(i) one or more unmanned aerial vehicles;

(ii) one or more targets, of known geometry, each of said one or more targets positioned at an intended landing location;

wherein said vehicle is attempting to land on at least one of said one or more targets;

(iii) at least one sensor coupled to each of said one or more vehicles, wherein said at least one sensor is aligned with the direction of movement of said one or more vehicles, and wherein said at least one sensor captures one or more images in the direction of movement of said one or more vehicles and is capable of detecting said one or more targets;

(iv) at least processor-based device, wherein said processor-based device determines the visual distortion of at least one of said one or more targets visible in said one or more images as a function of said one or more vehicles'"'"' current position, such that a current glideslope and a current lineup angle can be determined for said one or more vehicles;

and wherein said processor-based device can adjust said current glideslope and said current lineup angle of said one or more vehicles to an intended glideslope and an intended lineup angle.

2. The system of claim 1, wherein said geometry is planar geometry.

3. The system of claim 1, wherein said system sends said current glideslope and said current lineup angle to an autopilot control loop to adjust said current glideslope and said current lineup angle.

4. The system of claim 1, wherein at least one of said one or more targets is a bilaterally symmetric cross.

5. The system of claim 1, wherein at least one of said one or more targets is a building rooftop.

6. The system of claim 1, wherein at least one of said one or more targets is painted on a building.

7. The system of claim 4, wherein said bilaterally symmetric cross comprises a horizontal arm and a vertical arm.

8. The system of claim 7, wherein the length of said vertical arm is greater than the length of said horizontal arm.

9. The system of claim 8, wherein the length of said vertical arm is between two and 20 times the length of said horizontal arm.

10. The system of claim 9, wherein the length of said vertical arm is between five and 15 times the length of said horizontal arm.

11. The system of claim 10, wherein the length of said vertical arm is five times the length of said horizontal arm.

12. The system of claim 10, wherein the length of said vertical arm is ten times the length of said horizontal arm.

13. The system of claim 7, wherein one end of said vertical arm is pre-designated as an approach arm by a special marker.

14. The system of claim 13, wherein said special marker is a stripe positioned within the outline of said vertical arm.

15. The system of claim 13, wherein said special marker is rectangular.

16. The system of claim 13, wherein said special marker is any color which is a different color than said target.

17. The system of claim 16, wherein said special marker is green.

18. The system of claim 1, wherein at least one of said one or more targets is red.

19. The system of claim 1, wherein at least one of said one or more targets is a runway.

20. The system of claim 1, wherein at least one of said one or more targets is a taxiway.

21. The system of claim 1, wherein at least one of said one or more targets is a building.

22. The system of claim 1, wherein at least one of said one or more targets is the entire airfield.

23. The system of claim 1, wherein at least one of said one or more targets is permanently placed on a runway.

24. The system of claim 1, wherein at least one of said one or more targets is fixed on a portable mat.

25. The system of claim 1, wherein at least one of said one or more targets is coupled to one or more lights, wherein said lights are positioned on signature corners of said at least one of said one or more targets.

26. The system of claim 25, wherein said lights are chemical lights.

27. The system of claim 25, wherein said signature corners are non-collinear signature corners.

28. The system of claim 27, wherein said non-collinear signature corners establish a geometric two-dimensional plane of said at least one of said one or more targets.

29. The system of claim 1, wherein at least one of said one or more targets is coupled to one or more infrared strobe lights, wherein said infrared strobe lights are positioned on signature corners of said at least one of said one or more targets.

30. The system of claim 1, wherein said sensor is a camera.

31. The system of claim 30, wherein said camera is a single-lens reflex camera.

32. The system of claim 30, wherein said camera is a digital camera.

33. The system of claim 30, wherein said camera is an infrared camera.

34. A method for landing an unmanned aerial vehicle, comprising:

(i) capturing an image in the direction of movement of said unmanned aerial vehicle;

(ii) analyzing said image to determine whether said image includes a possible target;

(iii) assessing the dimensions of said possible target and comparing said dimensions of said possible target to the known dimensions of an actual target, to determine a current glideslope and a current lineup angle; and

(iv) forcing said vehicle to adjust its altitude, using said current glideslope, and to adjust its alignment, using said current lineup angle.

35. The method of claim 34, wherein said method sends said current glideslope and said current lineup angle to an autopilot control loop to force said vehicle to adjust its said current glideslope and said current lineup angle.

36. The method of claim 35, wherein said capturing is accomplished using one or more cameras.

37. The method of claim 36, wherein at least one of said one or more cameras is a single-lens reflex camera.

38. The method of claim 36, wherein at least one of said one or more cameras is a digital camera.

39. The method of claim 36, wherein at least one of said one or more cameras is an infrared camera.

40. The method of claim 34, wherein said step of analyzing to determine whether said image includes a possible target is performed by a human operator.

41. The method of claim 34, wherein said step of analyzing to determine whether said image includes a possible target is performed by image processing.

42. The method of claim 41, wherein said image processing comprises:

(i) identifying said possible target by identifying a continuous region of the known color of said actual target;

(ii) deriving the outline of said possible target;

(iii) selecting at least three signature corners of said possible target;

(iv) comparing said at least three signature corners of said possible target to the known signature corners of said actual target; and

(v) determining whether said three signature corners of said continuous region substantially match said signature corners of said actual target.

43. The method of claim 42, wherein the outline of said possible target is identified by:

(i) converting said image into a binary mask, such that said continuous region is represented in said binary mask by a value which is the inverse of the value of all other colors represented in said binary mask;

(ii) using basic morphology operations to smooth the silhouette of said continuous region to form a more precise outline.

44. The method of claim 42, wherein said at least three signature corners are non-collinear signature corners.

45. The method of claim 44, wherein said at least three non-collinear signature corners establish a geometric two-dimensional plane of said possible target.

46. The method of claim 42, wherein one or more signature corners of said possible target is derived from a special marker.

47. The method of claim 42, wherein said step of analyzing to determine whether said possible target is an actual target is further verified by a human operator.

48. The method of claim 34, wherein said current glideslope is between 2 and 45 degrees above the horizon.

49. The method of claim 34, wherein said current glideslope is between 3 and 10 degrees above the horizon.

50. The method of claim 34, wherein the entire series of steps (i) through (iv) is repeated one or more times until said vehicle has landed.

51. The method of claim 50, wherein said capturing is accomplished using at least one camera.

52. The method of claim 51, wherein said camera is a single-lens reflex camera.

53. The method of claim 51, wherein said camera is a digital camera.

54. The method of claim 51, wherein said camera is an infrared camera.

55. The method of claim 51, wherein said current glideslope is determined as a function of the apparent height-to-width ratio of said target, as captured by said camera in the direction of movement of said vehicle.

56. The method of claim 55, wherein said height-to-width ratio is related to the current glideslope by the equation H/W=PAR*(h/w)*sin(α

), wherein;

H=the apparent height of said target as captured in said image, measured in pixels;

W=the apparent width of said target as captured in said image, measured in pixels;

PAR=pixel aspect ratio of said electro-optic camera;

h=the actual height of said target;

w=the actual width of said target; and

α

=current glideslope of said vehicle'"'"'s current position.

57. The method of claim 56, wherein said current glideslope is determined by the equation α

=sin<sup>−

1</sup>(H*w/(PAR*h*W)), wherein;

H=the apparent height of said target as captured in said image, measured in pixels;

W=the apparent width of said target as captured in said image, measured in pixels;

PAR=pixel aspect ratio of said electro-optic camera;

h=the actual height of said target;

w=the actual width of said target; and

α

=current glideslope of said vehicle'"'"'s current position.

58. The method of claim 51, wherein said lineup angle is determined by solving the system of equations generated by applying the equation

$\left[\begin{array}{c}{S}_X\\ S_Y\\ S_Z\end{array}\right]$

[ cos 

( β

) - sin 

( β

) 0 sin 

( β

) cos 

( β

) 0 0 0 1 ] 

[ 1 0 0 0 cos 

( α

) - sin 

( α

) 0 sin 

( α

) cos 

( α

) ] = [ D X D Y 0 ] ,

wherein;

S_X, S_Y, S_Z=world coordinates for one signature corner of said target;

D_X, D_Y=unit vector of said signature corner of said target;

α

=said current glideslope; and

β

=said lineup angle,

to at least three signature corners of said target.

59. A method for preventing a vehicle from landing by executing a wave-off procedure, comprising:

(i) increasing the power of said vehicle;

(ii) forcing said vehicle to climb to a safe altitude; and

(iii) causing said vehicle to attempt another landing.

60. The method of claim 59, wherein said vehicle executes said wave-off procedure upon receipt of an instruction from a human operator.

61. The method of claim 59, wherein said vehicle executes said wave-off procedure upon the occurrence of one or more preprogrammed conditions.

62. The method of claim 61, wherein at least one of said one or more preprogrammed conditions is that said vehicle cannot sufficiently adjust its direction prior to an expected time to impact.

63. The method of claim 62, wherein said expected time to impact is determined by the equation

${\mathrm{TTI}}_1=\frac{w_2}{}$

( t 2 - t 1 ) w 2 - w 1 ,

wherein;

TTI₁=expected time to impact;

t₁=the time at which a first image is captured;

t₂=the time at which a subsequent image is captured;

w₁=the apparent width of said target as captured in said first image, measured in pixels; and

w₂=the apparent width of said target as captured in said subsequent image, measured in pixels.