SYSTEMS AND METHODS FOR RELIABLE RELATIVE NAVIGATION AND AUTONOMOUS FOLLOWING BETWEEN UNMANNED AERIAL VEHICLE AND A TARGET OBJECT

• US 20160304198A1
• Filed: 12/03/2014
• Published: 10/20/2016
• Est. Priority Date: 12/03/2014
• Status: Active Grant

• Current Assignee: Wing Aviation LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Jourdan, Damien Bruno
• Granted Patent: US 9,896,202 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/10 Rotorcrafts
• CPC Class Codes: B64U 10/14 with four distinct rotor axes, e.g. quadcopters
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2101/31 for surveillance
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/042 specially adapted for aircraft
• CPC Class Codes: G05D 1/102 specially adapted for vertical take-off of aircraft
• CPC Class Codes: G08G 5/0069 specially adapted for an unmanned aircraft
• CPC Class Codes: G08G 5/0086 for monitoring terrain radar or analogous systems specially adapted for terrain avoidance G01S13/935
• CPC Class Codes: G08G 5/045 Navigation or guidance aids, e.g. determination of anti-collision manoeuvers radar or analogous systems specially adapted for anti-collision between aircraft G01S13/933

Abstract

A method for navigating an airborne device relative to a target comprises detecting, at an optical detector on the airborne device, an optical signal generated by one or more LEDs on the target. The method also comprises comparing, by a processor on the airborne device, the detected optical signal with a previously-detected optical signal. The method further comprises determining, by the processor based on the comparison, a change in location of at least one of the airborne device or the target. The method also comprises adjusting a position of the airborne device based on the determined change in location. The method also comprises predicting, by the processor, a movement of the target based on information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude of the target, wherein the position of the airborne device is adjusted based on the predicted movement of the target. The method also comprises detecting an obstacle in a flight path associated with the airborne device and adjusting a position of the airborne device is further based, at least in part, on detected obstacle information.

Claims

1. A method for navigating an airborne device relative to a target, comprising:

detecting, at an signal detector associated with the airborne device, at least one signal generated by a signal emitter associated with the target;

correlating, by a processor associated with the airborne device, the detected signal with a previously-detected signal;

determining, by the processor based on the comparison, a change in location of at least one of the airborne device or the target; and

adjusting a position of the airborne device based, at least in part, on the determined change in location.

2. The method of claim 1, wherein determining the change in location of the at least one airborne device or the target includes calculating a range between the airborne device and the target.

3. The method of claim 2, wherein adjusting the position of the airborne device includes adjusting the position of the airborne device to maintain the range between the airborne device and the target to within a threshold range value.

4. The method of claim 1, where relative motion of the airborne device relative to the target device is used to estimate relative distance between the target and the airborne devices.

5. The method of claim 1, further comprising receiving, at the processor associated with the airborne device from at least one sensor located on-board the target, information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target.

6. The method of claim 5, further comprising:

predicting, by the processor associated with the airborne device based on the received information, a movement of the target; and

wherein adjusting the position of the airborne device is further based, at least in part, on the predicted movement of the target.

7. The method of claim 5, wherein receiving information indicative of information indicative of at least one of a position, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving orientation information from at least one orientation sensor located on-board the target.

8. The method of claim 5, wherein receiving information indicative of information indicative of at least one of a position, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving acceleration and rotational information from at least one inertial measurement unit located on-board the target.

9. The method of claim 5, wherein receiving information indicative of information indicative of at least one of a position, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving barometric pressure information from at least one barometric sensor located on-board the target.

10. The method of claim 9, where a difference in static barometric pressure between target and airborne devices is used to estimate relative distance between the target device and the airborne device.

11. The method of claim 5, wherein receiving information indicative of information indicative of at least one of a position, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving directional information from at least magnetometer located on-board the target.

12. The method of claim 5, wherein receiving information indicative of information indicative of at least one of a position, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving GPS coordinates and velocities from at least one GPS module located on-board the target.

13. The method of claim 1, wherein the source emitter includes a plurality of LEDs, and wherein detecting at least one signal generated by a signal emitter associated with the target includes detecting a plurality of optical signals, each of the plurality of optical signals generated by a respective one of a plurality of LEDs located on-board the target.

14. The method of claim 13, wherein comparing the detected optical signal associated with a previously-detected optical signal include comparing a pattern defined by the plurality of detected optical signals with a pattern associated with the previously-detected optical signal.

15. The method of claim 14, wherein determining the change in location is based on the comparison between the pattern defined by the plurality of detected optical signals with the pattern associated with the previously-detected optical signal.

16. The method of claim 13, wherein detecting at least one signal is performed by an optical detector that includes an image sensor associated with a digital camera, the method further comprising transmitting, by the processor associated with the airborne device, a control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target.

17. The method of claim 16, wherein the transmitting of the control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target is responsive to a detection of suspected ambient light sources by the image sensor that are not associated with the target.

18. The method of claim 16, wherein the optical detector includes an image sensor associated with a digital camera, the method further comprising adjusting, by the processor associated with the airborne device, an exposure of the digital camera based on an intensity or flashing pattern of the at least one optical signal generated by the LED associated with the target.

19. The method of claim 13, wherein the plurality of LEDs are placed on the target device, the method further comprising estimating one or more of relative distance, relative velocity, a relative pitch, a relative roll, a relative heading, and relative inertial biases.

20. The method of claim 1, wherein the signal includes a signal comprising one or more of one or more of a thermal signal, light in a visible wavelength, light in an infrared (IR) wavelength, light in a near-IR wavelength, or light in an ultraviolet (UV) wavelength.

21. The system of claim 1, further comprising wherein the airborne device further comprises a proximity sensor and the optical detector includes an image sensor associated with a digital camera, wherein the method further comprises:

detecting an obstacle in a flight path associated with the airborne device based on obstacle information detected by the proximity sensor and the optical detector; and

wherein adjusting a position of the airborne device is further based, at least in part, on the obstacle information.

22. The method of claim 21, wherein adjusting a position of the airborne device includes adjusting, based on the detection of an obstacle, the position of the airborne device to follow closely behind the target as it travels along an obstacle-free path, and wherein the position of the airborne is returned to the flight path when the obstacle is no longer detected.

23. A system for aerial monitoring of a target, comprising:

a target device coupled to a target, the target device comprising;

at least one LED configured to generate an optical signal;

an airborne device coupled to an airborne vehicle and in data communication with the target device, the airborne device comprising;

an optical detector configured to detect the optical signal generated by the target device; and

a processor communicatively coupled to the optical detector and configured to;

compare the detected optical signal with a previously-detected optical signal;

determine a change in location of at least one of the airborne device or the target; and

generate a control signal for adjusting a position of the airborne device based, at least in part, on the determined change in location.

24. The system of claim 23, wherein the airborne vehicle further comprises at least one motor that is communicatively coupled to the processor of the airborne device and configured to operate in response to the control signal received by the processor.

25. The system of claim 23, wherein the processor is configured to determine the change in location of the at least one airborne device or the target includes calculating a range between the airborne device and the target.

26. The system of claim 25, wherein the processor is configured to generate a control signal for adjusting a position of the airborne device to maintain the range between the airborne device and the target to within a threshold range value.

27. The system of claim 23, wherein the target device further comprises at least one sensor configured to measure information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target.

28. The system of claim 27, wherein the processor is further configured to:

receive, from the target device, the information indicative of the at least one of a position, an orientation, an acceleration, a velocity, or an altitude associated with the target;

predict, based on the received information, a movement of the target; and

wherein the control signal for adjusting the position of the airborne device is further based, at least in part, on the predicted movement of the target.

29. The system of claim 27, wherein the at least one sensor is an orientation sensor configured to measure information indicative of orientation information of the target.

30. The system of claim 27, wherein the at least one sensor is an inertial measurement unit configured to measure acceleration and rotational information of the target.

31. The system of claim 27, wherein the at least one sensor is a barometric sensor configured to measure barometric pressure information of the target.

32. The system of claim 27, wherein the at least one sensor includes at least one magnetometer configured to measure directional information of the target.

33. The system of claim 27, wherein the at least one sensor includes a GPS module configured to measure GPS coordinates and velocities of the target.

34. The system of claim 23, wherein the at least one LED includes a plurality of LEDs, each of which is configured to generate a respective optical signal.

35. The system of claim 23, wherein the optical detector is further configured to capture images in close succession with one another based, at least in part, on the flash pattern of the at least one LED, and wherein the processor is further configured to compensate the images for motion that occurred between successive images, wherein the resulting images are compared to extract the location of the LED(s) robustly, regardless of other disturbance light present in one or more of the images.

36. The system of claim 35, wherein the processor is configured to compare the detected optical signal associated with a previously-detected optical signal by comparing the pattern defined by the plurality of detected optical signals with a pattern associated with the previously-detected optical signal.

37. The system of claim 36, wherein the processor is configured to determine the change in location based on the comparison between the pattern defined by the plurality of detected optical signals and the pattern associated with the previously-detected optical signal.

38. The system of claim 23, wherein the optical detector includes an image sensor associated with a digital camera, wherein the processor is further configured to transmit a control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target.

39. The system of claim 38, wherein the transmitting of the control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target is responsive to a detection of suspected ambient light sources by the image sensor that are not associated with the target.

40. The system of claim 23, wherein the optical detector includes an image sensor associated with a digital camera, wherein the processor is further configured to adjust an exposure of the digital camera based on an intensity or flashing pattern of the at least one optical signal generated by the LED associated with the target.

41. The system of claim 23, wherein the optical signal includes a signal comprising one or more of light that is in one or more of a thermal signal, a visible wavelength, an infrared (IR) wavelength, a near-IR wavelength, or ultraviolet (UV) wavelength.

42. The system of claim 23, wherein the airborne device further comprises a proximity sensor and the optical detector includes an image sensor associated with a digital camera, wherein the processor is further configured to:

detect obstacles in a flight path associated with the airborne device based on obstacle information detected by the proximity sensor and the optical detector; and

wherein the control signal for adjusting a position of the airborne device further based, at least in part, on the obstacle information.

43. The system of claim 23, wherein the optical detector includes an image sensor associated with a digital camera, the method further comprising transmitting, by the processor associated with the airborne device, a control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target.

44. The system of claim 43, wherein the transmitting of the control signal for at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target is responsive to a detection of suspected ambient light sources by the image sensor that are not associated with the target.

45. A method for aerial tracking of a target, comprising:

detecting, at an optical detector associated with the airborne device, an optical signal pattern generated by a plurality of LEDs associated with the target;

comparing, by a processor associated with the airborne device, the detected pattern with a baseline pattern;

determining, by the processor based on the comparison, a change in location of at least one of the airborne device or the target;

receiving, at the processor associated with the airborne device from at least one sensor located on-board the target, information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target;

predicting, by the processor associated with the airborne device based on the received information, a future movement of the target; and

adjusting a position of the airborne device based, at least in part, on the determined change in location and the predicted movement of the target.

46. The method of claim 45, wherein determining the change in location of the at least one airborne device or the target includes calculating a range between the airborne device and the target.

47. The method of claim 45, wherein adjusting the position of the airborne device includes adjusting the position of the airborne device to maintain the range between the airborne device and the target to within a threshold range value.

48. The method of claim 45, wherein the baseline pattern is a pattern detected at an initial calibration or a pattern that was previously-detected by the processor.

49. The method of claim 45, wherein receiving information indicative of information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving orientation information from at least one orientation sensor located on-board the target.

50. The method of claim 45, wherein receiving information indicative of information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving acceleration and rotational information from at least one inertial measurement unit located on-board the target.

51. The method of claim 45, wherein receiving information indicative of information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving barometric pressure information from at least one barometric sensor located on-board the target.

52. The method of claim 45, wherein receiving information indicative of information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving directional information from at least magnetometer located on-board the target.

53. The method of claim 45, wherein receiving information indicative of information indicative of at least one of a position, a rotation, an orientation, an acceleration, a velocity, or an altitude associated with the target includes receiving GPS coordinates and velocities from at least one GPS module located on-board the target.

54. The method of claim 45, wherein the optical detector includes an image sensor associated with a digital camera, the method further comprising transmitting, by the processor associated with the airborne device, a control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target.

55. The method of claim 54, wherein the transmitting of the control signal for adjusting at least one of an intensity, frequency, or flashing pattern of the LED device associated with the target is responsive to a detection of suspected ambient light sources by the image sensor that are not associated with the target.

56. The method of claim 45, wherein the optical detector includes an image sensor associated with a digital camera, the method further comprising adjusting, by the processor associated with the airborne device, an exposure of the digital camera based on an intensity or flashing pattern of the at least one optical signal generated by the LED associated with the target.

57. The method of claim 45, wherein the optical signal includes a signal comprising one or more of light that is in one or more of a thermal signal, a visible wavelength, an infrared (IR) wavelength, a near-IR wavelength, or ultraviolet (UV) wavelength.

58. The method of claim 57, further comprising wherein the airborne device further comprises a proximity sensor and the optical detector includes an image sensor associated with a digital camera, wherein the method further comprises:

detecting an obstacle in a flight path associated with the airborne device based on obstacle information detected by the proximity sensor and the optical detector; and

wherein adjusting a position of the airborne device is further based, at least in part, on the obstacle information.