Drone flight control

• US 10,379,534 B2
• Filed: 06/17/2016
• Issued: 08/13/2019
• Est. Priority Date: 01/28/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Visser, Erik, Kim, Lae-Hoon, Bernal Castillo, Ricardo De Jesus, Zhang, Shuhua, Peri, Raghuveer
• Primary Examiner(s): Chace, Christian
• Assistant Examiner(s): King, Rodney P
• Application Number:
  US15/186,215
• Publication Number:
  US 20170220036A1
• Time in Patent Office: 1,152 Days
• Field of Search: 701 2
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: B64C 39/024 of the remote controlled vehicle type, i.e. RPV
• CPC Class Codes: B64U 10/13 Flying platforms
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: B64U 30/20 Rotors; Rotor supports
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0808 specially adapted for aircraft
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G08C 23/02 using infrasonic, sonic or ultrasonic waves
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: H04R 1/245 of microphones
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 29/004 for microphones H04R29/007 takes precedence

Abstract

A drone system and method. Audio signals are received via one or more microphones positioned relative to a location on a drone and one or more of the audio signals are identified as of interest. Flight characteristics of the drone are then controlled based on the audio signals that are of interest.

Claims

1. A method comprising:

receiving, at a processor, audio signals via a plurality of microphones positioned relative to a location on a drone, the audio signals including audio signals emanating from a source outside the drone as well as rotor noise or wind noise;

identifying, by the processor, audio signals that are of interest from the received audio signals; and

controlling, by the processor, flight characteristics of the drone based on the audio signals that are of interest such that a signal-to-noise ratio of the audio signals that are of interest is increased.

2. The method of claim 1, wherein identifying audio signals that are of interest includes selecting audio signals received from a target of interest.

3. The method of claim 2,

wherein controlling flight characteristics of the drone includes positioning the drone relative to the rotor noise or the wind noise and to the target of interest to better capture audio signals from the target of interest in the presence of the audio signals from the rotor noise or the wind noise.

4. The method of claim 2, wherein the selected audio signals are audio signals received from an object being tracked, and

wherein controlling flight characteristics of the drone includes determining a direction of arrival (DOA) of one or more of the selected audio signals and positioning the drone based on the DOA of the one or more selected audio signals.

5. The method of claim 2,

wherein the selected audio signals are audio signals received from an object being tracked, and

wherein controlling flight characteristics of the drone includes determining a direction of arrival (DOA) of one or more of the selected audio signals and positioning the drone relative to the rotor noise or the wind noise and to the target of interest to separate the DOA of the audio signals from the object being tracked from the DOA of the audio signals from the rotor noise or the wind noise.

6. The method of claim 5, wherein the drone includes a rotor that produces the rotor noise and wherein one or more of the plurality of microphones are positioned relative to the rotor, wherein positioning the drone includes orienting the microphones and the rotor relative to the DOA of the selected audio signals.

7. The method of claim 5, wherein the drone includes a rotor that produces the rotor noise and wherein one or more of the plurality of microphones are positioned relative to the rotor, wherein positioning the drone includes moving the microphones relative to the rotor.

8. The method of claim 5, wherein the drone includes a rotor that produces the rotor noise and wherein one or more of the plurality of microphones are positioned relative to the rotor, wherein positioning the drone includes enhancing capture of the selected audio signals via beam forming.

9. The method of claim 2, wherein the selected audio signals are audio signals received from an object being tracked, and

wherein controlling flight characteristics of the drone includes following the object being tracked based on the selected audio signals.

10. The method of claim 2, wherein receiving audio signals includes positioning two or more of the plurality of microphones to perform beam forming on the selected audio signals.

11. The method of claim 10, wherein controlling flight characteristics of the drone includes following the target of interest based on the selected audio signals.

12. The method of claim 11, wherein following the target of interest includes:

measuring a magnitude of one of the selected audio signals;

determining an average magnitude based on the magnitude measured over a last pre-defined period of time; and

maintaining position of the drone relative to the object of interest to keep the average magnitude within a range of values.

13. The method of claim 1, wherein the audio signals include one or more audio signals received from an object of interest,

wherein identifying audio signals that are of interest includes transmitting, via a transmitter of the drone, data representative of one or more of the received audio signals, and

wherein controlling flight characteristics of the drone includes:

receiving, at a receiver of the drone, flight control commands, wherein the flight control commands select one or more of the audio signals received from the object of interest and instruct the drone to track the object of interest based on the selected audio signals.

14. The method of claim 13, wherein tracking the object of interest includes:

measuring a magnitude of one of the selected audio signals received from the object of interest;

determining an average magnitude based on the magnitude measured over a last pre-defined period of time; and

maintaining position of the drone relative to the object of interest to keep the average magnitude within a range of values.

15. The method of claim 13, wherein receiving flight control commands includes receiving flight commands positioning the drone relative to the rotor noise or the wind noise and to the object of interest to better capture audio signals from the object of interest in the presence of the audio signals from the rotor noise or the wind noise.

16. The method of claim 13, wherein the received audio signals include audio signals received from a rotor that produces the rotor noise, and

wherein receiving flight control commands includes receiving a command identifying drone movements needed to separate a DOA of the selected audio signals from DOAs of audio signals received from the rotor.

17. The method of claim 16, wherein the drone includes the rotor and wherein one or more of the plurality of microphones are positioned relative to the rotor, wherein receiving flight control commands includes receiving a command orienting the microphones and the rotor relative to the DOA of the selected audio signals.

18. The method of claim 16, wherein the drone includes the rotor and wherein receiving flight control commands includes receiving a command moving the microphones relative to the rotor.

19. The method of claim 1, wherein identifying audio signals that are of interest includes:

determining, by the processor, one or more audio sources that are not the rotor noise or the wind noise;

selecting an audio source from the one or more audio sources that are not the rotor noise or the wind noise; and

identifying the audio signals that are from the selected audio source.

20. The method of claim 19, wherein the selected audio source is an object being tracked, and

wherein controlling flight characteristics of the drone includes positioning the drone to keep a DOA of audio signals from the object being tracked away from DOAs of audio signals from the rotor noise or the wind noise.

21. The method of claim 20, wherein positioning the drone to follow the object being tracked includes:

measuring a magnitude of an audio signal from the object;

determining an average magnitude of the audio signal from the object as a function of a magnitude of the audio signal from the object measured over a last pre-defined period of time; and

maintaining position of the drone relative to the object being tracked to keep the average magnitude approximately constant.

22. A nonvolatile computer-readable storage medium storing instructions that, when executed, cause one or more processors to:

receive audio signals via one or more microphones positioned relative to a location on a drone, the audio signals including audio signals emanating from a source outside the drone as well as rotor noise or wind noise;

identify audio signals that are of interest from the received audio signals; and

control flight characteristics of the drone based on the audio signals that are of interest such that a signal-to-noise ratio of the audio signals that are of interest is increased.

23. A drone remote control, comprising:

at least one processor;

a receiver;

a transmitter; and

a nonvolatile computer-readable storage medium storing instructions that are executable by the at least one processor to;

receive audio signal information representing audio signals received by a drone, the audio signal information including audio signals emanating from a source outside the drone as well as rotor noise or wind noise;

identify, based on the audio signal information, one or more audio signals that are of interest from the received audio signals; and

control flight characteristics of the drone based on the audio signals that are of interest such that a signal-to-noise ratio of the audio signals that are of interest is increased.

24. The drone remote control of claim 23, wherein the drone remote control further comprises a display and an input interface, wherein the instructions further include instructions that are executable by at least one processor to display, on the display, a phase difference plot corresponding to the received audio signal information and to receive, via the input interface, an indication of one or more audio signals of interest.

25. The drone remote control of claim 23, wherein the audio signal information includes target audio signal information representative of audio signals received by the drone from a target of interest and audio signals received from a rotor that produces the rotor noise, wherein the drone remote control further comprises a display and an input interface, wherein the instructions further include instructions that are executable by at least one processor to display, on the display, an image representing aspects of the audio signal information,

wherein the instructions that are executable by the at least one processor to identify, based on the audio signal information, one or more audio signals that are of interest include instructions to receive, via the input interface, an input identifying a portion of the image that includes representations of one or more of the audio signals received by the drone from the target of interest, and

wherein the instructions that are executable by the at least one processor to control flight characteristics of the drone based on the audio signals that are of interest include instructions to form flight commands, wherein the flight commands, when received by a drone, instruct the drone to orient the drone relative to the rotor and to the target of interest to better capture audio signals from the target of interest in the presence of the audio signals from the rotor.

26. A drone, comprising:

a processor;

a plurality of microphones;

a receiver;

a transmitter; and

a nonvolatile computer-readable storage medium storing instructions that are executable by the processor to;

receive audio signals via the microphones, the audio signals including audio signals emanating from a source outside the drone as well as rotor noise or wind noise;

identify audio signals that are of interest from the received audio signals; and

control flight characteristics of the drone based on the audio signals that are of interest such that a signal-to-noise ratio of the audio signals that are of interest is increased.

27. The drone of claim 26, wherein one or more of the microphones are movable around the drone under control of the processor.

28. The drone of claim 26, wherein the drone further comprises a camera, wherein the microphones are positioned relative to the camera and relative to a rotor that produces the rotor noise for approximately maximum separation from the rotor when the camera is pointing at a target of interest.