DRONE FLIGHT CONTROL

• US 20170220036A1
• Filed: 06/17/2016
• Published: 08/03/2017
• Est. Priority Date: 01/28/2016
• Status: Active Grant

• Current Assignee: Qualcomm, Inc.
• Original Assignee: Qualcomm, Inc.
• Inventors: Peri, Raghuveer, Bernal Castillo, Ricardo De Jesus, Visser, Erik, Kim, Lae-Hoon, Zhang, Shuhua
• Granted Patent: US 10,379,534 B2
• Time in Patent Office: Days
• CPC Class Codes: A61B 5/0205 Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/20 Remote controls
• CPC Class Codes: G05D 1/0038 by providing the operator with simple or augmented images from one or more cameras located onboard the vehicle, e.g. tele-operation images analyzed by a computer and used for automatic navigation G05D1/0246
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G05D 1/0808 specially adapted for aircraft
• CPC Class Codes: G05D 1/101 specially adapted for aircraft
• CPC Class Codes: G08C 23/02 using infrasonic, sonic or ultrasonic waves
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: H04R 1/245 of microphones
• CPC Class Codes: H04R 1/406 microphones
• CPC Class Codes: H04R 2201/401 2D or 3D arrays of transducers
• CPC Class Codes: H04R 2430/20 Processing of the output signals of the acoustic transducers of an array for obtaining a desired directivity characteristic H04R2203/12 takes precedence
• CPC Class Codes: H04R 29/004 for microphones H04R29/007 takes precedence

Abstract

A drone system and method. Audio signals are received via one or more microphones positioned relative to a location on a drone and one or more of the audio signals are identified as of interest. Flight characteristics of the drone are then controlled based on the audio signals that are of interest.

Claims

1. A method comprising:

receiving audio signals via a plurality of microphones positioned relative to a location on a drone;

identifying audio signals that are of interest; and

controlling flight characteristics of the drone based on the audio signals that are of interest.

2. The method of claim 1, wherein identifying audio signals that are of interest includes selecting audio signals received from a target of interest.

3. The method of claim 2, wherein the audio signals include audio signals received from one or more noise sources, and

wherein controlling flight characteristics of the drone includes positioning the drone relative to the noise sources and to the target of interest to better capture audio signals from the target of interest in the presence of the audio signals from the one or more noise sources.

4. The method of claim 2, wherein the selected audio signals are audio signals received from an object being tracked, and

wherein controlling flight characteristics of the drone includes determining a direction of arrival (DOA) of one or more of the selected audio signals and positioning the drone based on the DOA of the one or more selected audio signals.

5. The method of claim 2, wherein the audio signals include audio signals received from one or more noise sources,

wherein the selected audio signals are audio signals received from an object being tracked, and

wherein controlling flight characteristics of the drone includes determining a direction of arrival (DOA) of one or more of the selected audio signals and positioning the drone relative to the noise sources and to the target of interest to separate the DOA of the audio signals from the object being tracked from the DOA of the audio signals from the one or more noise sources.

6. The method of claim 5, wherein the drone includes one or more of the noise sources and wherein one or more of the plurality of microphones are positioned relative to the drone noise sources, wherein positioning the drone includes orienting the microphones and the drone noise sources relative to the DOA of the selected audio signals.

7. The method of claim 5, wherein the drone includes one or more noise sources and wherein one or more of the plurality of microphones are positioned relative to the noise sources, wherein positioning the drone includes moving the microphones relative to the drone noise sources.

8. The method of claim 5, wherein the drone includes one or more noise sources and wherein one or more of the plurality of microphones are positioned relative to the drone noise sources, wherein positioning the drone includes enhancing capture of the selected audio signals via beam forming.

9. The method of claim 2, wherein the selected audio signals are audio signals received from an object being tracked, and

wherein controlling flight characteristics of the drone includes following the object being tracked based on the selected audio signals.

10. The method of claim 2, wherein receiving audio signals includes positioning two or more of the plurality of microphones to perform beam forming on the selected audio signals.

11. The method of claim 10, wherein controlling flight characteristics of the drone includes following the target of interest based on the selected audio signals.

12. The method of claim 11, wherein following the target of interest includes:

measuring a magnitude of one of the selected audio signals;

determining an average magnitude based on the magnitude measured over a last pre-defined period of time; and

maintaining position relative to the object of interest to keep the average magnitude within a range of values.

13. The method of claim 1, wherein the audio signals include one or more audio signals received from an object of interest,

wherein identifying audio signals that are of interest includes transmitting, via a transmitter of the drone, data representative of one or more of the received audio signals, and

wherein controlling flight characteristics of the drone includes:

receiving, at a receiver of the drone, flight control commands, wherein the flight control commands select one or more of the audio signals received from the object of interest and instruct the drone to track the object of interest based on the selected audio signals.

14. The method of claim 13, wherein tracking the object of interest includes:

measuring a magnitude of one of the selected audio signals received from the object of interest;

determining an average magnitude based on the magnitude measured over a last pre-defined period of time; and

maintaining position relative to the object of interest to keep the average magnitude within a range of values.

15. The method of claim 13, wherein receiving flight control commands includes receiving flight commands positioning the drone relative to noise sources and to the object of interest to better capture audio signals from the object of interest in the presence of the audio signals from the noise sources.

16. The method of claim 13, wherein the received audio signals include audio signals received from one or more noise sources, and

wherein receiving flight control commands includes receiving a command identifying drone movements needed to separate a DOA of the selected audio signals from DOAs of audio signals received from the one or more noise sources.

17. The method of claim 16, wherein the drone includes one or more of the noise sources and wherein one or more of the plurality of microphones are positioned relative to the drone noise sources, wherein receiving flight control commands includes receiving a command orienting the microphones and the drone noise sources relative to the DOA of the selected audio signals.

18. The method of claim 16, wherein the drone includes one or more noise sources and wherein receiving flight control commands includes receiving a command moving the microphones relative to the drone noise sources.

19. The method of claim 1, wherein identifying audio signals that are of interest includes:

determining, in a processor, one or more audio sources that are not noise sources;

selecting an audio source from the one or more audio sources that are not noise sources; and

identifying the audio signals that are from the selected audio source.

20. The method of claim 19, wherein the selected audio source is an object being tracked, and

wherein controlling flight characteristics of the drone includes positioning the drone to keep a DOA of audio signals from the object being tracked away from DOAs of audio signals from noise sources.

21. The method of claim 20, wherein positioning the drone to follow the object being tracked includes:

measuring a magnitude of an audio signal from the object;

determining an average magnitude of the audio signal from the object as a function of a magnitude of the audio signal from the object measured over a last pre-defined period of time; and

maintaining position relative to the object being tracked to keep the average magnitude approximately constant.

22. The method of claim 1, wherein identifying audio signals that are of interest includes:

storing audio signal signatures representative of targets of interest;

comparing the received audio signals to the stored audio signal signatures; and

identifying audio sources with audio signals that match the stored audio signal signatures.

23. A nonvolatile computer-readable storage medium storing instructions that, when executed, cause one or more processors to:

receive audio signals via one or more microphones positioned relative to a location on a drone;

identify audio signals that are of interest; and

control flight characteristics of the drone based on the audio signals that are of interest.

24. The nonvolatile computer-readable storage medium of claim 23, wherein the instructions that, when executed, cause the one or more processors to identify audio signals that are of interest include instructions that, when executed, cause one or more processors to:

store audio signal signatures representative of targets of interest;

compare the received audio signals to the stored audio signal signatures; and

identify audio sources with audio signals that match the stored audio signal signatures.

25. A drone remote control, comprising:

at least one processor;

a receiver;

a transmitter; and

a nonvolatile computer-readable storage medium storing instructions that are executable by the at least one processor to;

receive audio signal information representing audio signals received by a drone;

identify, based on the audio signal information, one or more audio signals that are of interest; and

control flight characteristics of the drone based on the audio signals that are of interest.

26. The drone remote control of claim 25, wherein the drone remote control further comprises a display and an input interface, wherein the instructions further include instructions that are executable by at least one processor to display, on the display, a phase difference plot corresponding to the received audio signal information and to receive, via the input interface, an indication of one or more audio signals of interest.

27. The drone remote control of claim 25, wherein the audio signal information includes target audio signal information representative of audio signals received by the drone from a target of interest and audio signals received from one or more noise sources, wherein the drone remote control further comprises a display and an input interface, wherein the instructions further include instructions that are executable by at least one processor to display, on the display, an image representing aspects of the audio signal information,

wherein the instructions that are executable by the at least one processor to identify, based on the audio signal information, one or more audio signals that are of interest include instructions to receive, via the input interface, an input identifying a portion of the image that includes representations of one or more of the audio signals received by the drone from the target of interest, and

wherein the instructions that are executable by the at least one processor to control flight characteristics of the drone based on the audio signals that are of interest include instructions to form flight commands, wherein the flight commands, when received by a drone, instruct the drone to orient the drone relative to the noise sources and to the target of interest to better capture audio signals from the target of interest in the presence of the audio signals from the one or more noise sources.

28. A drone, comprising:

a processor;

a plurality of microphones;

a receiver;

a transmitter; and

a nonvolatile computer-readable storage medium storing instructions that are executable by the processor to;

receive audio signals via the microphones;

identify audio signals that are of interest; and

control flight characteristics of the drone based on the audio signals that are of interest.

29. The drone of claim 28, wherein one or more of the microphones are movable around the drone under control of the processor.

30. The drone of claim 28, wherein the drone further comprises a camera, wherein the microphones are positioned relative to the camera and relative to noise sources on the drone for approximately maximum separation from the noise sources of the drone when the camera is pointing at a target of interest.