Acoustic monitoring system

• US 10,394,239 B2
• Filed: 04/04/2017
• Issued: 08/27/2019
• Est. Priority Date: 04/04/2017
• Status: Active Grant

• Current Assignee: Hyundai Motor Company (Hyundai Motor Group), Kia Corp.
• Original Assignee: AT&T Intellectual Property I LP (AT&T, Inc.)
• Inventors: Pratt, James H., Edwards, Gregory
• Primary Examiner(s): Elchanti, Hussein
• Assistant Examiner(s): Berns, Michael A
• Application Number:
  US15/478,877
• Publication Number:
  US 20180284773A1
• Time in Patent Office: 875 Days
• CPC Class Codes: B64U 2101/00 UAVs specially adapted for particular uses or applications
• CPC Class Codes: B64U 2101/20 for use as communications relays, e.g. high-altitude platforms
• CPC Class Codes: B64U 2101/30 for imaging, photography or videography
• CPC Class Codes: B64U 2201/10 autonomous, i.e. by navigating independently from ground or air stations, e.g. by using inertial navigation systems [INS]
• CPC Class Codes: G05D 1/005 by providing the operator with signals other than visual, e.g. acoustic, haptic
• CPC Class Codes: G05D 1/0088 characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours using knowledge based models G06N5/00
• CPC Class Codes: G05D 1/0094 involving pointing a payload, e.g. camera, weapon, sensor, towards a fixed or moving target
• CPC Class Codes: G06F 16/61 Indexing; Data structures therefor; Storage structures
• CPC Class Codes: G06F 16/683 using metadata automatically derived from the content
• CPC Class Codes: H04B 7/18506 Communications with or from aircraft, i.e. aeronautical mobile service

Abstract

Systems and methods for monitoring a monitored space include producing a first audio signal from received acoustic energy. The first audio signal is then processed against a whitelist of acoustic profiles and, based on lack of substantial correspondence with any of the acoustic profiles, a drone is navigated toward an apparent position of an apparent source. While in-flight, additional acoustic energy is received and a second audio signal is produced from the additional acoustic energy. The second audio signal is processed against the whitelist and, based on lack of substantial correspondence with any of the acoustic profiles of the whitelist, an investigate mode of the drone is initiated. The investigate mode includes notifying a remote monitor and supplying the remote monitor with an audiovisual feed. Responsive to a characterization by the remote monitor, an entry of the whitelist may be updated, added or replaced.

Claims

1. A method for use in a monitored space, the method comprising:

producing a first audio signal from acoustic energy received using a directionally-discriminating acoustic sensor;

computationally processing the first audio signal against a whitelist of acoustic profiles;

based on lack of substantial correspondence between the first audio signal and any of the acoustic profiles of the whitelist, dispatching a drone, the drone being in a standby mode immediately prior to the dispatching;

autonomously navigating the drone toward an apparent position within the monitored space of an apparent source of the acoustic energy;

while the drone is in-flight and using an on-board acoustic sensor, producing a second audio signal from additional acoustic energy received from the apparent source;

computationally processing the second audio signal against the whitelist;

based on continued lack of substantial correspondence with any of the acoustic profiles of the whitelist, initiating an investigate mode of the drone, wherein the drone notifies a remote monitor and supplies the remote monitor with an audiovisual feed from the drone via a wireless communications link; and

responsive to a characterization received from the remote monitor, adding an entry to the whitelist, updating an entry of the whitelist, or replacing an entry of the whitelist.

2. The method of claim 1,

wherein at least a portion of the whitelist is stored at the drone as a local copy, and the adding, updating, or replacing takes place at the drone using the local copy.

3. The method of claim 1,

wherein at least a portion of the whitelist is maintained remote from the drone on a service platform accessible via the wireless communications link, and

wherein, in at least some modes of operation, the computational processing of the first or second audio signal against the whitelist includes communicating to the service platform, via the wireless communications link, an encoding of the first or second audio signal or one or more feature vectors computationally derived from the first or second audio signal.

4. The method of claim 3, further comprising:

adding, updating or replacing an entry of a local portion of the whitelist stored at the drone based on substantial correspondence of the first or second audio signal with an entry of the at least the portion of the whitelist maintained remote from the drone.

5. The method of claim 3, further comprising:

initiating a return of the drone to the standby mode based on computational processing that determines that the second audio signal has substantial correspondence with an entry of the at least the portion of the whitelist maintained remote from the drone.

6. The method of claim 1, further comprising:

computationally processing the first audio signal against a blacklist of acoustic profiles and based on substantial correspondence with any of the acoustic profiles of the blacklist, dispatching and autonomously navigating the drone from a monitoring location or a standby location toward the apparent position within the monitored space of the apparent source of the acoustic energy.

7. The method of claim 1, further comprising

initializing on-board storage of the acoustic profiles on the drone based on at least one of:

a mission parameter or an operating parameter.

8. A monitoring system for use in a monitored space, the monitoring system comprising:

a directionally-discriminating acoustic sensor for producing audio signals; and

a drone including an on-board storage for at least a portion of a whitelist of acoustic profiles and an on-board processor configured to computationally evaluate correspondence of the audio signals with the acoustic profiles of the whitelist, the drone configured to respond to a first audio signal of the audio signals produced from acoustic energy received using the directionally-discriminating acoustic sensor, wherein the on-board processor of the drone, while the drone is in a standby mode, is configured to computationally process the first audio signal and, based on lack of substantial correspondence between the first audio signal and any of the acoustic profiles of the whitelist, to autonomously navigate toward an apparent position within the monitored space of an apparent source of the acoustic energy,

the drone further configured to, while in-flight and using an on-board acoustic sensor, produce a second audio signal from additional received acoustic energy, computationally process the second audio signal against the whitelist and, based on lack of substantial correspondence with any of the acoustic profiles of the whitelist, initiate an investigate mode, wherein the drone notifies a remote monitor and supplies the remote monitor with an audiovisual feed via a wireless communications link,

wherein the drone is configured to add, update or replace an entry of the whitelist responsive to a characterization received from the remote monitor.

9. The monitoring system of claim 8,

wherein the directionally-discriminating acoustic sensor is or includes the on-board acoustic sensor.

10. The monitoring system of claim 8,

wherein at least a portion of the whitelist is stored remote from the drone on a service platform accessible via the wireless communications link, and

wherein, in at least some modes of operation, the computational processing of an audio signal of the audio signals against the whitelist includes communicating to the service platform, via the wireless communications link, an encoding of, or a feature vector computationally derived from, the audio signal.

11. The monitoring system of claim 10,

wherein the drone is further configured to add, update or replace an entry of the portion of the whitelist stored on-board the drone based on substantial correspondence of the first or second audio signal with an entry of the portion of the whitelist maintained remote from the drone.

12. The monitoring system of claim 10,

wherein the drone is further configured to initiate a return to the standby mode based on a determination that the second audio signal has substantial correspondence with an entry of the portion of the whitelist maintained remote from the drone.

13. The monitoring system of claim 8,

wherein the drone is further configured to computationally process the first audio signal against a blacklist of acoustic profiles and, based on substantial correspondence with any of the acoustic profiles of the blacklist, dispatch and autonomously navigate toward the apparent position within the monitored space of the apparent source of the acoustic energy.

14. The monitoring system of claim 8, wherein the remote monitor includes one or more of:

a human operator at a remote location; and

an automated system programmed to characterize the apparent source using one or more of;

a statistical classifier technique, a machine learning technique and a signal processing technique.

15. A method for use in a monitored space, the method comprising:

producing a first audio signal from acoustic energy received using a directionally-discriminating acoustic sensor;

computationally processing the first audio signal to determine correspondence with stored acoustic profiles, wherein the stored acoustic profiles include either or both of (i) include-type entries characteristic of sounds to be investigated and (ii) exclude-type entries characteristic of sounds to be ignored;

based on either or both of (i) substantial correspondence with any include-type entries or (ii) lack of substantial correspondence with each of the exclude-type entries, dispatching a drone, the drone being in a standby mode immediately prior to the dispatching;

autonomously navigating the drone toward an apparent position within the monitored space of an apparent source of the acoustic energy;

while the drone is in-flight and using an on-board acoustic sensor, producing a second audio signal from additional acoustic energy received from the apparent source;

computationally processing the second audio signal to determine correspondence with the stored acoustic profiles;

based on either or both of (i) substantial correspondence with any of the include-type entries or (ii) lack of substantial correspondence with each of the exclude-type entries, initiating an investigate mode of the drone, wherein the drone notifies a remote monitor via a wireless communications link and supplies the remote monitor with an audiovisual feed; and

responsive to a characterization received from the remote monitor, adding, updating, or replacing an include-type or exclude-type entry to the stored acoustic profiles.

16. The method of claim 15,

wherein at least some of the stored acoustic profiles are stored on-board the drone, and the adding, updating, or replacing takes place at the drone using the at least some of the stored acoustic profiles that are stored on-board the drone.

17. The method of claim 15,

wherein at least some of the stored acoustic profiles are maintained remote from the drone on a service platform accessible via the wireless communications link, and

wherein, in at least some modes of operation, the computational processing of the first and second audio signals against include- and exclude-type entries includes communicating to the service platform, via the wireless communications link, an encoding of the first or second audio signal or one or more feature vectors computationally derived from the first or second audio signal.

18. The method of claim 17,

wherein the exclude-type entries constitute a whitelist, and

wherein the include-type entries constitute a blacklist.

19. The method of claim 15, further comprising:

initializing on-board storage of the stored acoustic profiles on the drone based on at least one of;

a mission parameter or an operating parameter.

20. The method of claim 15, further comprising:

managing storage for the acoustic profiles on-board the drone as cache of recently relied upon include-type and exclude-type entries.