Detection and classification of abnormal sounds

• US 9,767,661 B2
• Filed: 07/13/2016
• Issued: 09/19/2017
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: The Invention Science Fund II, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Brav, Ehren J., Hyde, Roderick A., Urzhumov, Yaroslav A., Weaver, Thomas A., Wood, Jr., Lowell L.
• Primary Examiner(s): TRAN, THANG V
• Application Number:
  US15/209,130
• Publication Number:
  US 20160321886A1
• Time in Patent Office: 433 Days
• CPC Class Codes: G08B 13/1672 using sonic detecting means, e.g. a microphone operating in the audio frequency range
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: G08B 17/00 Fire alarms; Alarms responsive to explosion
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0469 Presence detectors to detect unsafe condition, e.g. infrared sensor, microphone G08B21/0476 takes precedence
• CPC Class Codes: G08B 29/188 Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: G08B 3/10 using electric transmission; using electromagnetic transmission
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones

Abstract

An audio surveillance system includes a plurality of nodes and each node includes a microphone, a speaker, and a control unit. The microphone is configured to detect sound and the speaker is configured to provide sound. The control unit is configured to receive a plurality of inputs from the plurality of nodes and the plurality of inputs are based on a detected sound; determine a location of the source of the detected sound based on the plurality of inputs; classify the detected sound according to predefined alert conditions and based on the location of the source of the detected sound; provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and control at least one node from the plurality of nodes to provide an audio response to the detected sound.

Claims

1. An audio surveillance node, comprising:

a microphone configured to detect sound and to detect acoustic pulses;

a speaker configured to provide sound and to emit acoustic pulses; and

a control unit configured to;

receive a plurality of inputs, including a plurality of sound inputs based on a detected sound detected by the microphone and a plurality of acoustic pulses transmitted by a second audio surveillance node and detected by the microphone;

determine a location of the audio surveillance node relative to the second audio surveillance node based on the acoustic pulses detected by the microphone;

determine a location of the source of the detected sound based on the plurality of sound inputs and the location of the second audio surveillance node;

classify the detected sound according to predefined alert conditions and based on the determination of the location of the source of the detected sound;

provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and

provide an audio response to the detected sound.

2. The audio surveillance node of claim 1, further comprising a wireless transceiver configured to transmit time-of-emission data for each emitted acoustic pulse and receive time-of-emission data for each detected acoustic pulse.

3. The audio surveillance node of claim 1, wherein the control unit is further configured to determine time-of-flight data for each detected acoustic pulse.

4. The audio surveillance node of claim 1, further comprising a wireless transceiver configured to transmit time-of-flight data for each detected acoustic pulse and receive time-of-emission data for a plurality of acoustic pulses detected by other audio surveillance nodes.

5. The audio surveillance node of claim 1, wherein the determination of the location of the audio surveillance node is based on a plurality of time-of-flight data.

6. The audio surveillance node of claim 1, further comprising a wireless transceiver configured to transmit a signal based on the detected sound and receive a signal indicative of the audio response.

7. The audio surveillance node of claim 1, wherein the alert includes an audio recording based on the detected sound.

8. The audio surveillance node of claim 1, wherein the alert includes at least one of a captured image and a video recording.

9. The audio surveillance node of claim 1, wherein the control unit is configured to determine the location of the source of the detected sound based on at least one sound characteristic from the group including time of arrival, relative intensity, relative spectral content, and triangulation.

10. The audio surveillance node of claim 1, wherein the control unit is further configured to automatically direct the microphone toward the location of the source of the detected sound.

11. The audio surveillance node of claim 1, further comprising a camera, wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

12. The audio surveillance node of claim 11, wherein the control unit is further configured to automatically direct the camera toward the location of the source of the detected sound.

13. A method for detecting and classifying sounds, comprising:

receiving, by a control unit, a plurality of inputs from a plurality of nodes, wherein the plurality of inputs are based on a detected sound;

determining, by the control unit, a location of the source of the detected sound based on the plurality of inputs;

classifying, by the control unit, the detected sound according to predefined alert conditions and based on the location of the source of the detected sound;

providing, by the control unit, an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and

controlling, by the control unit, at least one node from the plurality of nodes to provide an audio response to the detected sound.

14. The method of claim 13, further comprising transmitting a signal based on the detected sound and receiving a signal indicative of the audio response.

15. The method of claim 13, wherein the detected sound is detected by a microphone, and wherein the microphone is a directional microphone.

16. The method of claim 13, wherein the audio response is provided by a speaker, and wherein the speaker includes one of a wide angle speaker, a directional speaker, or a directional speaker using nonlinearly downconverted ultrasound.

17. The method of claim 13, wherein the plurality of inputs are further based on an image captured by a camera, wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

18. The method of claim 17, further comprising automatically directing the microphone based on input from the camera.

19. The method of claim 13, wherein the predefined alert conditions include a sound condition and an image condition.

20. The method of claim 13, further comprising altering the predefined alert conditions based on a time of day.

21. The method of claim 13, further comprising applying a first alert condition to a first node and a second alert condition to a second node.

22. The method of claim 21, wherein the applied alert condition is based on the location of the node.

23. The method of claim 13, further comprising determining the location of at least some of the plurality of nodes based on acoustic pulses transmitted by a speaker and received by a microphone.

24. The method of claim 13, further comprising detecting a moving audio shadow caused by a person blocking a portion of a sound source.

25. A method for detecting and classifying sounds, comprising:

receiving, by a control unit, a plurality of inputs, including a plurality of sound inputs based on a detected sound and time-of-flight data for a plurality of acoustic pulses transmitted and received by different audio surveillance nodes;

determining, by the control unit, a location of an audio surveillance node based on the plurality of time-of-flight data;

determining, by the control unit, a location of the source of the detected sound based on the plurality of sound inputs and based on the location of the audio surveillance node;

classifying, by the control unit, the detected sound according to predefined alert conditions and based on the location of the source of the detected sound;

providing, by the control unit, an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and

controlling, by the control unit, a speaker to provide an audio response to the detected sound.

26. The method of claim 25, wherein the predefined alert conditions include a sound condition and an image condition.

27. The method of claim 25, further comprising altering the predefined alert conditions based on a time of day.

28. The method of claim 25, wherein the predefined alert conditions include conditions based on at least one of voices, glass breaking, running, falling, screams, fighting noises, or gun shots.

29. The method of claim 25, further comprising determining a movement of a sound source based on Doppler shifts in sound.

30. The method of claim 25, further comprising determining a velocity of the sound source based on the plurality of inputs.

31. The method of claim 25, wherein the detected sound is detected by a microphone, and wherein the microphone is configured to detect sound only during a predefined time period.

32. The method of claim 31, wherein the predefined time period occurs once per second.

33. The method of claim 25, wherein the predefined alert conditions are updated based on the detected sound.

34. The method of claim 25, wherein the predefined alert conditions are further updated based on a monitoring input provided to the monitoring device.

35. The method of claim 25, further comprising providing an audio response to the detected sound based on at least one of the classification of the detected sound or a monitoring input received from the monitoring device.