Detection and classification of abnormal sounds

• US 9,396,632 B2
• Filed: 12/05/2014
• Issued: 07/19/2016
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: The Invention Science Fund II, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Brav, Ehren J., Hyde, Roderick A., Urzhumov, Yaroslav A., Weaver, Thomas A., Wood, Lowell L. Jr.
• Primary Examiner(s): TRAN, THANG V
• Application Number:
  US14/562,282
• Publication Number:
  US 20160163168A1
• Time in Patent Office: 592 Days
• Field of Search: None
• US Class Current: 1/1
• CPC Class Codes: G08B 13/1672 using sonic detecting means, e.g. a microphone operating in the audio frequency range
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: G08B 17/00 Fire alarms; Alarms responsive to explosion
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0469 Presence detectors to detect unsafe condition, e.g. infrared sensor, microphone G08B21/0476 takes precedence
• CPC Class Codes: G08B 29/188 Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: G08B 3/10 using electric transmission; using electromagnetic transmission
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones

Abstract

An audio surveillance system includes a plurality of nodes and each node includes a microphone, a speaker, and a control unit. The microphone is configured to detect sound and the speaker is configured to provide sound. The control unit is configured to receive a plurality of inputs from the plurality of nodes and the plurality of inputs are based on a detected sound; determine a location of the source of the detected sound based on the plurality of inputs; classify the detected sound according to predefined alert conditions and based on the location of the source of the detected sound; provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and control at least one node from the plurality of nodes to provide an audio response to the detected sound.

Claims

1. An audio surveillance system, comprising:

a plurality of nodes, wherein each node includes a microphone configured to detect sound and a speaker configured to provide sound; and

a control unit configured to;

receive a plurality of inputs from the plurality of nodes, wherein the plurality of inputs are based on a detected sound;

determine a location of the source of the detected sound based on the plurality of inputs;

classify the detected sound according to predefined alert conditions and based on the location of the source of the detected sound;

provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and

control at least one node from the plurality of nodes to provide an audio response to the detected sound.

2. The audio surveillance system of claim 1, wherein the control unit further comprises a wireless transceiver configured to transmit a signal based on the detected sound and configured to receive a signal indicative of the audio response.

3. The audio surveillance system of claim 1, wherein the control unit is further configured to store the detected sound in a memory.

4. The audio surveillance system of claim 1, wherein the alert includes an audio recording based on the detected sound.

5. The audio surveillance system of claim 4, wherein the audio recording includes at least thirty seconds of audio.

6. The audio surveillance system of claim 5, wherein the at least thirty seconds of audio includes at least ten seconds of audio before the occurrence of the detected sound and at least ten seconds of audio after the occurrence of the detected sound.

7. The audio surveillance system of claim 4, wherein the alert includes at least one of a captured image and a video recording.

8. The audio surveillance system of claim 1, wherein the control unit is further configured to determine the location of the source of the detected sound based on at least one sound characteristic from the group including time of arrival, relative intensity, relative spectral content, and triangulation.

9. The audio surveillance system of claim 1, wherein the microphone is a directional microphone.

10. The audio surveillance system of claim 1, wherein the speaker includes one of a wide angle speaker, a directional speaker, or a directional speaker using nonlinearly downconverted ultrasound.

11. The audio surveillance system of claim 1, wherein the control unit is further configured to automatically direct the microphone toward the location of the source of the detected sound.

12. The audio surveillance system of claim 1, wherein the control unit is further configured to determine the location of at least some of the plurality of nodes based on acoustic pulses transmitted by a speaker and received by a microphone.

13. An audio surveillance system, comprising:

a plurality of nodes, wherein each node includes a microphone configured to detect sound, a camera configured to capture an image, and a speaker configured to provide sound; and

a control unit configured to;

receive a plurality of inputs from the plurality of nodes, wherein the plurality of inputs are based on at least one of the detected sound and the captured image;

determine a location of the source of the detected sound based on the plurality of inputs and further based on at least one of the detected sound and the captured image;

classify the detected sound according to predefined alert conditions and based on the location of the source of the detected sound;

provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and

control at least one node from the plurality of nodes to provide an audio response to the detected sound.

14. The audio surveillance system of claim 13, wherein the audio recording includes at least thirty seconds of audio, including the detected sound and at least ten seconds of audio before the occurrence of the detected sound and at least ten seconds of audio after the occurrence of the detected sound.

15. The audio surveillance system of claim 13, wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

16. The audio surveillance system of claim 13, wherein the control unit is further configured to automatically direct the camera toward the location of the source of the detected sound.

17. The audio surveillance system of claim 13, wherein the control unit is further configured to automatically direct the microphone based on input from the camera.

18. The audio surveillance system of claim 13, wherein the predefined alert conditions include a sound condition and an image condition.

19. The audio surveillance system of claim 13, wherein the control unit is further configured to alter the predefined alert conditions based on a time of day.

20. The audio surveillance system of claim 13, wherein the control unit is further configured to apply a first alert condition to a first node and a second alert condition to a second node.

21. The audio surveillance system of claim 20, wherein the applied alert condition is based on the location of the node.

22. The audio surveillance system of claim 13, wherein the control unit is further configured to determine the location of at least some of the plurality of nodes based on acoustic pulses transmitted by a speaker and received by a microphone.

23. The audio surveillance system of claim 13, wherein the plurality of inputs include information regarding moving audio shadows caused by a person blocking a portion of a sound source.

24. The audio surveillance system of claim 13, wherein the predefined alert conditions include conditions based on at least one of voices, glass breaking, running, falling, screams, fighting noises, or gun shots.

25. The audio surveillance system of claim 13, further comprising a monitoring device configured to provide monitoring input comprising at least one of audio and video based on the alert.

26. The audio surveillance system of claim 13, wherein the control unit is further configured to provide an audio response to the detected sound based on at least one of the classification of the detected sound or a monitoring input received from the monitoring device.

27. A method for detecting and classifying sounds, comprising:

receiving, by a control unit, a plurality of inputs from a plurality of nodes, wherein the plurality of inputs are based on at least one of a detected sound and a captured image;

determining, by the control unit, a location of the source of the detected sound based on at least one of the detected sound and the captured image;

classifying, by the control unit, the detected sound according to predefined alert conditions and based on the location of the source of the detected sound;

providing, by the control unit, an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and

controlling, by the control unit, at least on node from the plurality of nodes to provide an audio response to the detected sound.

28. The method of claim 27, further comprising detecting movement of a sound source based on Doppler shifts in sound.

29. The method of claim 27, further comprising determining a velocity of the sound source based on the plurality of inputs.

30. The method of claim 27, wherein the detected sound is detected by a microphone, and wherein the microphone is configured to continuously detect sound.

31. The method of claim 27, wherein the detected sound is detected by a microphone, and wherein the microphone is configured to detect sound only during a predefined time period.

32. The method of claim 31, wherein the predefined time period occurs once per second.

33. The method of claim 27, wherein the predefined alert conditions are updated based on the detected sound.

34. The method of claim 27, wherein the predefined alert conditions are further updated based on a monitoring input provided to the monitoring device.

35. The method of claim 27, further comprising providing an audio response to the detected sound based on at least one of the classification of the detected sound or a received monitoring input.