Detection and classification of abnormal sounds

• US 10,068,446 B2
• Filed: 09/07/2017
• Issued: 09/04/2018
• Est. Priority Date: 12/05/2014
• Status: Active Grant

• Current Assignee: The Invention Science Fund II, L.L.C. (Intellectual Ventures LLC)
• Original Assignee: Elwha LLC (Intellectual Ventures LLC)
• Inventors: Brav, Ehren J., Hyde, Roderick A., Urzhumov, Yaroslav A., Weaver, Thomas A., Wood, Jr., Lowell L.
• Primary Examiner(s): Tran, Thang
• Application Number:
  US15/697,837
• Publication Number:
  US 20170372571A1
• Time in Patent Office: 362 Days
• CPC Class Codes: G08B 13/1672 using sonic detecting means, e.g. a microphone operating in the audio frequency range
• CPC Class Codes: G08B 13/19697 Arrangements wherein non-video detectors generate an alarm themselves
• CPC Class Codes: G08B 17/00 Fire alarms; Alarms responsive to explosion
• CPC Class Codes: G08B 21/02 Alarms for ensuring the safety of persons
• CPC Class Codes: G08B 21/043 detecting an emergency event, e.g. a fall
• CPC Class Codes: G08B 21/0469 Presence detectors to detect unsafe condition, e.g. infrared sensor, microphone G08B21/0476 takes precedence
• CPC Class Codes: G08B 29/188 Data fusion; cooperative systems, e.g. voting among different detectors
• CPC Class Codes: G08B 3/10 using electric transmission; using electromagnetic transmission
• CPC Class Codes: H04R 1/028 associated with devices performing functions other than acoustics, e.g. electric candles
• CPC Class Codes: H04R 2420/07 Applications of wireless loudspeakers or wireless microphones

Abstract

An audio surveillance system includes a plurality of nodes and each node includes a microphone, a speaker, and a control unit. The microphone is configured to detect sound and the speaker is configured to provide sound. The control unit is configured to receive a plurality of inputs from the plurality of nodes and the plurality of inputs are based on a detected sound; determine a location of the source of the detected sound based on the plurality of inputs; classify the detected sound according to predefined alert conditions and based on the location of the source of the detected sound; provide an alert to a monitoring device regarding the detected sound based on the classification of the detected sound; and control at least one node from the plurality of nodes to provide an audio response to the detected sound.

Claims

1. A system comprising:

a primary audio surveillance node including a control unit, the control unit configured to;

receive a sound input based on a sound emitted by a source and detected by a microphone configured to detect sound and acoustic pulses;

receive an acoustic pulse input based on an acoustic pulse provided by a secondary audio surveillance node and detected by the microphone;

determine a relative location of the primary audio surveillance node with respect to the secondary audio surveillance node based on the acoustic pulse;

determine a location of the source of the detected sound based on the sound input and the determined relative location; and

classify the detected sound based on a characteristic of the detected sound and the determined location of the source.

2. The system of claim 1, wherein the primary audio surveillance node further comprises a speaker configured to emit acoustic pulses, and a wireless transceiver configured to transmit time-of-emission data for each acoustic pulse emitted by the speaker and receive time-of-emission data for each acoustic pulse provided by the secondary audio surveillance node.

3. The system of claim 1, wherein the control unit is further configured to determine a time-of-flight for each detected acoustic pulse.

4. The system of claim 1, wherein the primary audio surveillance node further comprises a wireless transceiver configured to transmit time-of-flight data for each detected acoustic pulse and receive time-of-emission data for an acoustic pulse detected by another audio surveillance node.

5. The system of claim 1, wherein the determination of the relative location of the primary audio surveillance node is based on a plurality of time-of-flight data.

6. The system of claim 1, wherein the primary audio surveillance node further comprises a wireless transceiver configured to transmit a signal based on the detected sound and receive a signal from a monitoring device regarding an audio response, wherein the control unit is configured to provide the audio response using a speaker of the primary audio surveillance node.

7. The system of claim 1, wherein the control unit is configured to provide an alert, the alert including audio received from a monitoring device.

8. The system of claim 1, wherein the control unit is configured to provide an alert, the alert including at least one of an image and a video.

9. The system of claim 1, wherein the control unit is configured to determine the location of the source based on a characteristic of the detected sound, the sound characteristic at least one of a time of arrival, a relative intensity, a relative spectral content, and a triangulation.

10. The system of claim 1, wherein the control unit is configured to automatically direct the microphone toward the determined location of the source.

11. The system of claim 1, wherein the primary audio surveillance node further comprises a camera, and wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

12. The system of claim 11, wherein the control unit is configured to automatically direct the camera toward the determined location of the source.

13. A method comprising:

receiving, by a control unit, a plurality of inputs from a plurality of nodes, wherein the plurality of inputs are based on a sound detected by a microphone;

determining, by the control unit, a location of a source of the detected sound based on the plurality of inputs;

classifying, by the control unit, the detected sound based on a characteristic of the detected sound and the location of the source; and

providing, by the control unit, an alert to a monitoring device regarding the detected sound based on the classification.

14. The method of claim 13, further comprising transmitting a signal based on the detected sound and receiving a signal from the monitoring device regarding an audio response.

15. The method of claim 13, wherein the microphone is a directional microphone.

16. The method of claim 13, further comprising providing an audio response using a speaker of a node of the plurality of nodes, wherein the speaker includes one of a wide angle speaker, a directional speaker, or a directional speaker using nonlinearly downconverted ultrasound.

17. The method of claim 13, further comprising receiving, by the control unit, an image captured by a camera, wherein the camera is at least one of a video camera, a still camera, or a spectral camera.

18. The method of claim 17, further comprising automatically directing the microphone based on the image.

19. The method of claim 13, wherein the classification is based on a predefined alert condition.

20. The method of claim 19, further comprising altering the predefined alert condition based on a time of day.

21. The method of claim 19, wherein the predefined alert condition comprises a first alert condition and a second alert condition, and wherein the first alert condition is applied to a first node and a second alert condition is applied to a second node.

22. The method of claim 21, wherein the applied alert condition is based on the location of the node.

23. The method of claim 13, further comprising determining the location of at least a first node of the plurality of nodes based on an acoustic pulse transmitted by a speaker of a second node and received by a microphone of at least one of the first node and a third node.

24. The method of claim 13, further comprising detecting, by the control unit, a moving audio shadow caused by a portion of the source being blocked.

25. A method for detecting and classifying sounds, comprising:

receiving, by a control unit, sound data from a plurality of audio surveillance nodes, the sound data indicative of a sound detected by at least one of the audio surveillance nodes and an acoustic pulse received by a primary audio surveillance node and transmitted by a secondary audio surveillance node;

determining, by the control unit, a time-of-flight of the acoustic pulse;

determining, by the control unit, a location of the primary audio surveillance node based on the time-of-flight;

determining, by the control unit, a location of a source of the detected sound based on the sound data and based on the location of the primary audio surveillance node;

classifying, by the control unit, the detected sound based on the location of the source; and

controlling, by the control unit, a speaker to provide an audio response to the detected sound.

26. The method of claim 25, wherein classifying the detected sound is based on a predefined alert condition, the predefined alert condition including a sound characteristic.

27. The method of claim 26, further comprising altering the predefined alert condition based on a time of day.

28. The method of claim 26, wherein the predefined alert condition includes a condition based on at least one of a voice, glass breaking, running, falling, a scream, fighting, or a gun shot.

29. The method of claim 26, wherein the predefined alert condition is updated based on the detected sound.

30. The method of claim 26, wherein the predefined alert condition is updated based on a user input received at a monitoring device.

31. The method of claim 25, further comprising determining a movement of the source based on Doppler shifts in the detected sound.

32. The method of claim 25, further comprising determining a velocity of the source based on the sound data.

33. The method of claim 25, wherein the detected sound is detected by a microphone, and wherein the microphone is configured to detect sound only during a predefined time period.

34. The method of claim 33, wherein the predefined time period occurs once per second.

35. The method of claim 25, further comprising providing an audio response to the detected sound based on at least one of the classification of the detected sound or a user input received at a monitoring device.