System and method for beep detection and interpretation

• US 10,482,901 B1
• Filed: 09/28/2018
• Issued: 11/19/2019
• Est. Priority Date: 09/28/2017
• Status: Active Grant

• Current Assignee: Alarm.com Inc. (Alarm.com Holdings, Inc.)
• Original Assignee: Alarm.com Inc. (Alarm.com Holdings, Inc.)
• Inventors: Maurer, Kevin, Slavin, Alison Jane, Djiofack, Innocent, Trundle, Stephen
• Primary Examiner(s): Tran, Thang V
• Application Number:
  US16/147,333
• Time in Patent Office: 417 Days
• CPC Class Codes: G06N 3/045 Combinations of networks
• CPC Class Codes: G06N 3/08 Learning methods
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: H04R 29/00 Monitoring arrangements; Testing arrangements for hearing aids H04R25/30; detection of loudspeaker connection H04R5/04; sound-field adaptation dependent on speaker detection H04S7/308
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Methods, systems, and apparatus, including computer programs encoded on a storage device, for beep detection and interpretation is disclosed. In one aspect, a monitoring system is disclosed that includes a processor and a storage device storing instructions that, when executed by the processor, causes the processor to perform operations. The operations may include obtaining sound data of audio sounds detected by a microphone that is located at the property, determining whether the sound data includes data representing one or more audio tones generated by a device, based on determining that the obtained sound data includes one or more audio tones generated by a device, generating an audio fingerprint of the sound data, determining, using the generated audio fingerprint, a state of the device that generated the one or more audio tones, and performing an operation based on the state of the device that generated the one or more audio tones.

Claims

1. A monitoring system for monitoring a property, the monitoring system comprising:

one or more processors; and

one or more storage devices, the one or more storage devices storing instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising;

obtaining, by the monitoring system, sound data of audio sounds detected by a microphone that is located at the property;

determining, by the monitoring system, whether the sound data includes data representing one or more audio tones generated by a device;

based on determining that the obtained sound data includes one or more audio tones generated by a device, generating, by the monitoring system, an audio fingerprint of the sound data;

generating, by the monitoring system, a query that includes the generated audio fingerprint as a search parameter;

obtaining, by the monitoring system and in response to the generated query, (i) a device identifier of the device that generated the one or more audio tones and (ii) a state of the device that generated the one or more audio tones; and

performing, by the monitoring system, one or more operations based on at least one of the device identifier and the state of the device that generated the one or more audio tones.

2. The monitoring system of claim 1, the operations further comprising:

extracting, by the monitoring system, the one or more audio tones from the sound data.

3. The monitoring system of claim 2, wherein generating, by the monitoring system, an audio fingerprint, of the sound data, comprises:

generating, by the monitoring system, the audio fingerprint based on one or more of (i) a frequency or pattern of the one or more extracted audio tones, (ii) a loudness of the one or more extracted audio tones, (iii) a duration of each of the one or more extracted audio tones, (iv) a pattern of the one or more extracted audio tones, or (v) a duration and pattern of the one or more extracted audio tones in the obtained sound data.

4. The monitoring system of claim 1, the operations further comprising:

obtaining, by the monitoring system, second sound data of audio sounds detected by a microphone that is located at the property;

determining, by the monitoring system, that the obtained second sound data does not include one or more audio tones generated by a device;

based on determining, by the monitoring system, that the obtained second sound data does not include one or more audio tones generated by a device, obtaining, by the monitoring system, additional sound data of audio sounds detected by the microphone that is located at the property; and

determining, by the monitoring system and based on the additional sound data, whether the additional sound data includes one or more audio tones generated by a device.

5. The monitoring system of claim 1, wherein obtaining sound data of audio sounds detected by a microphone that is located at the property comprises recording, using the microphone, audio sounds at the property.

6. The monitoring system of claim 1, wherein obtaining sound data of audio sounds detected by a microphone located at the property comprises receiving, by the monitoring system, a recording of audio sounds that occurred at the property.

7. The monitoring system of claim 1, wherein determining, by the monitoring system, whether the sound data includes data representing one or more audio tones comprises:

providing, by the monitoring system, the sound data as an input to a machine learning model that has been trained to determine whether sound data includes one or more audio tones generated by a device;

receiving, by the monitoring system, an output from the machine learning model that indicates whether the detected sound data includes the one or more audio tones generated by the device; and

determining, by the monitoring system and based on the received output from the machine learning model, whether the sound data includes the one or more audio tones generated the device.

8. The monitoring system of claim 1, the operations further comprising:

determining, by the monitoring system, a particular operation to perform based on (i) the device identifier and (ii) the state of the device; and

wherein performing one or more operations, by the monitoring system, based on at least one of the device identifier and the state of the device that generated the one or more audio tones comprises;

transmitting, by the monitoring system, a notification that prompts the user to perform the particular operation.

9. The monitoring system of claim 1, the operations further comprising:

determining, by the monitoring system, a particular operation to perform based on (i) the device identifier and (ii) the state of the device; and

wherein performing one or more operations, by the monitoring system, based on at least one of the device identifier and the state of the device that generated the one or more audio tones comprises;

transmitting, by the monitoring system, an instruction to a component of the monitoring system to perform the particular operation.

10. The monitoring system of claim 1, wherein determining, by the monitoring system, whether the sound data includes data representing one or more audio tones generated by a device comprises:

determining, by the monitoring system, whether the sound data includes data representing one or more audio tones generated by a speaker of the device.

11. A method for monitoring a property, the method comprising:

obtaining, by the monitoring system, sound data of audio sounds detected by a microphone that is located at the property;

determining, by the monitoring system, whether the sound data includes data representing one or more audio tones generated by a device;

based on determining that the obtained sound data includes one or more audio tones generated by a device, generating, by the monitoring system, an audio fingerprint of the sound data;

generating, by the monitoring system, a query that includes the generated audio fingerprint as a search parameter;

obtaining, by the monitoring system and in response to the generated query, (i) a device identifier of the device that generated the one or more audio tones and (ii) a state of the device that generated the one or more audio tones; and

performing, by the monitoring system, one or more operations based on at least one of the device identifier and the state of the device that generated the one or more audio tones.

12. The method of claim 11, the method further comprising:

extracting, by the monitoring system, the one or more audio tones from the sound data.

13. The method of claim 12, wherein generating, by the monitoring system, an audio fingerprint, of the sound data, comprises:

generating, by the monitoring system, the audio fingerprint based on one or more of (i) a frequency or pattern of the one or more extracted audio tones, (ii) a loudness of the one or more extracted audio tones, (iii) a duration of each of the one or more extracted audio tones, (iv) a pattern of the one or more extracted audio tones, or (v) a duration and pattern of the one or more extracted audio tones in the obtained sound data.

14. The method of claim 11, the method further comprising:

obtaining, by the monitoring system, second sound data of audio sounds detected by a microphone that is located at the property;

determining, by the monitoring system, that the obtained second sound data does not include one or more audio tones generated by a device;

based on determining, by the monitoring system, that the obtained second sound data does not include one or more audio tones generated by a device, obtaining, by the monitoring system, additional sound data of audio sounds detected by the microphone that is located at the property; and

determining, by the monitoring system and based on the additional sound data, whether the additional sound data includes one or more audio tones generated by a device.

15. The method of claim 11, wherein obtaining sound data of audio sounds detected by a microphone that is located at the property comprises recording, using the microphone, audio sounds at the property.

16. The method of claim 11, wherein obtaining sound data of audio sounds detected by a microphone located at the property comprises receiving, by the monitoring system, a recording of audio sounds that occurred at the property.

17. The method of claim 11, wherein determining, by the monitoring system, whether the sound data includes data representing one or more audio tones comprises:

providing, by the monitoring system, the sound data as an input to a machine learning model that has been trained to determine whether sound data includes one or more audio tones generated by a device;

receiving, by the monitoring system, an output from the machine learning model that indicates whether the detected sound data includes the one or more audio tones generated by the device; and

determining, by the monitoring system and based on the received output from the machine learning model, whether the sound data includes the one or more audio tones generated the device.

18. The method of claim 11, the method further comprising:

determining, by the monitoring system, a particular operation to perform based on (i) the device identifier and (ii) the state of the device; and

wherein performing one or more operations, by the monitoring system, based on at least one of the device identifier and the state of the device that generated the one or more audio tones comprises;

transmitting, by the monitoring system, a notification that prompts the user to perform the particular operation.

19. The method of claim 11, the method further comprising:

determining, by the monitoring system, a particular operation to perform based on (i) the device identifier and (ii) the state of the device; and

wherein performing one or more operations, by the monitoring system, based on at least one of the device identifier and the state of the device that generated the one or more audio tones comprises;

transmitting, by the monitoring system, an instruction to a component of the monitoring system to perform the particular operation.

20. The method of claim 11, wherein determining, by the monitoring system, whether the sound data includes data representing one or more audio tones generated by a device comprises:

determining, by the monitoring system, whether the sound data includes data representing one or more audio tones generated by a speaker of the device.