Acoustic and other waveform event detection and correction systems and methods

• US 10,665,223 B2
• Filed: 09/28/2018
• Issued: 05/26/2020
• Est. Priority Date: 09/29/2017
• Status: Active Grant

• Current Assignee: Udifi, Inc.
• Original Assignee: Udifi, Inc.
• Inventors: Neil, Jack Edward
• Primary Examiner(s): Vo, Huyen X
• Application Number:
  US16/146,370
• Publication Number:
  US 20190103094A1
• Time in Patent Office: 606 Days
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G10L 15/063 Training
• CPC Class Codes: G10L 15/16 using artificial neural networks
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 25/51 for comparison or discrimination
• CPC Class Codes: G10L 25/72 for transmitting results of analysis

Abstract

Systems and methods for detecting, classifying, and correcting acoustic (waveform) events are provided. In one example embodiment, a computer-implemented method includes obtaining, by a computing system, audio data from a source. The method includes accessing, by the computing system, data indicative of a machine-learned acoustic detection model. The method includes inputting, by the computing system, the audio data from the source into the machine-learned acoustic detection model. The method includes obtaining, by the computing system, an output from the machine-learned acoustic detection model. The output is indicative of an acoustic event associated with the source. The method includes providing, by the computing system, data indicative of a notification to a user device. The notification indicates the acoustic event and response(s) for selection by a user. The computing system, via a continuously learned hierarchical process, may initiate autonomous actions in an effort to halt or otherwise modify the acoustic event.

Claims

1. A computer-implemented method for detecting acoustic events, comprising:

obtaining, by a computing system comprising one or more computing devices, audio data associated with a source;

accessing, by the computing system, data indicative of a machine-learned acoustic detection model;

inputting, by the computing system, the audio data from the source into the machine-learned acoustic detection model;

obtaining, by the computing system, an output from the machine-learned acoustic detection model, wherein the output is indicative of a first acoustic event associated with the source; and

providing, by the computing system, data indicative of a notification to a user device of a user, wherein the notification indicates the first acoustic event and one or more responses for selection by the user, wherein one or more first peripheral devices are associated with addressing the first acoustic event;

obtaining, by the computing system, data indicative of a success of one or more second peripheral devices in addressing a second acoustic event, wherein the second acoustic event is of a similar type to the first acoustic event; and

adjusting, by the computing, system, a peripheral device hierarchy based at least in part on the data indicative of the success of the one or more second peripheral devices in addressing the second acoustic event.

2. The computer-implemented method of claim 1, wherein the source is a human child and the first acoustic event is a cry of the human child.

3. The computer-implemented method of claim 1, wherein the user device is configured to present the notification on a user interface via a display device of the user device.

4. The computer-implemented method of claim 1, wherein the one or more responses comprise an activation of one or more of the first peripheral devices within proximity of the source.

5. The computer-implemented method of claim 1, further comprising:

obtaining, by the computing system, data indicative of a selection of at least one of the responses by the user.

6. The computer-implemented method of claim 5, further comprising:

activating, by the computing system, one or more of the peripheral devices based at least in part on a selection of at least one of the responses by the user.

7. The computer-implemented method of claim 1, further comprising:

activating, by the computing system, one or more peripheral devices based at least in part on the acoustic event.

8. The computer-implemented method of claim 1, further comprising:

activating, by the computing system, one or more of the first peripheral devices based at least in part on the peripheral device hierarchy.

9. The computer-implemented method of claim 1, further comprising:

activating, by the computing system, one or more of the first peripheral devices based at least in part on a random order.

10. The computer-implemented method of claim 1, wherein the output indicates a type of the first acoustic event.

11. The computer-implemented method of claim 1, wherein the output indicates a confidence level.

12. The computer-implemented method of claim 1, wherein the machine-learned acoustic detection model is trained based at least in part on training data indicative of a test acoustic event that is of a similar type to the first acoustic event.

13. The computer-implemented method of claim 1, wherein the machine-learned acoustic detection model comprises a neural network.

14. The computer-implemented method of claim 1, further comprising:

obtaining, by the computing system, data indicative of a success of at least one of the first peripheral devices in addressing the first acoustic event; and

adjusting, by the computing system, the peripheral device hierarchy based at least in part on the data indicative of the success of the at least one first peripheral device in addressing the first acoustic event.

15. The computer-implement method of claim 1, further comprising:

obtaining, by the computing system, feedback data associated with the first acoustic event, wherein the feedback data indicates that the first acoustic event was enoneously detected.

16. The computer-implemented method of claim 15, wherein the machine-learned acoustic detection model is trained based at least in part on the feedback data, wherein the feedback data is associated with user input provided by the user to a user interface.

17. A computing device comprising:

one or more input devices;

one or more processors; and

one or more memory devices, the one or, more memory devices storing instructions that when executed by the one or more processors cause the one or more processors to perform operations, the operations comprising;

obtaining, via the one or more input devices, audio data from a source;

accessing data indicative of a machine-learned acoustic detection model;

inputting the audio data from the source into the machine-learned acoustic detection model;

obtaining an output from the machine learned acoustic detection model, wherein the output is indicative of first acoustic event associated with the source;

providing data indicative of a notification to a user device of a user, wherein the notification indicates the first acoustic event and one or more responses for selection by the user, wherein one or more first peripheral devices are associated with addressing the first acoustic event;

obtaining data indicative of a success of one or more second peripheral devices in addressing a second acoustic event, wherein the second acoustic event is of a similar type to the first acoustic event; and

adjusting a peripheral device hierarchy based at least in part on the data indicative of the success of the one or more second peripheral device in addressing the second acoustic event.

18. One or more tangible, non-transitory computer-readable media storing computer-readable instructions that when executed by one or more processors cause the one or more processors to perform operations, the operations comprising:

obtaining audio data associated with a source, wherein the source is a human child;

accessing data indicative of a machine-learned acoustic detection model;

inputting the audio data from the source into the machine-learned acoustic detection model;

obtaining an output from the machine-learned acoustic detection model, wherein the output is indicative of a first acoustic event associated with a cry of the human child;

providing data indicative of a notification to a user device of a user, wherein the notification indicates the first acoustic event and one or more responses for selection by the user, wherein one or more first peripheral devices are associated with addressing the first acoustic event;

obtaining data indicative of a success of one or more second peripheral devices in addressing a second acoustic event, wherein the second acoustic event is of a similar type to the first acoustic event; and

adjusting a peripheral device hierarchy based at least in part on the data indicative of the success of the one or more second peripheral devices in addressing the second acoustic event.

19. The one or more tangible, non-transitory computer-readable media of claim 18, further comprising:

activating one or more of the first peripheral devices within proximity of the human child.