Removing noise from audio

• US 8,411,874 B2
• Filed: 06/30/2010
• Issued: 04/02/2013
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Leichter, Jerrold
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): SUTHERS, DOUGLAS JOHN
• Application Number:
  US12/827,487
• Publication Number:
  US 20120002820A1
• Time in Patent Office: 1,007 Days
• Field of Search: 381/73.1, 381/94.1, 381/71.1, 381/71.2, 381/56, 381/71.8, 381/71.9, 381/71.14
• US Class Current: 381/71.8
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/02 for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453

Abstract

The subject matter of this specification can be embodied in, among other things, a computer-implemented method for removing noise from audio that includes building a sound model that represents noises which result from activations of input controls of a computer device. The method further includes receiving an audio signal produced from a microphone substantially near the computer device. The method further includes identifying, without using the microphone, an activation of at least one input control from among the input controls. The method further includes associating a portion of the audio signal as corresponding to the identified activation. The method further includes applying, from the audio model, a representation of a noise for the identified activation to the associated portion of the audio signal so as to cancel at least part of the noise from the audio signal.

Claims

1. A computer-implemented method for filtering noise from audio, the method comprising:

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model was generated by aggregating multiple signals that encode audio that resulted from multiple different respective activations of the first input control;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

providing, by the computing system, the first filtered signal to the first computing device for output by a speaker of the first computing device;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

2. The computer-implemented method of claim 1, further comprising:

providing, by the computing system, the first filtered signal to a third computing device for output by a speaker of the third computing device, the third computing device being different than the computing system and the first computing device.

3. The computer-implemented method of claim 1, further comprising generating, by the computing system, the selected first sound model by averaging multiple waveforms that correspond to activation of the first input control at multiple different respective computing devices that are different than the computing system.

4. The computer-implemented method of claim 1, further comprising:

adding, by the computing system, the first filtered signal to other filtered signals so as to generate a single filtered signal, the other filtered signals having been generated by the computing system by applying particular ones of the multiple sound models to multiple different respective received signals that encode audio obtained by microphones of multiple different respective computing devices that are different than the computing system; and

forwarding the single filtered signal for receipt by a particular computing device that is different than the computing system.

5. The computer-implemented method of claim 1, further comprising periodically recalculating, by the computing system, the selected first sound model that corresponds to the first input control.

6. The computer-implemented method of claim 1, wherein the first audio was obtained by the microphone of the first computing device while a user of the first computing device typed content into a word processor application.

7. The computer-implemented method of claim 1, further comprising receiving, by the computing system, a first indication of user activation with the first input control.

8. A computer-implemented method for filtering noise from audio, the method comprising:

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

receiving, by the computing system, an indication of a type of the first computing device;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model is selected to be specific to the type of the first computing device, and at least a plurality of the multiple sound models are specific to multiple different respective types of computing devices that are different than the computing system;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

9. A computer-implemented method for filtering noise from audio, the method comprising:

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model was generated as a result of a training process in which a block of text was displayed on a screen and a user was prompted to type the block of text;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

10. A computer-implemented method for filtering noise from audio, the method comprising:

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

receiving, by the computing system, a first indication of user activation with a first input control;

receiving, by the computing system, a first indication of a time that the user activation with the first input control occurred relative to the first signal that encodes the first audio;

selecting, by the computing system, a first sound model that corresponds to the first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

11. The computer-implemented method of claim 10, wherein the indication of the time includes an indication of an amount of time between (a) the first computing device identifying the user activation with the first input control and (b) the first computing device encoding the first audio in the first signal.

12. A computer-implemented method, the method comprising:

receiving, by a computing system, multiple signals that encode audio obtained by microphones of multiple respective computing devices that are different than the computing system and that represent sounds generated by user activations of a type of input control at the multiple respective computing devices;

generating, by the computing system, a sound model by combining the multiple signals that represent the sounds generated by the user activations of the type of input control at the multiple respective computing devices; and

storing the sound model for application to a particular signal that encodes audio, in order to filter, from the particular signal, audio of a user activation of the type of input control.

13. The computer-implemented method of claim 12, further comprising:

receiving, by the computing system, the particular signal that encodes the audio from a particular computing device that is different than the computing system, the audio having been obtained by a microphone of the particular computing device, the particular signal representing sound generated by user activation of the type of input control at the particular computing device; and

applying, by the computing system, the stored sound model to the particular signal to filter, from the particular signal, audio of user activation with the type of input control so as to generate a filtered signal.

14. The computer-implemented method of claim 13, further comprising providing the filtered signal to the particular computing device.

15. The computer-implemented method of claim 12, wherein combining the multiple signals comprises averaging the multiple signals.

16. A computer-implemented method for filtering noise from audio, the method comprising:

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model was generated by aggregating multiple waveforms that correspond to activation of the first input control at multiple different respective computing devices that are different than the computing system;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

17. A system, comprising:

a processor; and

a computer-readable device including instructions that, when executed by the processor, cause performance of a method that comprises;

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model was generated by aggregating multiple signals that encode audio that resulted from multiple different respective activations of the first input control;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

providing, by the computing system, the first filtered signal to the first computing device for output by a speaker of the first computing device;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

18. The system of claim 17, wherein the method further comprises:

providing, by the computing system, the first filtered signal to a third computing device for output by a speaker of the third computing device, the third computing device being different than the computing system and the first computing device.

19. The system of claim 17, wherein the method further comprises:

generating, by the computing system, the selected first sound model by averaging multiple waveforms that correspond to activation of the first input control at multiple different respective computing devices that are different than the computing system.

20. The system of claim 17, wherein the method further comprises:

adding, by the computing system, the first filtered signal to other filtered signals so as to generate a single filtered signal, the other filtered signals having been generated by the computing system by applying particular ones of the multiple sound models to multiple different respective received signals that encode audio obtained by microphones of multiple different respective computing devices that are different than the computing system; and

forwarding the single filtered signal for receipt by a particular computing device that is different than the computing system.

21. The system of claim 17, wherein the method further comprises:

periodically recalculating, by the computing system, the selected first sound model that corresponds to the first input control.

22. The system of claim 17, wherein the first audio was obtained by the microphone of the first computing device while a user of the first computing device typed content into a word processor application.

23. The system of claim 17, wherein the method further comprises:

receiving, by the computing system, a first indication of user activation with the first input control.

24. A system, comprising:

a processor; and

a computer-readable device including instructions that, when executed by the processor, cause performance of a method that comprises;

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

receiving, by the computing system, an indication of a type of the first computing device;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model is selected to be specific to the type of the first computing device, and at least a plurality of the multiple sound models are specific to multiple different respective types of computing devices that are different than the computing system;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

25. A system, comprising:

a processor; and

a computer-readable device including instructions that, when executed by the processor, cause performance of a method that comprises;

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model was generated as a result of a training process in which a block of text was displayed on a screen and a user was prompted to type the block of text;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

26. A system, comprising:

a processor; and

a computer-readable device including instructions that, when executed by the processor, cause performance of a method that comprises;

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

receiving, by the computing system, a first indication of user activation with a first input control;

receiving, by the computing system, a first indication of a time that the user activation with the first input control occurred relative to the first signal that encodes the first audio;

selecting, by the computing system, a first sound model that corresponds to the first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.

27. The system of claim 26, wherein the indication of the time includes an indication of an amount of time between (a) the first computing device identifying the user activation with the first input control and (b) the first computing device encoding the first audio in the first signal.

28. A system, comprising:

a processor; and

a computer-readable device including instructions that, when executed by the processor, cause performance of a method that comprises;

receiving, by a computing system, multiple signals that encode audio obtained by microphones of multiple respective computing devices that are different than the computing system and that represent sounds generated by user activations of a type of input control at the multiple respective computing devices;

generating, by the computing system, a sound model by combining the multiple signals that represent the sounds generated by the user activations of the type of input control at the multiple respective computing devices; and

storing the sound model for application to a particular signal that encodes audio, in order to filter, from the particular signal, audio of a user activation of the type of input control.

29. The system of claim 28, wherein the method further comprises:

receiving, by the computing system, the particular signal that encodes the audio from a particular computing device that is different than the computing system, the audio having been obtained by a microphone of the particular computing device, the particular signal representing sound generated by user activation of the type of input control at the particular computing device; and

applying, by the computing system, the stored sound model to the particular signal to filter, from the particular signal, audio of user activation with the type of input control so as to generate a filtered signal.

30. The system of claim 29, wherein the method further comprises providing the filtered signal to the particular computing device.

31. The system of claim 28, wherein combining the multiple signals comprises averaging the multiple signals.

32. A system, comprising:

a processor; and

a computer-readable device including instructions that, when executed by the processor, cause performance of a method that comprises;

receiving, by a computing system, a first signal that encodes first audio obtained by a microphone of a first computing device that is different than the computing system;

selecting, by the computing system, a first sound model that corresponds to a first input control of the first computing device, the first sound model selected from among multiple sound models that correspond to multiple respective input controls, wherein the selected first sound model was generated by aggregating multiple waveforms that correspond to activation of the first input control at multiple different respective computing devices that are different than the computing system;

applying, by the computing system, the selected first sound model to the first signal that encodes the first audio, in order to filter, from the first signal, audio of user activation with the first input control so as to generate a first filtered signal;

receiving, by the computing system, a second signal that encodes second audio obtained by a microphone of a second computing device that is different than the computing system and the first computing device;

selecting, by the computing system, a second sound model that corresponds to a second input control of the second computing device, the second sound model selected from among the multiple sound models; and

applying, by the computing system, the selected second sound model to the second signal that encodes the second audio, in order to filter, from the second signal, audio of user activation with the second input control so as to generate a second filtered signal.