Removing noise from audio

• US 8,265,292 B2
• Filed: 09/30/2011
• Issued: 09/11/2012
• Est. Priority Date: 06/30/2010
• Status: Active Grant

• Current Assignee: Google LLC (Alphabet Inc.)
• Original Assignee: Google Inc. (Alphabet Inc.)
• Inventors: Leichter, Jerrold
• Primary Examiner(s): Chin, Vivian
• Assistant Examiner(s): SUTHERS, DOUGLAS JOHN
• Application Number:
  US13/250,528
• Publication Number:
  US 20120020490A1
• Time in Patent Office: 347 Days
• Field of Search: 381/73.1, 381/94.1, 381/71.1, 381/71.2, 381/56, 381/71.8, 381/71.9, 381/71.14
• US Class Current: 381/71.1
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04R 1/1083 Reduction of ambient noise active noise reduction per se G10K11/175; protective devices for the ear, e.g. providing acoustic protection A61F11/06
• CPC Class Codes: H04R 2227/003 Digital PA systems using, e.g. LAN or internet
• CPC Class Codes: H04R 27/00 Public address systems circuits for preventing acoustic reaction H04R3/02; circuits for distributing signals to loudspeakers H04R3/12; monitoring or testing arrangements for public address systems H04R29/007; amplifiers H03F
• CPC Class Codes: H04R 3/02 for preventing acoustic reaction , i.e. acoustic oscillatory feedback specially adapted for hearing aids H04R25/453

Abstract

The subject matter of this specification can be embodied in, among other things, a computer-implemented method for removing noise from audio that includes building a sound model that represents noises which result from activations of input controls of a computer device. The method further includes receiving an audio signal produced from a microphone substantially near the computer device. The method further includes identifying, without using the microphone, an activation of at least one input control from among the input controls. The method further includes associating a portion of the audio signal as corresponding to the identified activation. The method further includes applying, from the audio model, a representation of a noise for the identified activation to the associated portion of the audio signal so as to cancel at least part of the noise from the audio signal.

Claims

1. A computer-implemented method for removing noise from audio, the method comprising:

building a sound model that represents noises which result from activations of input controls of a computer device;

receiving an audio signal produced from a microphone substantially near the computer device;

identifying, without using the microphone, an activation of at least one input control from among the input controls;

associating a portion of the audio signal as corresponding to the identified activation; and

applying, from the sound model, a representation of a noise for the identified activation to the associated portion of the audio signal so as to cancel at least part of the noise from the audio signal, wherein;

the input controls include keys on a keyboard, the activations include physical actuations of the keys on the keyboard, and identifying the activation includes receiving a software event for the activation,

the noises include audible sounds that result from the physical actuations of the keys,

building the model comprises obtaining, through the microphone, the audible sounds of the physical actuations of the keys, and

obtaining the audible sounds of the physical actuations of the keys occurs as a background operation for training the computer device while one or more other operations are performed that use the keys.

2. The method of claim 1, wherein the microphone is mounted to the computer device.

3. The method of claim 1, wherein the model defines the audible sounds of the physical actuations of the keys by frequency and duration.

4. The method of claim 1, wherein the noise includes electromagnetic noise.

5. The method of claim 1, further comprising sending the audio signal with the part of the noise cancelled over a network for receipt by participants in a teleconference.

6. The method of claim 1, wherein associating the portion of the audio signal as corresponding to the identified activation includes correlating timing of receiving the portion and of receiving the activation.

7. The method of claim 6, further comprising automatically calibrating the computer device to determine an amount of time between receiving the portion and receiving the activation.

8. A computer-implemented method for removing noise from audio, the method comprising:

building a sound model that represents noises which result from activations of input controls of a computer device;

receiving an audio signal produced from a microphone substantially near the computer device;

identifying, without using the microphone, an activation of at least one input control from among the input controls;

associating a portion of the audio signal as corresponding to the identified activation; and

applying, from the sound model, a representation of a noise for the identified activation to the associated portion of the audio signal so as to cancel at least part of the noise from the audio signal, wherein;

the input controls include keys on a keyboard, the activations include physical actuations of the keys on the keyboard, and identifying the activation includes receiving a software event for the activation,

the noises include audible sounds that result from the physical actuations of the keys,

building the model comprises obtaining, through the microphone, the audible sounds of the physical actuations of the keys, and

building the model includes receiving the obtained audible sounds of the physical actuations of the keys at a server system that is remote from the computer device.

9. The method of claim 8, further comprising receiving the audio signal and data representing timing of the activation of the key on the computer device at the server system.

10. The method of claim 8, wherein the model defines the audible sounds of the physical actuations of the keys by frequency and duration.

11. The method of claim 8, further comprising sending the audio signal with the part of the noise removed over a network for receipt by participants in a teleconference.

12. The method of claim 8, wherein associating the portion of the audio signal as corresponding to the identified activation includes correlating timing of receiving the portion and of receiving the activation.

13. The method of claim 12, further comprising automatically calibrating the computer device to determine an amount of time between receiving the portion and receiving the activation.

14. One or more non-transitory computer-readable media storing instructions that, when executed by one or more processing devices, cause a computing system to perform operations, the operations comprising:

building a sound model that represents noises which result from activations of input controls of a computer device;

receiving an audio signal produced from a microphone substantially near the computer device;

identifying, without using the microphone, an activation of at least one input control from among the input controls;

associating a portion of the audio signal as corresponding to the identified activation; and

applying, from the sound model, a representation of a noise for the identified activation to the associated portion of the audio signal so as to cancel at least part of the noise from the audio signal, wherein

the input controls include keys on a keyboard, the activations include physical actuations of the keys on the keyboard, and identifying the activation includes receiving a software event for the activation,

the noises include audible sounds that result from the physical actuations of the keys,

building the model comprises obtaining, through the microphone, the audible sounds of the physical actuations of the keys,

obtaining the audible sounds of the physical actuations of the keys occurs as a background operation for training the computer device while one or more other operations are performed that use the keys.

15. The one or more computer-readable media of claim 14, wherein the microphone is mounted to the computer device.

16. The one or more computer-readable media of claim 14, wherein the model defines the audible sounds of the physical actuations of the keys by frequency and duration.

17. The one or more computer-readable media of claim 14, wherein the noise includes electromagnetic noise.

18. The one or more computer-readable media of claim 14, the operations further comprising sending the audio signal with the part of the noise removed over a network for receipt by participants in a teleconference.

19. The one or more computer-readable media of claim 14, wherein associating the portion of the audio signal as corresponding to the identified activation includes correlating timing of receiving the portion and of receiving the activation.

20. The one or more computer-readable media of claim 19, the operations further comprising automatically calibrating the computer device to determine an amount of time between receiving the portion and receiving the activation.

21. A computer-implemented system for removing noise during a teleconference, the system comprising:

one or more processing devices;

one or more non-transitory computer-readable media;

a sound model, stored by the one or more non-transitory computer-readable media, generated to define noises which result from input controls being activated on a computer device, wherein;

the input controls include keys on a keyboard of the computing device,

activation of the input controls includes physical actuation of the keys on the keyboard,

the noises include audible sounds that result from the physical actuations of the keys,

the sound model was built by obtaining, through a microphone, the audible sounds of the physical actuations of the keys, and

the sound model was built by receiving the obtained audible sounds of the physical actuations of the keys at a server system that is remote from the computer device;

an interface, executable by the one or more processing devices, to receive first data that reflects electrical activation of the input controls and second data that reflects an audio signal received by the microphone in communication with the computer device, wherein at least a portion of the audio signal includes one or more of the noises which result from activation of the input controls on the computer device, wherein receiving the first data that reflects electrical activation includes receiving a software event for the electrical activation; and

a noise cancellation module, executable by the one or more processing devices, programmed to correlate the first data with the second data and to use representations of the one or more noises from the sound model to cancel the one or more noises from the portion of the audio signal received from the microphone.

22. The system of claim 21, wherein the microphone is mounted to the computer device.

23. The system of claim 21, wherein the audio signal and data representing timing of the activation of the key on the computer device was received at the server system.