Noise reduction systems and methods for voice applications

• US 20040213419A1
• Filed: 04/25/2003
• Published: 10/28/2004
• Est. Priority Date: 04/25/2003
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Florencio, Dinei, Varma, Ankur
• Granted Patent: US 7,519,186 B2
• Time in Patent Office: Days
• US Class Current: 381/92
• CPC Class Codes: A63F 2300/1081 Input via voice recognition
• CPC Class Codes: G10L 2021/02087 the noise being separate speech, e.g. cocktail party
• CPC Class Codes: G10L 2021/02166 Microphone arrays; Beamforming
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04R 3/005 for combining the signals of two or more microphones specially adapted for hearing aids H04R25/407

Abstract

Various embodiments reduce noise within a particular environment, while isolating and capturing speech in a manner that allows operation within an otherwise noisy environment. In one embodiment, an array of one or more microphones is used to selectively eliminate noise emanating from known, generally fixed locations, and pass signals from a pre-specified region or regions with reduced distortion.

Claims

1. A method comprising:

providing a computing device having an array of microphones comprising one or more microphones; and

using the microphone array, training the device to recognize noise from known locations by equipping the device with a filter system that can filter noise from the known locations.

2. The method of claim 1, wherein the device comprises a keyboard.

3. The method of claim 1, wherein the device comprises a game controller.

4. The method of claim 1, wherein the device comprises a laptop computer.

5. The method of claim 1, wherein at least some of the known locations are fixed relative to the microphone array.

6. The method of claim 1, wherein at least some of the known locations are located on the device itself.

7. The method of claim 1, wherein at least some of the known locations are not located on the device itself.

8. The method of claim 1, wherein:

at least some of the known locations are located on the device itself; and

at least some of the known locations are not located on the device itself.

9. The method of claim 1, wherein the microphone array does not comprise a headset-mounted microphone.

10. The method of claim 1, wherein the microphone array comprises one or more headset-mounted microphones.

11. A method comprising:

providing a computing device having an array of microphones comprising one or more microphones; and

using the microphone array, training the device to recognize noise from particular known sources by equipping the device with a filter system that can filter noise from the particular known sources.

12. The method of claim 11, wherein the device comprises a keyboard.

13. The method of claim 11, wherein the device comprises a keyboard, and at least some of the sources comprise keys on the keyboard.

14. The method of claim 11, wherein the device comprises a game controller.

15. The method of claim 11, wherein the device comprises a game controller, and at least some of the sources comprise buttons on the controller.

16. The method of claim 11, wherein the device comprises a laptop computer.

17. The method of claim 11, wherein the device comprises a laptop computer, and at least some of the sources comprise keys on the laptop computer.

18. The method of claim 11, wherein at least some of the known sources are fixed relative to the microphone array.

19. The method of claim 11, wherein at least some of the known sources are fixed relative to the microphone array, and at least one source comprises a button.

20. The method of claim 11, wherein at least some of the known sources are located on the device itself.

21. The method of claim 11, wherein at least some of the known sources are not located on the device itself.

22. The method of claim 11, wherein:

at least some of the known sources are located on the device itself; and

at least some of the known sources are not located on the device itself.

23. The method of claim 11, wherein the microphone array does not comprise a headset-mounted microphone.

24. The method of claim 11, wherein the microphone array comprises one or more headset-mounted microphones.

25. The method of claim 11, wherein said training comprises equipping the device with filters associated with individual sources of noise.

26. A method comprising:

providing a game controller having an array of microphones comprising one or more microphones;

using the microphone array, training the game controller to recognize audio signals from particular known locations and sources by equipping the game controller with a filter system that can (a) filter noise from particular known locations and sources, and (b) pass signals associated with desired speech from particular locations.

27. The method of claim 26, wherein at least some of the known locations are fixed relative to the microphone array.

28. The method of claim 26, wherein at least some of the known locations are located on the game controller itself.

29. The method of claim 26, wherein at least some of the known locations are not located on the game controller itself.

30. The method of claim 26, wherein:

at least some of the known locations are located on the game controller itself; and

at least some of the known locations are not located on the game controller itself.

31. The method of claim 26, wherein the noise that the filter system is designed to filter comprises noise associated with button clicks on the game controller.

32. The method of claim 26, wherein the noise that the filter system is designed to filter comprises undesired speech that emanates from particular locations relative to the game controller.

33. The method of claim 26, wherein said training comprises equipping the game controller with at least some filters that are associated with individual sources of noise.

34. The method of claim 26, wherein the microphone array does not comprise a headset-mounted microphone.

35. The method of claim 26, wherein the microphone array comprises one or more headset-mounted microphones.

36. A method comprising:

providing a user-engagable input device comprising a housing that supports an array of microphones, at least one of the microphones being mounted inside of the housing;

using the microphone array, capturing audio signals associated with noise;

correlation processing the audio signals associated with the noise and constructing one or more filter components as a function of the processed audio signals;

using the microphone array, capturing audio signals associated with desired speech;

correlation processing the audio signals associated with desired speech and constructing one or more filter components as a function of the processed audio speech signals; and

incorporating a filter system comprising the filter components into one or more user engagable input devices.

37. The method of claim 36, wherein the user-engagable input device comprises a game controller.

38. The method of claim 36, wherein said filter system comprises one or more spatial filters computed as generalized Wiener filters having the form:

w_opt=(R_ss+β

R_nn)<sup>−

1</sup>(E{ds}),

where R_ss is the correlation matrix for a desired speech signal, R_nn is the correlation matrix for the noise component, β

is a weighting parameter for the noise component, and E{ds} is the expected value of the product of the desired signal d and the actual signal s that is received by a microphone.

39. The method of claim 36, wherein at least some sources and locations of noise are known in advance.

40. The method of claim 36, wherein at least some locations of the desired speech are known in advance.

41. A method comprising:

providing a computing device having an array of microphones comprising one or more microphones, the computing device comprising a trained filter system configured to recognize noise from particular known locations relative to the computing device;

capturing audio signals using the microphone array;

filtering noise from the captured audio signals using the trained filter system.

42. The method of claim 41, wherein the device comprises a keyboard.

43. The method of claim 41, wherein the device comprises a game controller.

44. The method of claim 41, wherein the device comprises a laptop computer.

45. The method of claim 41, wherein at least some of the known locations are fixed relative to the microphone array.

46. The method of claim 41, wherein at least some of the known locations are located on the device itself.

47. The method of claim 41, wherein at least some of the known locations are not located on the device itself.

48. The method of claim 41, wherein:

at least some of the known locations are located on the device itself; and

at least some of the known locations are not located on the device itself.

49. The method of claim 41 further comprising after said filtering, attempting to remove noise from a filtered signal as a function of a ratio of a signal energy before filtering to a signal energy after filtering.

50. The method of claim 41, wherein the microphone array does not comprise a headset-mounted microphone.

51. The method of claim 41, wherein the microphone array comprises one or more headset-mounted microphones.

52. A method comprising:

providing a computing device having an array of microphones comprising one or more microphones, the computing device comprising a trained filter system configured to recognize noise from particular known sources;

capturing audio signals using the microphone array; and

filtering noise from the captured audio signals using the trained filter system.

53. The method of claim 52, wherein the device comprises a keyboard.

54. The method of claim 52, wherein the device comprises a keyboard, and at least some of the sources comprise keys on the keyboard.

55. The method of claim 52, wherein the device comprises a game controller.

56. The method of claim 52, wherein the device comprises a game controller, and at least some of the sources comprise buttons on the controller.

57. The method of claim 52, wherein the device comprises a laptop computer.

58. The method of claim 52, wherein the device comprises a laptop computer, and at least some of the sources comprise keys on the laptop computer.

59. The method of claim 52, wherein at least some of the known sources are fixed relative to the microphone array.

60. The method of claim 52, wherein at least some of the known sources are fixed relative to the microphone array, and at least one source comprises a button.

61. The method of claim 52, wherein at least some of the known sources are located on the device itself.

62. The method of claim 52, wherein at least some of the known sources are not located on the device itself.

63. The method of claim 52, wherein:

at least some of the known sources are located on the device itself; and

at least some of the known sources are not located on the device itself.

64. The method of claim 52 further comprising after said filtering, attempting to remove noise from a filtered signal as a function of a ratio of a signal energy before filtering to a signal energy after filtering.

65. The method of claim 52, wherein the microphone array does not comprise a headset-mounted microphone.

66. The method of claim 52, wherein said filter system comprises one or more filters that are associated with individual sources of noise, and wherein said filtering comprises detecting whether an individual noise source has been engaged by a user and responsively selecting a filter associated with the engaged noise source to filter noise produced by the engaged noise source.

67. A method comprising:

providing a game controller having an array of microphones comprising one or more microphones, the game controller comprising a trained filter system configured to recognize audio signals from particular known locations and sources;

capturing audio signals using the microphone array;

filtering the captured signals using the trained filter system effective to (a) filter noise from particular locations and sources, and (b) pass signals associated with desired speech from particular locations.

68. The method of claim 67, wherein at least some of the known locations are fixed relative to the microphone array.

69. The method of claim 67, wherein at least some of the known locations are located on the game controller itself.

70. The method of claim 67, wherein at least some of the known locations are not located on the game controller itself.

71. The method of claim 67, wherein:

at least some of the known locations are located on the game controller itself; and

at least some of the known locations are not located on the game controller itself.

72. The method of claim 67, wherein the noise that the filter system is designed to filter comprises noise associated with button clicks on the game controller.

73. The method of claim 67, wherein the noise that the filter system is designed to filter comprises undesired speech that emanates from particular locations relative to the game controller.

74. The method of claim 67 further comprising after said filtering, attempting to remove noise from a filtered signal as a function of a ratio of a signal energy before filtering to a signal energy after filtering.

75. The method of claim 67, wherein the microphone array does not comprise a headset-mounted microphone.

76. A method comprising:

providing a user-engagable input device comprising a housing that supports an array of microphones, at least one of the microphones being mounted inside of the housing;

capturing audio signals associated with the environment in which the user-engagable input device is used, wherein the audio signals can comprise both noise and desired speech;

filtering the captured audio signals using a trained filter system that is configured to recognize noise and desired speech, the filter system comprising multiple filters computed as generalized Wiener filters having the form;

w_opt=(R_ss+β

R_nn)<sup>−

1</sup>(E{ds}),

where R_ss is the correlation matrix for a desired speech signal, R_nn is the correlation matrix for the noise component, β

is a weighting parameter for the noise component, and E{ds} is the expected value of the product of the desired signal d and the actual signal s that is received by a microphone.

77. The method of claim 76, wherein the user-engagable input device comprises a game controller.

78. The method of claim 76, wherein at least some sources and locations of noise are known in advance.

79. The method of claim 76, wherein at least some locations of the desired speech are known in advance.

80. The method of claim 76, wherein the filter system is configured to adaptively filter audio signals.

81. The method of claim 76 further comprising after said filtering, attempting to remove noise from a filtered signal as a function of a ratio of a signal energy before filtering to a signal energy after filtering.

82. A system comprising:

a housing;

one or more user input mechanisms supported by the housing;

a processor;

a computer-readable media;

a microphone array at least some of which supported by the housing and comprising one or more microphones;

a noise reduction component comprising a filter system embodied on the computer-readable media, the filter system being trained to recognize noise from particular known locations; and

the noise reduction component being configured to cause the processor to use the trained filter system to filter noise, from said known locations, from audio signals captured by the microphone array.

83. The system of claim 82, wherein the filter system is trained to recognize noise from locations that are fixed relative to the microphone array.

84. The system of claim 82, wherein the filter system is trained to recognize noise from locations that are fixed on the housing.

85. The system of claim 82, wherein the filter system is trained to recognize noise from locations that are not fixed relative to the microphone array.

86. The system of claim 82, wherein the filter system is trained to recognize noise from locations that are both fixed relative to the microphone array, and not fixed relative to the microphone array.

87. The system of claim 82, wherein the processor is supported within the housing.

88. The system of claim 82, wherein the computer-readable media is supported within the housing.

89. The system of claim 82, wherein the processor and the computer-readable media are supported within the housing.

90. A system comprising:

a housing;

one or more user input mechanisms supported by the housing;

a processor;

a computer-readable media;

a microphone array comprising one or more microphones;

a noise reduction component comprising a filter system embodied on the computer-readable media, the filter system being trained to recognize noise from particular known sources; and

the noise reduction component being configured to cause the processor to use the trained filter system to filter noise, from said known sources, from audio signals captured by the microphone array.

91. The system of claim 90, wherein at least some of the sources are fixed relative to the microphone array.

92. The system of claim 90, wherein at least some of the sources are located on the housing.

93. The system of claim 90, wherein at least some of the sources are not located on the housing.

94. The system of claim 90, wherein at least some of the sources are not located on the housing, and at least one source that is not on the housing comprises speech.

95. The system of claim 90, wherein at least some of the sources are located on the housing, and at least some of the sources are not located on the housing.

96. A system comprising:

a housing;

one or more user input mechanisms supported by the housing;

a processor;

a computer-readable media;

a microphone array comprising one or more microphones, at least one of the microphones being mounted within the housing;

a noise reduction component comprising a filter system embodied on the computer-readable media, the filter system being trained to recognize audio signals from particular known sources and locations; and

the noise reduction component being configured to cause the processor to use the trained filter system to (a) filter noise, from said known sources and locations, from audio signals captured by the microphone array, and (b) pass signals associated with desired speech from particular locations.

97. The system of claim 96, wherein the filter system is trained to recognize noise from locations that are fixed relative to the microphone array.

98. The system of claim 96, wherein the filter system is trained to recognize noise from locations that are fixed on the housing.

99. The system of claim 96, wherein the filter system is trained to recognize noise from locations that are not fixed relative to the microphone array.

100. The system of claim 96, wherein the filter system is trained to recognize noise from locations that are not fixed relative to the microphone array, and at least some of the noise from locations that are not fixed relative to the microphone array comprises speech.

101. The system of claim 96, wherein the filter system is trained to recognize noise that emanates from one or more of the user input mechanisms.

102. The system of claim 96, wherein the filter system is trained to recognize noise from sources mounted on and contained within the housing.

103. A noise reduction component comprising:

a transform component configured to transform audio samples from a microphone array from the time domain into the frequency domain;

a filter system associated with the transform component and configured to filter frequency samples produced by the transform component, the filter system comprising multiple filters each of which being associated with a frequency bin, individual filters comprising a generalized Wiener filter having the form;

w_opt=(R_ss+β

R_nn)<sup>−

1</sup>(E{ds}),

where R_ss is the correlation matrix for a desired speech signal, R_nn is the correlation matrix for a noise component, β

is a weighting parameter for the noise component, and E{ds} is the expected value of the product of the desired signal d and the actual signal s that is received by a microphone.

104. The noise reduction component of claim 103, wherein the transform component comprises a Modulated Complex Lapped Transform (MCLT).

105. The noise reduction component of claim 103, wherein at least some sources and locations of noise are known in advance.

106. The noise reduction component of claim 103, wherein at least some locations of the desired speech are known in advance.

107. The noise reduction component of claim 103, wherein at least some sources and locations of noise are known in advance, and at least some locations of the desired speech are known in advance.

108. The noise reduction component of claim 103, wherein the filter system is configured to adaptively filter audio signals.

109. A device embodying the noise reduction component of claim 103.

110. A game controller embodying the noise reduction component of claim 103.

111. The noise reduction component of claim 103 further comprising an energy ratio component configured to receive a filtered output from the filter system and process the filtered output to attempt to further remove noise from the signal as a function of the energy of the samples before filtering by the filter system and the energy of the samples after filtering by the filter system.