Method and apparatus for multi-sensory speech enhancement on a mobile device

• US 20060079291A1
• Filed: 10/12/2004
• Published: 04/13/2006
• Est. Priority Date: 10/12/2004
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Sinclair, Michael J., Zhang, Zhengyou, Liu, Zicheng, Granovetter, Randy Phyllis
• Granted Patent: US 7,283,850 B2
• Time in Patent Office: Days
• US Class Current: 455/563
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G10L 15/20 Speech recognition techniques specially adapted for robustness in adverse environments, e.g. in noise, of stress induced speech G10L21/02 takes precedence
• CPC Class Codes: G10L 15/24 Speech recognition using non-acoustical features
• CPC Class Codes: G10L 2021/02161 Number of inputs available containing the signal or the noise to be suppressed
• CPC Class Codes: G10L 21/0208 Noise filtering
• CPC Class Codes: H04M 1/05 specially adapted for use on head, throat or breast
• CPC Class Codes: H04M 1/6008 in the transmitter circuit
• CPC Class Codes: H04M 1/605 involving control of the receiver volume to provide a dual operational mode at close or far distance from the user
• CPC Class Codes: H04M 1/6066 including a wireless connection
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces
• CPC Class Codes: H04M 2250/02 including a Bluetooth interface
• CPC Class Codes: H04M 2250/06 including a wireless LAN interface
• CPC Class Codes: H04M 2250/10 including a GPS signal receiver
• CPC Class Codes: H04R 1/083 Special constructions of mouthpieces
• CPC Class Codes: H04W 88/02 Terminal devices
• CPC Class Codes: H04W 92/18 between terminal devices

Abstract

A mobile device includes an air conduction microphone and an alternative sensor that provides an alternative sensor signal indicative of speech. A communication interface permits the mobile device to communicate directly with other mobile devices.

Claims

1. A mobile device comprising:

an air conduction microphone that converts acoustic waves into an electric microphone signal;

an alternative sensor that provides an electric alternative sensor signal comprising information about speech; and

a communication interface that permits the mobile device to communicate directly with other mobile devices.

2. The mobile device of claim 1 wherein the communication interface permits multiple types of communication with other mobile devices.

3. The mobile device of claim 2 wherein one of the multiple types of communication comprises one-to-one communication with one other mobile device even when more than one other mobile device is available for communication.

4. The mobile device of claim 2 wherein one of the multiple types of communication comprises one-to-many communication with a plurality of other mobile devices.

5. The mobile device of claim 2 wherein on of the multiple types of communication comprises many-to-many communication.

6. The mobile device of claim 1 wherein the communication interface further permits communication to a network.

7. The mobile device of claim 6 wherein the communication interface further permits communication with a server on the network.

8. The mobile device of claim 7 wherein the communication with the server comprises audio and video.

9. The mobile device of claim 8 wherein communication with the server further comprises data corresponding to an image of a document that is shared with another computing device.

10. The mobile device of claim 9 wherein communication with the server further comprises instructions to modify the document and wherein based on the instructions, the server modifies the document and provides data corresponding to the modifications of the document to the other computing device.

11. The mobile device of claim 7 wherein the mobile device further comprises a global positioning satellite receiver that is capable of generating a position value indicative of the position of the mobile device.

12. The mobile device of claim 11 wherein communication with a server comprises providing the position value to the server.

13. The mobile device of claim 7 wherein the server hosts a speech recognition program and wherein communicating with the server comprises providing audio data to the server and receiving recognized text from the server.

14. The mobile device of claim 7 wherein the server hosts a translation program and wherein communicating with the server comprises providing content in one language to the server and receiving a translation in another language from the server.

15. The mobile device of claim 1 wherein the mobile device comprises a cellular phone.

16. The mobile device of claim 1 wherein the mobile device comprises a personal digital assistant.

17. The mobile device of claim 1 wherein the communication interface further permits direct communication to another device.

18. A method in a mobile device, the method comprising:

receiving an air conduction microphone signal;

receiving an alternative sensor signal that comprises information about speech;

estimating an enhanced clean speech value based on the air conduction microphone signal and the alternative sensor signal; and

providing the enhanced clean speech value directly to another mobile device.

19. The method of claim 18 wherein providing the enhanced clean speech value directly to another mobile device comprises providing the enhanced clean speech value during a one-to-one communication between the mobile device and the other mobile device.

20. The method of claim 18 wherein providing the enhanced clean speech value directly to another mobile device comprises providing the enhanced clean speech value during a one-to-many communication between the mobile device and a plurality of other mobile devices.

21. The method of claim 18 wherein providing the enhanced clean speech value directly to another mobile device comprises providing the enhanced clean speech value during a many-to-many communication between a plurality of mobile devices.

22. The method of claim 18 further comprising communicating between the mobile device and a network of computing devices.

23. The method of claim 22 further comprising communicating between the mobile device and a server on the network of computing devices.

24. The method of claim 23 wherein communicating between the mobile device and the server comprises communicating instant messages.

25. The method of claim 23 wherein communicating between the mobile device and the server comprises communicating information that is shared with another computing device that is in communication with the server.

26. The method of claim 25 wherein the information that is shared comprises changes to an object displayed on the mobile device and the other computing device.

27. The method of claim 26 wherein the information that is shared further comprises audio information.

28. The method of claim 27 wherein the information that is shared further comprises video information.

29. The method of claim 23 further comprising determining a position for the mobile device and communicating the position of the mobile device from the mobile device to the server.

30. The method of claim 29 further comprising using the position of the mobile device to generate a map and communicating the map from the server to another computing device.

31. The method of claim 23 further comprising using a translation service on the server to translate text from one language to another language based on data sent from the mobile device to the server.

32. The method of claim 31 further comprising returning values representing an audio signal from the server to the mobile device, the audio signal representing the translation of the text.

33. The method of claim 31 further comprising returning text from the server to the mobile device, the text representing the translation of the text.

34. The method of claim 18 further comprising communicating between the mobile device and another device.