Translation and capture architecture for output of conversational utterances

• US 20060293874A1
• Filed: 06/27/2005
• Published: 12/28/2006
• Est. Priority Date: 06/27/2005
• Status: Active Grant

• Current Assignee: Microsoft Technology Licensing LLC (Microsoft Corporation)
• Original Assignee: Microsoft Corporation
• Inventors: Kong, Yuan, Zhang, Zhengyou, Liu, Zicheng, Williams, David W.
• Granted Patent: US 7,991,607 B2
• Time in Patent Office: Days
• US Class Current: 704/2
• CPC Class Codes: G06F 18/256 of results relating to different input data, e.g. multimodal recognition
• CPC Class Codes: G06F 40/58 Use of machine translation, e.g. for multi-lingual retrieval, for server-side translation for client devices or for real-time translation
• CPC Class Codes: G06V 10/811 the classifiers operating on different input data, e.g. multi-modal recognition
• CPC Class Codes: G06V 40/20 Movements or behaviour, e.g. gesture recognition recognition of facial expressions G06V40/16
• CPC Class Codes: G10L 15/1822 Parsing for meaning understanding

Abstract

Architecture that combines capture and translation of concepts, goals, needs, locations, objects, locations, and items (e.g., sign text) into complete conversational utterances that take a translation of the item, and morph it with fluidity into sets of sentences that can be echoed to a user, and that the user can select to communicate speech (or textual utterances). A plurality of modalities that process images, audio, video, searches and cultural context, for example, which are representative of at least context and/or content, and can be employed to glean additional information regarding a communications exchange to facilitate more accurate and efficient translation. Gesture recognition can be utilized to enhance input recognition, urgency, and/or emotional interaction, for example. Speech can be used for document annotation. Moreover, translation (e.g., speech to speech, text to speech, speech to text, handwriting to speech, text or audio, . . . ) can be significantly improved in combination with this architecture.

Claims

1. A system that facilitates data translation, comprising:

an input component that processes input data received from a plurality of sources that represent at least one of context and content; and

a translation component that translates the processed data into a translated output which includes at least one of text and audible signals for perception by a recipient.

2. The system of claim 1, wherein the translated output includes audible signals that are understandable utterances to the recipient.

3. The system of claim 1, wherein the translated output includes text that is understandable by the recipient.

4. The system of claim 1, wherein the input component outputs image data to the translation component that facilitates determination of the context.

5. The system of claim 1, wherein the input component receives speech signals from a user in a first language which are translated into a second language for understandable perception of the recipient.

6. The system of claim 5, wherein the second language is presented understandably as text to the recipient via a display.

7. The system of claim 5, wherein the second language is presented understandably as speech to the recipient via an audio system.

8. The system of claim 1, further comprising an assembler component that assembles translated words into sentences which are understandable by the recipient.

9. The system of claim 1, further comprising a feedback component that facilitates feedback from a user which increases understandability of the translated output by the recipient.

10. The system of claim 1, wherein the sources of the input component are user selectable in order to form the translated output.

11. A computer-readable medium having stored thereon computer-executable instructions for carrying out the system of claim 1.

12. A computer-implemented method of translating data, comprising:

receiving different input data types from two or more sensing sources;

generating at least one of context data and content data from the input data types that are representative of context and content of a user;

translating the at least one of context data and content data into a translated output; and

presenting the translated output to a recipient in a language that is understandable by the recipient.

13. The method of claim 12, further comprising an act of presenting the translated output to the user to receive user feedback.

14. The method of claim 12, further comprising an act of selecting the translated output to be formatted as audio signals in a form of speech which is understandable by the recipient.

15. The method of claim 12, further comprising an act of assembling words representative of the at least one of context data and content data into sentences understandable by the recipient.

16. The method of claim 12, further comprising an act of assembling words representative of the at least one of context data and content data into sentences understandable by the user.

17. The method of claim 12, further comprising an act of searching at least one of a local datastore and a network-based resource for contextual information that indicates the context.

18. The method of claim 12, further comprising an act of imaging text of the context that is processed, in part, to determine the translated output.

19. A system that facilitates data translation, comprising:

means for sensing at least one of context data and content data of a user and a recipient;

means for processing the at least one of context data and content data into word guesses of an understandable language;

means for translating the word guesses into a translated output; and

means for presenting the translated output to the recipient in a recipient language that is understandable by the recipient.

20. The system of claim 19, wherein the understandable language is understandable by the user, to which the user provides feedback as to accuracy as to user understandability.